Download Image under the Mouse pointer via WebBrowser

Question

I'm navigating to Google Images using a WebBrowser control. The aim is to be able to right click on any image and download and populate a PictureBox background.

I have my own ContextMenuStrip with Copy on it and have disabled the built in context menu.

The issue I am having is that the coordinate returned from CurrentDocument.MouseMove are always relative to the first (top left) image.
So my code works correctly if the Image I want is the very first image on the page, however clicking on any other Images always returns the coordinates of the first image.

It would appear that the coordinates are relative to each Image rather than the page.

Private WithEvents CurrentDocument As HtmlDocument
Dim MousePoint As Point
Dim Ele As HtmlElement

Private Sub Google_covers_Load(sender As Object, e As EventArgs) Handles MyBase.Load
    WebBrowser1.IsWebBrowserContextMenuEnabled = False
    WebBrowser1.ContextMenuStrip = ContextMenuStrip1
End Sub

Private Sub WebBrowser1_Navigated(sender As Object, e As WebBrowserNavigatedEventArgs) Handles WebBrowser1.Navigated
    CurrentDocument = WebBrowser1.Document

End Sub
Private Sub CurrentDocument_MouseMove(sender As Object, e As HtmlElementEventArgs) Handles CurrentDocument.MouseMove
    MousePoint = New Point(e.MousePosition.X, e.MousePosition.Y)
    Me.Text = e.MousePosition.X & " | " & e.MousePosition.Y
End Sub

Private Sub ContextMenuStrip1_Opening(sender As Object, e As System.ComponentModel.CancelEventArgs) Handles ContextMenuStrip1.Opening
    Ele = CurrentDocument.GetElementFromPoint(MousePoint)
    If Ele.TagName = "IMG" Then
        CopyToolStripMenuItem.Visible = True
    Else
        CopyToolStripMenuItem.Visible = False
    End If
End Sub

Private Sub CopyToolStripMenuItem_Click(sender As System.Object, e As System.EventArgs) Handles CopyToolStripMenuItem.Click
    Dim ToImg = Ele.GetAttribute("src")
    mp3_row_edit.PictureBox1.BackgroundImage = New System.Drawing.Bitmap(New IO.MemoryStream(New System.Net.WebClient().DownloadData(ToImg)))
    ToImg = Nothing
End Sub

Read the notes here: [Getting mouse click coordinates in a WebBrowser Document](https://stackoverflow.com/a/54232729/7444103). — Jimi, May 16 '19 at 10:40
Thanks but even using their code it still behaves the same, It's got something to do with how Google Image Search displays it's results, works on other pages. Still looking — Mark, May 16 '19 at 11:28
It works perfectly. What you probably didn't count is that the image is directly inserted in the HTML Document using a Base64 string. There's no `source`. You have to get the string and decode it. See: [Convert.FromBase64String](https://learn.microsoft.com/en-us/dotnet/api/system.convert.frombase64string). — Jimi, May 16 '19 at 11:48
It does not work with this url: https://www.google.com/search?biw=1600&bih=777&tbm=isch&sa=1&ei=5E7dXIj5OuSIjLsPooWsqA4&q=rolling+stones+album+cover — Mark, May 16 '19 at 11:53
Decode yes, but it's always the first image on the page. top left. The coordinates for any other image on google images page always start back at XY zero, even though they could be half way across or down the page. It correctly memory streams via webclient an image, just the wrong image regardless to which image we click on the page. If you say it works then please provide the relevant code so we can see where the issue might be? — Mark, May 16 '19 at 12:32

Jimi · Accepted Answer · 2019-05-16T14:44:22.873

This code allow to use a standard WebBrowser control to navigate to the Google Image search page and select/download an Image with a right-click of the Mouse.

To test it, drop a WebBrowser Control and a FlowLayoutPanel on a Form and navigate to a Google Image search page.

Things to know:

WebBrowser.DocumentCompleted: This event is raised each time one of the Sub-Documents inside a main HtmlDocument page is completed. Thus, it can be raised multiple times. We need to check whether the WebBrowser.ReadyState = WebBrowserReadyState.Complete.
Read these note about this: How to get an HtmlElement value inside Frames/IFrames?
The images in the Google search page can be inserted in the Document in 2 different manners: both using a Base64Encoded string and using the classic src=[URI] format. We need to be ready to get both.
The mouse click position can be espressed in either absolute or relative coordinates, referenced by the e.ClientMousePosition or e.OffsetMousePosition.
Read the notes about this feature here: Getting mouse click coordinates in a WebBrowser Document
The WebBrowser emulation mode can be important. We should use the most recent compatible mode available in the current machine.
Read this answer and apply the modifications needed to have the most recent Internet Explorer mode available: How can I get the WebBrowser control to show modern contents?.

Note that an event handler is wired up when the current Document is completed and is removed when the Browser navigates to another page. This prevents undesired calls to the DocumentCompleted event.

When the current Document is complete, clicking with the right button of the Mouse on an Image, creates a new PictureBox control that is added to a FlowLayouPanel for presentation.

The code in the Mouse click handler (Protected Sub OnHtmlDocumentClick()) detects whether the current image is a Base64Encoded string or an external source URI.
In the first case, it calls Convert.FromBase64String to convert the string into a Byte array, in the second case, it uses a WebClient class to download the Image as a Byte array.

In both cases, the array is then passed to another method (Private Function GetBitmapFromByteArray()) that returns an Image from the array, using Image.FromStream() and a MemoryStream initialized with the Byte array.

The code here is not performing null checks and similar fail-proof tests. It ought to, that's up to you.

Public Class frmBrowser
    Private WebBrowserDocumentEventSet As Boolean = False
    Private base64Pattern As String = "base64,"

    Private Sub frmBrowser_Load(sender As Object, e As EventArgs) Handles MyBase.Load
        WebBrowser1.ScriptErrorsSuppressed = True
        WebBrowser1.IsWebBrowserContextMenuEnabled = False
    End Sub

    Private Sub WebBrowser1_DocumentCompleted(sender As Object, e As WebBrowserDocumentCompletedEventArgs) Handles WebBrowser1.DocumentCompleted
        If WebBrowser1.ReadyState = WebBrowserReadyState.Complete AndAlso WebBrowserDocumentEventSet = False Then
            WebBrowserDocumentEventSet = True
            AddHandler WebBrowser1.Document.MouseDown, AddressOf OnHtmlDocumentClick
        End If
    End Sub

    Protected Sub OnHtmlDocumentClick(sender As Object, e As HtmlElementEventArgs)
        Dim currentImage As Image = Nothing

        If Not (e.MouseButtonsPressed = MouseButtons.Right) Then Return
        Dim source As String = WebBrowser1.Document.GetElementFromPoint(e.ClientMousePosition).GetAttribute("src")

        If source.Contains(base64Pattern) Then
            Dim base64 As String = source.Substring(source.IndexOf(base64Pattern) + base64Pattern.Length)
            currentImage = GetBitmapFromByteArray(Convert.FromBase64String(base64))
        Else
            Using wc As WebClient = New WebClient()
                currentImage = GetBitmapFromByteArray(wc.DownloadData(source))
            End Using
        End If

        Dim p As PictureBox = New PictureBox() With {
            .Image = currentImage,
            .Height = Math.Min(FlowLayoutPanel1.ClientRectangle.Height, FlowLayoutPanel1.ClientRectangle.Width)
            .Width = .Height,
            .SizeMode = PictureBoxSizeMode.Zoom
        }
        FlowLayoutPanel1.Controls.Add(p)
    End Sub

    Private Sub WebBrowser1_Navigating(sender As Object, e As WebBrowserNavigatingEventArgs) Handles WebBrowser1.Navigating
        If WebBrowser1.Document IsNot Nothing Then
            RemoveHandler WebBrowser1.Document.MouseDown, AddressOf OnHtmlDocumentClick
            WebBrowserDocumentEventSet = False
        End If
    End Sub

    Private Function GetBitmapFromByteArray(imageBytes As Byte()) As Image
        Using ms As MemoryStream = New MemoryStream(imageBytes)
            Return DirectCast(Image.FromStream(ms).Clone(), Image)
        End Using
    End Function
End Class

Best answer we ever had. Thank you for your effort it's working great! — Mark, May 16 '19 at 13:53
BTW, if anyone else uses this code and wants to save the image to a file then you need to do it in the GetBitmapFromByteArray function. As the returned image "currentImage" will not write it's contents to disk like you may expect it too. — Mark, May 16 '19 at 17:11
You can use `currentImage` to save an Image to disc. Just insert `currentImage.Save("currentImage.png", ImageFormat.Png)` after `If source.Contains(base64Pattern) Then ( ... ) End If`. It will save the Image in the application's path. You could use the `HtmlElement`'s `ID `attribute to name it. Not that it makes much sense as a name, it's a bunch of random chars, anyway... — Jimi, May 16 '19 at 17:45

Download Image under the Mouse pointer via WebBrowser

1 Answers1

Linked