Using WebClient to login and download files

Question

I've found different examples of doing this, but haven't been able to get any combination of them to work.

Basically, I have an intranet system that can generate documents from a web link, and I know which ones I want to download. I am able to generate the list of links I want to download, but I run into problems with authenticating to the system within the program. I keep getting a 401 error with this this:

Public Shared Sub DownloadFiles(_tool As Tool)
    Dim links As List(Of String) = GetJiraLinks(_tool)

    Dim downloader As New WebClient

    ' Initialize the client
    Dim reqParm As New Specialized.NameValueCollection

    reqParm.Add("os_username", "user")
    reqParm.Add("os_password", "pass")
    reqParm.Add("os_destination", "/secure/")

    downloader.Credentials = New NetworkCredential("user", "pass")

    Dim uploadLocation As String = My.Settings.jiraLocation & "login.jsp"

    'downloader.Headers.Add("Content-Type", "application/x-www-form-urlencoded")

    Dim responseBytes = downloader.UploadValues(uploadLocation, "POST", reqParm)

    Dim responseBody = (New Text.UTF8Encoding).GetString(responseBytes)

    Dim workingDir As String = CreateWorkingDir()

    For Each link As String In links
        Dim tempUri As New Uri(link)

        Dim localpath As String = workingDir & "\" & System.IO.Path.GetFileName(tempUri.LocalPath)

        downloader.DownloadFile(tempUri, localpath)
    Next
End Sub

Have you used [Fiddler](http://www.fiddler2.com) to look at what's going on? — John Saunders, May 21 '13 at 01:37
Thanks for this. Using this I was able to see that there is a cookie involved somehow. My guess is that the WebClient is not saving this cookie. I found http://stackoverflow.com/questions/2825377/how-can-i-get-the-webclient-to-use-cookies, but it doesn't show how to use it. — Origin, May 21 '13 at 05:52

score 1 · Answer 1 · edited May 23 '17 at 11:49

I figured I would post a fully working example of the solution. Many other posts link to a cookie aware web client (How can I get the WebClient to use Cookies?) but fail to show it in action. Here's mine:

Public Shared Sub DownloadFiles(_tool As Tool)
    Dim links As List(Of String) = GetJiraLinks(_tool)

    Dim downloader As New CookieAwareWebClient

    ' Start by requesting the page.

    Dim loginPage As String = My.Settings.jiraLocation & "login.jsp"

    ' Initialize the client
    Dim reqParm As New Specialized.NameValueCollection

    reqParm.Add("os_username", "user")
    reqParm.Add("os_password", "pass")
    reqParm.Add("os_destination", "/secure/")

    Dim responseBytes = downloader.UploadValues(loginPage, "POST", reqParm)

    Dim responseBody = (New Text.UTF8Encoding).GetString(responseBytes)

    Dim workingDir As String = CreateWorkingDir()

    For Each link As String In links
        Dim tempUri As New Uri(link)

        Dim localpath As String = workingDir & "\" & System.IO.Path.GetFileName(tempUri.LocalPath)

        downloader.DownloadFile(tempUri, localpath)
    Next
End Sub

Using WebClient to login and download files

1 Answers1

Linked