Getting the Response of a Asynchronous HttpWebRequest

Question

Im wondering if theres an easy way to get the response of an async httpwebrequest.

I have already seen this question here but all im trying to do is return the response (which is usually json or xml) in the form of a string to another method where i can then parse it/ deal with it accordingly.

Heres some code:

I have these two static methods here which i think are thread safe as all the params are passed in and there are no shared local variables that the methods use?

public static void MakeAsyncRequest(string url, string contentType)
{
    HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
    request.ContentType = contentType;
    request.Method = WebRequestMethods.Http.Get;
    request.Timeout = 20000;
    request.Proxy = null;

    request.BeginGetResponse(new AsyncCallback(ReadCallback), request);
}

private static void ReadCallback(IAsyncResult asyncResult)
{
    HttpWebRequest request = (HttpWebRequest)asyncResult.AsyncState;
    try
    {
        using (HttpWebResponse response = (HttpWebResponse)request.EndGetResponse(asyncResult))
        {
            Stream responseStream = response.GetResponseStream();
            using (StreamReader sr = new StreamReader(responseStream))
            {
                //Need to return this response 
                string strContent = sr.ReadToEnd();
            }
       }
       manualResetEvent.Set();
    }
    catch (Exception ex)
    {
        throw ex;
   }
}

Answer 1

Assuming the problem is that you're having a hard time getting to the returned content, the easiest path would likely be using async/await if you can use it. Even better would be to switch to HttpClient if you're using .NET 4.5 since it's 'natively' async.

Using .NET 4 and C# 4, you can still use Task to wrap these and make it a bit easier to access the eventual result. For instance, one option would be the below. Note that it has the Main method blocking until the content string is available, but in a 'real' scenario you'd likely pass the task to something else or string another ContinueWith off of it or whatever.

void Main()
{
    var task = MakeAsyncRequest("http://www.google.com", "text/html");
    Console.WriteLine ("Got response of {0}", task.Result);
}

// Define other methods and classes here
public static Task<string> MakeAsyncRequest(string url, string contentType)
{
    HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
    request.ContentType = contentType;
    request.Method = WebRequestMethods.Http.Get;
    request.Timeout = 20000;
    request.Proxy = null;

    Task<WebResponse> task = Task.Factory.FromAsync(
        request.BeginGetResponse,
        asyncResult => request.EndGetResponse(asyncResult),
        (object)null);

    return task.ContinueWith(t => ReadStreamFromResponse(t.Result));
}

private static string ReadStreamFromResponse(WebResponse response)
{
    using (Stream responseStream = response.GetResponseStream())
    using (StreamReader sr = new StreamReader(responseStream))
    {
        //Need to return this response 
        string strContent = sr.ReadToEnd();
        return strContent;
    }
}

Answer 2

"Even better would be to switch to HttpClient if you're using .Net 4.5 since it's 'natively' async." - absolutely right answer by James Manning.

This question was asked about 2 years ago. Now we have .Net Framework 4.5, which provides powerful asynchronous methods. Use HttpClient. Consider the following code:

 async Task<string> HttpGetAsync(string URI)
    {
        try
        {
            HttpClient hc = new HttpClient();
            Task<Stream> result = hc.GetStreamAsync(URI);

            Stream vs = await result;
            StreamReader am = new StreamReader(vs);

            return await am.ReadToEndAsync();
        }
        catch (WebException ex)
        {
            switch (ex.Status)
            {
                case WebExceptionStatus.NameResolutionFailure:
                    MessageBox.Show("domain_not_found", "ERROR",
                    MessageBoxButtons.OK, MessageBoxIcon.Error);
                break;
                    //Catch other exceptions here
            }
        }
    }

To use HttpGetAsync(), make a new method that is "async" too. async is required, because we need to use "await" in GetWebPage() method:

async void GetWebPage(string URI)
        {
            string html = await HttpGetAsync(URI);
            //Do other operations with html code
        }

Now if you want to get web-page HTML source code asynchronously, just call GetWebPage("web-address..."). Even Stream reading is asynchronous.

NOTE: to use HttpClient .Net Framework 4.5 is required. Also you need to add System.Net.Http reference in your project and add also "using System.Net.Http" for easy access.

For further reading how this approach works, visit: http://msdn.microsoft.com/en-us/library/hh191443(v=vs.110).aspx

Use of Async: Async in 4.5: Worth the Await

Answer 3

Once you go async, you can never go back. From there you only really have access to the async's callback. you can ramp up the complexity of this and do some threading & waithandles but that can be rather a painful endeavor.

Technically, you can also sleep the thread when you need to wait for the results, but I don't recommend that, you may as well do a normal http request at that point.

In C# 5 theyhave async/await commands that will make it easier to get the results of the async call to the main thread.

Answer 4

public static async Task<byte[]> GetBytesAsync(string url) {
    var request = (HttpWebRequest)WebRequest.Create(url);
    using (var response = await request.GetResponseAsync())
    using (var content = new MemoryStream())
    using (var responseStream = response.GetResponseStream()) {
        await responseStream.CopyToAsync(content);
        return content.ToArray();
    }
}

public static async Task<string> GetStringAsync(string url) {
    var bytes = await GetBytesAsync(url);
    return Encoding.UTF8.GetString(bytes, 0, bytes.Length);
}