GetStringAsync with ContinueWith and WhenAll -- Not Waiting for all requests to finish

Question

I'm trying to process each individual request as it finishes, which is what would happen in the ContinueWith after the GetStringAsync and then when they've all completed have an additional bit of processing.

However, it seems that the ContinueWith on the WhenAll fires right away. It appears as if the GetStringAsync tasks are faulting, but I can't figure out why.

When I use WaitAll instead of WhenAll and just add my processing after the WaitAll then my requests work just fine. But when I change it to WhenAll it fails.

Here is an example of my code:

using (var client = new HttpClient())
{
    Task.WhenAll(services.Select(x =>
    {
        return client.GetStringAsync(x.Url).ContinueWith(response =>
        {
            Console.WriteLine(response.Result);
        }, TaskContinuationOptions.AttachedToParent);

    }).ToArray())
    .ContinueWith(response => 
    {
        Console.WriteLine("All tasks completed");
    });
}

Paulo Morgado · Answer 1 · 2014-03-11T22:54:50.237

2

You shouldn't use ContinueWith and TaskContinuationOptions.AttachedToParent when using async-await. Use async-await only, instead:

async Task<IEnumerable<string>> SomeMethod(...)
{
    using (var client = new HttpClient())
    {
        var ss = await Task.WhenAll(services.Select(async x =>
        {
            var s = await client.GetStringAsync(x.Url);
            Console.WriteLine(response);
            return s;
        };
        Console.WriteLine("All tasks completed");
        return ss;
    }
}

edited Mar 11 '14 at 22:54

answered Mar 11 '14 at 15:20

Paulo Morgado

14,111
3
31
59

Interesting. I'll have to give this a try. I'm a little new to doing things with `Tasks` so, it's always good to learn the more correct way of doing things. Thanks. – njkremer Mar 11 '14 at 17:41
There, seems to be something off here in your example. You have it with `return s` but if you're awaiting it, isn't `s` a `string` not a `Task`? Also, `ss` would be out of scope since it's inside the `using` block. – njkremer Mar 11 '14 at 18:03
1

@njkremer `async` methods always return the result of the task, not a task itself. As for `ss` being out of scope, yes, it is, but you can and should, just move it into the `using` There's no reason for it to be outside of it. – Servy Mar 11 '14 at 20:41

score 1 · Answer 2 · answered Mar 10 '14 at 22:22

1

Well, I found the issue. I'll leave it here in case anyone else comes along looking for a similar answer. I still needed to await the Task.WhenAll method.

So, the correct code would be:

using (var client = new HttpClient())
{
    await Task.WhenAll(services.Select(x =>
    {
        return client.GetStringAsync(x.Url).ContinueWith(response =>
        {
            Console.WriteLine(response.Result);
        }, TaskContinuationOptions.AttachedToParent);

    }).ToArray())
    .ContinueWith(response => 
    {
        Console.WriteLine("All tasks completed");
    });
}

answered Mar 10 '14 at 22:22

njkremer

269
3
11

That's because the using statement disposes your HttpClient which causes the request to be cancelled. If you get rid of that Using block and make sure the httpClient hangs around then your request will complete correctly. – Darrel Miller Mar 10 '14 at 23:08
You're not actually awaiting `WhenAll`, you're awaiting the `ContinueWith` called on `WhenAll`. You probably *should* just be awaiting `WhenAll`. It'd clean up your code. – Servy Mar 11 '14 at 16:27

score 0 · Answer 3 · edited May 23 '17 at 12:31

I still see a couple issues with your solution:

Drop the using statement - you don't want to dispose HttpClient.
Drop the ContinueWiths - you don't need them if your are using await properly.
The Task.WhenAny approach described in this MSDN article is a somewhat cleaner way to process tasks as they complete.

I would re-write your example like this:

var client = new HttpClient();
var tasks = services.Select(x => client.GetStringAsync(x.Url)).ToList();
while (tasks.Count > 0)
{
    var firstDone = await Task.WhenAny(tasks);
    tasks.Remove(firstDone);
    Console.WriteLine(await firstDone);
}
Console.WriteLine("All tasks completed");

Edit to address comment:

If you need access to the service object as the tasks complete, one way would be to modify tasks to be a list of Task<ObjectWithMoreData> instead of Task<string>. Notice the lambda is marked async so we can await within it:

var client = new HttpClient();
var tasks = services.Select(async x => new 
{
    Service = x,
    ResponseText = await client.GetStringAsync(x.Url)
}).ToList();

while (tasks.Count > 0) 
{
    var firstDone = await Task.WhenAny(tasks);
    tasks.Remove(firstDone);
    var result = await firstDone;
    Console.WriteLine(result.ResponseText);
    // do something with result.Service
}
Console.WriteLine("All tasks completed");

I like this example, but I am wondering, what if I wanted to do something with the `service` object when the `GetStringAsync` returned in the `services` collection? (The `x` in the lambda) — njkremer, Mar 11 '14 at 18:05
Modified answer to provide a possible solution to your comment. — Todd Menier, Mar 11 '14 at 20:26

GetStringAsync with ContinueWith and WhenAll -- Not Waiting for all requests to finish

3 Answers3

Edit to address comment: