LINQ Select analogue for async method

Question

Let's say that I have the following sample code:

private static async Task Main(string[] args)
{
    var result = Enumerable.Range(0, 3).Select(x => TestMethod(x)).ToArray();
    Console.ReadKey();
}

private static int TestMethod(int param)
{
    Console.WriteLine($"{param} before");
    Thread.Sleep(50);
    Console.WriteLine($"{param} after");
    return param;
}

The TestMethod will run to completion 3 times, so I'll see 3 pairs of before and after:

0 before
0 after
1 before
1 after
2 before
2 after

Now, I need to make TestMethod asynchronous:

private static async Task<int> TestMethod(int param)
{
    Console.WriteLine($"{param} before");
    await Task.Delay(50);
    Console.WriteLine($"{param} after");
    return param;
}

How can I write a similar Select expression for this async method? If I just use an async lambda Enumerable.Range(0, 3).Select(async x => await TestMethod(x)).ToArray();, that won't work because it won't wait for completion, so before parts will be called first:

0 before
1 before
2 before
2 after
0 after
1 after

Note that I don't want to run all 3 calls in parallel - I need them to execute one by one, starting the next one only when the previous one has fully finished and returned the value.

Answer 1

I run into this requirement regularly, and I'm not aware of any built-in solution for addressing it as at C# 7.2. I generally just fall back to using await on each asynchronous operation within a foreach, but you could go for your extension method:

public static class EnumerableExtensions
{
    public static async Task<IEnumerable<TResult>> SelectAsync<TSource, TResult>(
        this IEnumerable<TSource> source,
        Func<TSource, Task<TResult>> asyncSelector)
    {
        var results = new List<TResult>();
        foreach (var item in source)
            results.Add(await asyncSelector(item));
        return results;
    }
}

You would then call await on the SelectAsync:

static async Task Main(string[] args)
{
    var result = (await Enumerable.Range(0, 3).SelectAsync(x => TestMethod(x))).ToArray();
    Console.ReadKey();
}

The disadvantage of this approach is that the SelectAsync is eager, not lazy. C# 8 promises to introduce async streams, which will allow this to be lazy again.

Answer 2

In C# 8 (the next major version as of the time of writing) there will be support for IAsyncEnumrable<T> where you can write async for each loops:

await foreach (var item in GetItemsAsync())
    // Do something to each item in sequence

I expect there will be a Select extension method to do projection but if not then it is not difficult to write your own. You will also be able to create IAsyncEnumrable<T> iterator blocks with yield return.

Answer 3

I guess this is what OP actually want, since I've spent a whole day solving the same question as OP wondered. But I want it to be done in fluent mehtod.

By Martin Liversage's answer, I found the correct way: IAsyncEnumerable and System.Linq.Async pacakge, which are available in C# 8.0, is what I want.

Example:

private static async Task Main(string[] args)
{
    var result = await Enumerable.Range(0, 3)
        .ToAsyncEnumerable()
        .SelectAwait(x => TestMethod(x))
        .ToArrayAsync();
    Console.ReadKey();
}

private static async Task<int> TestMethod(int param)
{
    Console.WriteLine($"{param} before");
    await Task.Delay(50);
    Console.WriteLine($"{param} after");
    return param;
}

Answer 4

You'll have to be aware what an Enumerable object is. An Enumerable object, isn't the enumeration itself. It gives you the possibility to get an Enumerator object. If you've got an Enumerator object, you can ask for the first elements of a sequence, and once you've got an enumerated element you can ask for the next one.

So creating an object that enables you to enumerate over a sequence, has nothing to do with the enumerating itself.

Your Select function only returns an Enumerable object, it does not start the enumeration. The enumeration is started by ToArray.

At it's lowest level, enumeration is done as follows:

IEnumerable<TSource> myEnumerable = ...
IEnumerator<TSource> enumerator = myEnumerable.GetEnumerator();
while (enumerator.MoveNext())
{
     // there is still an element in the enumeration
     TSource currentElement = enumerator.Current;
     Process(currentElement);
}

ToArray will internally call GetEnumerator and MoveNext. So to make your LINQ statement async, you will need a ToArrayAsync.

The code is fairly simple. I'll show you ToListAsync as an extension method, which is a little bit easier to make.

static class EnumerableAsyncExtensions
{
    public static async Task<List<TSource>> ToListAsync<TSource>(
       this IEnumerable<Task<TSource>> source)
    {
        List<TSource> result = new List<TSource>();
        var enumerator = source.GetEnumerator()
        while (enumerator.MoveNext())
        {
            // in baby steps. Feel free to do this in one step
            Task<TSource> current = enumerator.Current;
            TSource awaitedCurrent = await current;
            result.Add(awaitedCurrent);
        }
        return result;
    } 
}

You'll only have to create this once. You can use it for any ToListAsync where you'll have to await for each element:

 var result = Enumerable.Range(0, 3)
     .Select(i => TestMethod(i))
     .ToListAsync();

Note that the return of Select is IEnumerable<Task<int>>: an object that makes it possible to enumerate a sequence of Task<int> objects. In a foreach, every cycle you get a Task<int>

Answer 5

You can use syncronzation primitives

    class Program
    {
        static async Task Main(string[] args)
        {
            var waitHandle = new AutoResetEvent(true);
            var result = Enumerable
                .Range(0, 3)
                .Select(async (int param) => {
                    waitHandle.WaitOne();
                    await TestMethod(param);
                    waitHandle.Set();
                }).ToArray();
            Console.ReadKey();
        }
        private static async Task<int> TestMethod(int param)
        {
            Console.WriteLine($"{param} before");
            await Task.Delay(50);
            Console.WriteLine($"{param} after");
            return param;
        }
    }

Answer 6

Looks like there is no other solution currently (until C# 8.0) except enumerating it manually. I made an extension method for that (returns a List instead of Array as it's simpler):

public static async Task<List<T>> ToListAsync<T>(this IEnumerable<Task<T>> source)
{
    var result = new List<T>();
    foreach (var item in source)
    {
        result.Add(await item);
    }
    return result;
}