Actual maximum concurrent tasks of Parallel.ForEachAsync

Question

I would expect this code to take 1 second to execute:

public async void Test()
{
    DateTime start = DateTime.Now;
    await Parallel.ForEachAsync(new int[1000], new ParallelOptions { MaxDegreeOfParallelism = 1000 }, async (i, token) =>
    {
        Thread.Sleep(1000);
    });
    Console.WriteLine("End program: " + (DateTime.Now - start).Seconds + " seconds elapsed.");
}

Instead, it takes 37 seconds on my pc (i7-9700 8-core 8-thread):

End program: 37 seconds elapsed.

I am generating 1000 tasks with MaxDegreeOfParallelism = 1000....why don't they all run simultaneously?

@MichaelSchönbauer I think there is an interesting point here: since all the tasks are only calling `Thread.Sleep`, I would expect them to end almost together after the sleep period. It's true that there is an overhead for launching / putting to sleep / resuming / edning the tasks, but x37 (like the OP reported) seems a lot. Don't you agree ? — wohlstad, May 31 '22 at 12:55
@GuruStron That is not true, even on a 1 core machine, this would be able to finish in 1 second if you use `Thread` instead of `Task`. When you call `Sleep` on a thread, the operating system is not actually going to kill off a hardware core for the duration. — Petrusion, May 31 '22 at 12:58
@wohlstad It is probably because of how `Task`s are scheduled as opposed to `Thread`s, see my answer — Petrusion, May 31 '22 at 12:59
@Petrusion I think you are right (regarding my comment above). — wohlstad, May 31 '22 at 13:00
@GuruStron Your statement is indeed true for code that actually does CPU operations for 1 second, but the question is about `Thread.Sleep`. If you create 1000 threads that all just call `Thread.Sleep(1000)`, and start them all at the same time, they will all be finished after around 1 second. — Petrusion, May 31 '22 at 13:08
@Petrusion `var threads = Enumerable.Range(0, 1000).Select(t => new Thread(() => Thread.Sleep(1000))).ToList(); threads.ForEach(t => t.Start()); threads.ForEach(t => t.Join());` runs for 6 second for me. — Guru Stron, May 31 '22 at 13:14
@GuruStron I think that the `Parallel.ForEachAsync` method is sufficiently different than the `Parallel.ForEach` method, to make this question not an exact duplicate of the [linked](https://stackoverflow.com/questions/5824975/parallel-for-max-threads-and-cores) question. — Theodor Zoulias, May 31 '22 at 13:45
@GuruStron yes, there are similarities because the `Parallel.ForEachAsync` is used here with 100% synchronous delegate, but a valid answer to this question, which is to switch to `await Task.Delay(1000);`, is not a valid answer to the `Parallel.ForEach`-related question. Combining `Parallel.ForEach`+`async` is a bug. Also note that this question specifies intentionally the `ParallelOptions`, while the other question also intentionally doesn't specify the same option. So TBH I would not be inclined to consider them duplicates, even if they referred to the same API. — Theodor Zoulias, May 31 '22 at 14:04

score 3 · Accepted Answer · answered May 31 '22 at 13:42

The Parallel.ForEachAsync method invokes the asynchronous body delegate on ThreadPool threads. Usually this delegate returns a ValueTask quickly, but in your case this is not what happens, because your delegate is not really asynchronous:

async (i, token) => Thread.Sleep(1000);

You are probably getting here a compiler warning, about an async method lacking an await operator. Nevertheless giving a mixed sync/async workload to the Parallel.ForEachAsync method is OK. This method is designed to handle any kind of workload. But if the workload is mostly synchronous, the result might be a saturated ThreadPool.

The ThreadPool is said to be saturated when it has already created the number of threads specified by the SetMinThreads method, which by default is equal to Environment.ProcessorCount, and there is more demand for work to be done. In this case the ThreadPool switches to a conservative algorithm that creates one new thread every second (as of .NET 6). This behavior is not documented precisely, and might change in future .NET versions.

In order to get the behavior that you want, which is to run the delegate for all 1000 inputs in parallel, you'll have to increase the number of threads that the ThreadPool creates instantly on demand:

ThreadPool.SetMinThreads(1000, 1000); // At the start of the program

Some would say that after doing so you won't have a thread pool any more, since a thread pool is meant to be a small pool of reusable threads. But if you don't care about the semantics and just want to get the job done, whatever the consequences are regarding memory consumption and overhead at the operating system level, that's the easiest way to solve your problem.

The SetMinThreads method solves the problem. Great explanation, thank you. — Davide Briscese, May 31 '22 at 17:38

score 2 · Answer 2 · answered May 31 '22 at 12:56

I do not know the exact implementation of ForEachAsync, but I assume that they use Tasks, not Threads.

When you use 1000 Tasks to run 1000 CPU bound operations, you are not actually creating 1000 Threads, you are just asking a handful of ThreadPool Threads to run those operations. Those Threads are blocked by the Sleep calls, so most of the Tasks are queued up before they can start execution.

This is exactly why it is a horrible idea to call Thread.Sleep in a Task, or in async contexts in general. If you edit your code to wait asynchronously instead of synchronously, the time elapsed will probably be much closer to a second.

await Parallel.ForEachAsync(new int[1000], new ParallelOptions { MaxDegreeOfParallelism = 1000 }, async (i, token) =>
{
    await Task.Delay(1000);
});

Actual maximum concurrent tasks of Parallel.ForEachAsync

2 Answers2

Linked