Using Task.Yield to overcome ThreadPool starvation while implementing producer/consumer pattern

Question

Answering the question: Task.Yield - real usages? I proposed to use Task.Yield allowing a pool thread to be reused by other tasks. In such pattern:

  CancellationTokenSource cts;
  void Start()
  {
        cts = new CancellationTokenSource();

        // run async operation
        var task = Task.Run(() => SomeWork(cts.Token), cts.Token);
        // wait for completion
        // after the completion handle the result/ cancellation/ errors
    }

    async Task<int> SomeWork(CancellationToken cancellationToken)
    {
        int result = 0;

        bool loopAgain = true;
        while (loopAgain)
        {
            // do something ... means a substantial work or a micro batch here - not processing a single byte

            loopAgain = /* check for loop end && */  cancellationToken.IsCancellationRequested;
            if (loopAgain) {
                // reschedule  the task to the threadpool and free this thread for other waiting tasks
                await Task.Yield();
            }
        }
        cancellationToken.ThrowIfCancellationRequested();
        return result;
    }

    void Cancel()
    {
        // request cancelation
        cts.Cancel();
    }

But one user wrote

I don't think using Task.Yield to overcome ThreadPool starvation while implementing producer/consumer pattern is a good idea. I suggest you ask a separate question if you want to go into details as to why.

Anybody knows, why is not a good idea?

Answer 1

There are some good points left in the comments to your question. Being the user you quoted, I'd just like to sum it up: use the right tool for the job.

Using ThreadPool doesn't feel like the right tool for executing multiple continuous CPU-bound tasks, even if you try to organize some cooperative execution by turning them into state machines which yield CPU time to each other with await Task.Yield(). Thread switching is rather expensive; by doing await Task.Yield() on a tight loop you add a significant overhead. Besides, you should never take over the whole ThreadPool, as the .NET framework (and the underlying OS process) may need it for other things. On a related note, TPL even has the TaskCreationOptions.LongRunning option that requests to not run the task on a ThreadPool thread (rather, it creates a normal thread with new Thread() behind the scene).

That said, using a custom TaskScheduler with limited parallelism on some dedicated, out-of-pool threads with thread affinity for individual long-running tasks might be a different thing. At least, await continuations would be posted on the same thread, which should help reducing the switching overhead. This reminds me of a different problem I was trying to solve a while ago with ThreadAffinityTaskScheduler.

Still, depending on a particular scenario, it's usually better to use an existing well-established and tested tool. To name a few: Parallel Class, TPL Dataflow, System.Threading.Channels, Reactive Extensions.

There is also a whole range of existing industrial-strength solutions to deal with Publish-Subscribe pattern (RabbitMQ, PubNub, Redis, Azure Service Bus, Firebase Cloud Messaging (FCM), Amazon Simple Queue Service (SQS) etc).

Answer 2

After a bit of debating on the issue with other users - who are worried about the context switching and its influence on the performance. I see what they are worried about.

But I meant: do something ... inside the loop to be a substantial task - usually in the form of a message handler which reads a message from the queue and processes it. The message handlers are usually user defined and the message bus executes them using some sort of dispatcher. The user can implement a handler which executes synchronously (nobody knows what the user will do), and without Task.Yield that will block the thread to process those synchronous tasks in a loop.

Not to be empty worded i added tests to github: https://github.com/BBGONE/TestThreadAffinity They compare the ThreadAffinityTaskScheduler, .NET ThreadScheduler with BlockingCollection and .NET ThreadScheduler with Threading.Channels.

The tests show that for Ultra Short jobs the performance degradation is around 15%. To use the Task.Yield without the performance degradation (even small) - it is not to use extremely short tasks and if the task is too short then combine shorter tasks into a bigger batch.

[The price of context switch] = [context switch duration] / ([job duration]+[context switch duration]).

In that case the influence of the switching the tasks is negligible on the performance. But it adds a better task cooperation and responsiveness of the system.

For long running tasks it is better to use a custom Scheduler which executes tasks on its own dedicated thread pool - (like the WorkStealingTaskScheduler).

For the mixed jobs - which can contain different parts - short running CPU bound, asynchronous and long running code parts. It is better to split the task into subtasks.

private async Task HandleLongRunMessage(TestMessage message, CancellationToken token = default(CancellationToken))
{ 
            // SHORT SYNCHRONOUS TASK - execute as is on the default thread (from thread pool)
            CPU_TASK(message, 50);
            // IO BOUND ASYNCH TASK - used as is
            await Task.Delay(50);
            // BUT WRAP the LONG SYNCHRONOUS TASK inside the Task 
            // which is scheduled on the custom thread pool 
            // (to save threadpool threads)
            await Task.Factory.StartNew(() => {
                CPU_TASK(message, 100000);
            }, token, TaskCreationOptions.DenyChildAttach, _workStealingTaskScheduler);
}