large number of threads is wait state in threadpool causing performance problems

Question

My application connects to a large number of clients over http, downloads data from those clients and processes data as these results are received. Each request is sent in a separate thread so that the main thread does not remain occupied.

We have started encountering performance issues and it seems like these are mostly related to large number of threads in the ThreadPool that are just waiting for getting data back from those requests. I know with .NET 4.5 we have async and await for the same type of problem but we are still using .NET 3.5.

Any thoughts on what's the best way of sending these requests in a different thread but not to keep that thread alive while all its doing is to keep waiting for request to come back?

Thomas Levesque · Accepted Answer · 2014-05-04T00:13:51.143

3

You can use async operations in .NET 3.5, it's just not as convenient as in .NET 4.5. Most IO methods have a BeginX/EndX method pair that is the async equivalent of the X method. This is called the Asynchronous Programming Model (APM).

For instance, instead of Stream.Read, you could use Stream.BeginRead and Stream.EndRead.

Actually, many async IO methods in .NET 4.5 are just wrappers around the Begin/End methods.

edited May 04 '14 at 00:13

answered May 03 '14 at 23:55

Thomas Levesque

286,951
70
623
758

@PaulSnow, yes; the `BeginRead` method is non-blocking, and the callback you pass to it will be called on a ThreadPool thread when the IO operation is complete, without wasting a thread for waiting. – Thomas Levesque May 04 '14 at 00:04
Awesome. Exactly what I looking for :) – crazy novice May 04 '14 at 00:07
To add to the answer, BeginX/EndX methods use IO threads which internally use IO completion ports to make non blocking IO calls. Also, instead of explicitly creating threads, it is recommended to enqueue your tasks on to the threadpool and let it manage the threads for you. A thread pool will create a thread only if existing threads are running or waiting. Using IOCP, you minimize the waiting times - so it wouldn't be surprising if the thread pool sends m requests with n threads (m >= n). – swazza85 May 07 '14 at 16:28

score 1 · Answer 2 · edited May 23 '17 at 12:05

If you cannot use .NET 4.x and async/await, you still can achieve a sort of similar behavior using IEnumerator and yield. It allows to use pseudo-synchronous linear code flow with Begin/End-style callbacks, including statements like using, try/finally, while/for/foreach etc. You cannot use try/catch, though.

There are a few implementations of the asynchronous enumerator driver out there, e.g. Jeffrey Richter's AsyncEnumerator.

I used something like below in the past:

class AsyncIO
{
    void ReadFileAsync(string fileName)
    {
        AsyncOperationExt.Start(
            start => ReadFileAsyncHelper(fileName, start),
            result => Console.WriteLine("Result: " + result),
            error => Console.WriteLine("Error: " + error));
    }

    static IEnumerator<object> ReadFileAsyncHelper(string fileName, Action nextStep)
    {
        using (var stream = new FileStream(
            fileName, FileMode.Open, FileAccess.Read, FileShare.Read, bufferSize: 1024, useAsync: true))
        {
            IAsyncResult asyncResult = null;
            AsyncCallback asyncCallback = ar => { asyncResult = ar; nextStep(); };
            var buff = new byte[1024];
            while (true)
            {
                stream.BeginRead(buff, 0, buff.Length, asyncCallback, null);
                yield return Type.Missing;
                int readBytes = stream.EndRead(asyncResult);
                if (readBytes == 0)
                    break;
                // process the buff
            }
        }
        yield return true;
    }
}

// ...

// implement AsyncOperationExt.Start
public static class AsyncOperationExt
{
    public static void Start<TResult>(
        Func<Action, IEnumerator<TResult>> start,
        Action<TResult> oncomplete,
        Action<Exception> onerror)
    {
        IEnumerator<TResult> enumerator = null;

        Action nextStep = () =>
        {
            try
            {
                var current = enumerator.Current;
                if (!enumerator.MoveNext())
                    oncomplete(current);
            }
            catch (Exception ex)
            {
                onerror(ex);
            }
            enumerator.Dispose();
        };

        try
        {
            enumerator = start(nextStep);
        }
        catch (Exception ex)
        {
            onerror(ex);
            enumerator.Dispose();
        }
    }
}

large number of threads is wait state in threadpool causing performance problems

2 Answers2