Why is there a threshold of concurrent HTTP requests I can make using `HttpClient`?

Question

I noticed that there exists some kind of threshold for concurrent HTTP requests I can making using .NET core's HttpClient i.e. it seems to work fine when I have <= 1,000 requests, but nearing 10,000 is problematic. Here is the relevant failing code:

using System;
using System.Net.Http;
using System.Threading.Tasks;

namespace RequestsGalore
{
    class Program
    {
        static HttpClient Client { get; set; } = new HttpClient();

        static void Main(string[] args)
        {
            var url = "http://example.com";

            int requests = 10_000;
            var tasks = new Task<HttpResponseMessage>[requests];

            for (int i = 0; i < requests; i++)
            {
                tasks[i] = Client.GetAsync(url, HttpCompletionOption.ResponseHeadersRead);
            }

            Task.WaitAll(tasks);

            for (int i = 0; i < requests; i++)
            {
                Console.WriteLine(tasks[i].Result.StatusCode);
            }
        }
    }
}

, and the exception:

Unhandled Exception: System.AggregateException: One or more errors occurred.
(An error occurred while sending the request.)
(A task was canceled.)
.
. [MANY OF THE ABOVE TWO MESSAGES]
.
---> System.Net.Http.HttpRequestException: An error occurred while sending the request.
---> System.IO.IOException: Unable to read data from the transport connection: Connection reset by peer.
---> System.Net.Sockets.SocketException: Connection reset by peer
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token)
   at System.Net.Http.HttpConnection.FillAsync()
   at System.Net.Http.HttpConnection.ReadNextResponseHeaderLineAsync(Boolean foldedHeadersAllowed)
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithNtConnectionAuthAsync(HttpConnection connection, HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.FinishSendAsyncUnbuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.WaitAllCore(Task[] tasks, Int32 millisecondsTimeout, CancellationToken cancellationToken)
   at RequestsGalore.Program.Main(String[] args) in /home/[REDACTED]/Downloads/RequestsGalore/Program.cs:line 27

And some information about my machine:

$ dotnet --info
.NET Core SDK (reflecting any global.json):
 Version:   2.2.401
 Commit:    729b316c13

Runtime Environment:
 OS Name:     ubuntu
 OS Version:  19.04
 OS Platform: Linux
 RID:         ubuntu.19.04-x64
 Base Path:   /usr/share/dotnet/sdk/2.2.401/

Host (useful for support):
  Version: 2.2.6
  Commit:  7dac9b1b51

.NET Core SDKs installed:
  2.2.401 [/usr/share/dotnet/sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.2.6 [/usr/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.2.6 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.2.6 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

To install additional .NET Core runtimes or SDKs:
  https://aka.ms/dotnet-download

https://stackoverflow.com/questions/48778580/singleton-httpclient-vs-creating-new-httpclient-request — riffnl, Sep 05 '19 at 22:30
The above link is not relevant as my code _does indeed_ use a singleton instance of `HttpClient`. — adyavanapalli, Sep 05 '19 at 22:34

score 4 · Answer 1 · answered Sep 05 '19 at 22:32

4

"Connection reset by peer" points to the other end, not your code, dropping the connection.

answered Sep 05 '19 at 22:32

tymtam

31,798
8
86
126

1

And the code amounts to a denial-of-service attack. – David Browne - Microsoft Sep 05 '19 at 22:34
@DavidBrowne-Microsoft I see. This is just a toy example that requests the header information from the same `example.com` domain. My original use case is to request the header information for various resources on a given domain to see if the resource exists. To avoid DoS'ing the server, what can I do? Perhaps stagger the number of requests I make at the same time? What's a good approach for this? – adyavanapalli Sep 05 '19 at 22:41
@tymtam Given a list of URLs (which are necessarily served on the same domain), determine whether any of the URLs exist. I'm doing this by sending a `HTTP HEAD` request to each URL and seeing if I get a `200` response in return. – adyavanapalli Sep 05 '19 at 22:51
Connections require resources, such as [ephemeral ports](http://smallvoid.com/article/winnt-tcpip-max-limit.html). There are a limited number of these. They often have a timeout (e.g. to capture any delayed packets) meaning that they remain reserved for up to 120 seconds after they are used. These limitations apply not only to your client, and not just the server, but any proxies or network translation nodes in between. I suggest you limit yourself to 1,000 requests at a time by establishing a connection pool and running your transactions through it at a limited rate. – John Wu Sep 05 '19 at 23:21
@JohnWu Thanks for your answer. Can you point me to a resource where I can read about connection pools? Any specific .NET related implemented would be useful as well. Thanks again. – adyavanapalli Sep 05 '19 at 23:45

David Browne - Microsoft · Accepted Answer · 2019-09-06T00:47:31.460

3

This code opens too many concurrent connections to the target web server, probably triggering anti-denial-of-service protections. There's a simple way to limit the concurrent requests to any target in .NET Framework the ServicePoint, and there are default limits in place.

In .NET Core ServicePoint is not used. And you set the limit using the HttpClientHandler:

        var url = "http://example.com";
        HttpClientHandler handler = new HttpClientHandler();
        handler.MaxConnectionsPerServer = 10;
        Client = new HttpClient(handler);

edited Sep 06 '19 at 00:47

answered Sep 05 '19 at 23:39

David Browne - Microsoft

80,331
6
39
67

1

Can you back-up your claim that on .NET Core the number of allowed concurrent requests is unlimited? I'm looking at the documentation for `ServicePointManager.DefaultConnectionLimit` and it looks like it's set to `2`: https://learn.microsoft.com/en-us/dotnet/api/system.net.servicepointmanager.defaultconnectionlimit?view=netcore-2.2#System_Net_ServicePointManager_DefaultConnectionLimit – adyavanapalli Sep 05 '19 at 23:57
1

I mis-remembered. .NET Core simply ignores the ServicePoint and you limit the connections with HttpClientHandler. See edit. – David Browne - Microsoft Sep 06 '19 at 00:09
Awesome! Thanks for your help :) – adyavanapalli Sep 06 '19 at 01:46

Why is there a threshold of concurrent HTTP requests I can make using `HttpClient`?

2 Answers2