7

I need to upload large files (~200MB) over HTTP protocol. I want to avoid loading the files to memory and want to send them directly.

Thanks to this article I was able to make it with HttpWebRequest.

HttpWebRequest requestToServer = (HttpWebRequest)WebRequest.Create("....");

requestToServer.AllowWriteStreamBuffering = false;
requestToServer.Method = WebRequestMethods.Http.Post;
requestToServer.ContentType = "multipart/form-data; boundary=" + boundaryString;
requestToServer.KeepAlive = false;
requestToServer.ContentLength = ......;

using (Stream stream = requestToServer.GetRequestStream())
{
    // write boundary string, Content-Disposition etc.
    // copy file to stream
    using (var fileStream = new FileStream("...", FileMode.Open, FileAccess.Read))
    {
        fileStream.CopyTo(stream);
    }

    // add some other file(s)
}

However, I would like to do it via HttpClient. I found article which describes using of HttpCompletionOption.ResponseHeadersRead and I tried something like this but it does not work unfortunately.

WebRequestHandler handler = new WebRequestHandler();

using (var httpClient = new HttpClient(handler))
{
    httpClient.DefaultRequestHeaders.Add("ContentType", "multipart/form-data; boundary=" + boundaryString);
    httpClient.DefaultRequestHeaders.Add("Connection", "close");

    var httpRequest = new HttpRequestMessage(HttpMethod.Post, "....");

    using (HttpResponseMessage responseMessage = await httpClient.SendAsync(httpRequest, HttpCompletionOption.ResponseHeadersRead))
    {
        using (Stream stream = await responseMessage.Content.ReadAsStreamAsync())
        {
            // here I wanted to write content to the stream, but the stream is available only for reading
        }
    }
}

Maybe I overlooked or missed something...

UPDATE

On top of it, it is important to use StreamContent with proper headers:

Jakub Krampl
  • 1,774
  • 1
  • 16
  • 24
  • If you have a method that works, why bother? – theMayer Jan 19 '18 at 15:57
  • fyi comment - I have encountered different behaviors with the objects your are using between full framework and core2.0 and on prem and azure. You may have to consider your final target environment to feel confident in your final solution. – Sql Surfer Jan 19 '18 at 16:11
  • 1
    You need to read [You are using HttpClient wrong and it's destabilizing your software](https://aspnetmonsters.com/2016/08/2016-08-27-httpclientwrong/). – mason Jan 19 '18 at 16:13
  • @mason that article is dogma and shouldn't be recommended without changes. – Cory Nelson Jan 19 '18 at 21:23
  • @CoryNelson What are you talking about? This is a well known issue. You shouldn't create an HttpClient each time you use it. And if you use a singleton, depending on the application, you risk not getting new IP's in the event of DNS changes. That article is absolutely correct, and libraries that build on top of HttpClient [account for it](https://github.com/tmenier/Flurl/issues/144). – mason Jan 19 '18 at 21:27
  • 1
    @CoryNelson Before calling something "dogma" and dismissing it, make sure you've done adequate research. Microsoft has also stated "HttpClient is intended to be instantiated once and reused throughout the life of an application....Creating a new HttpClient instance per request can exhaust the available sockets." [Source](https://learn.microsoft.com/en-us/aspnet/web-api/overview/advanced/calling-a-web-api-from-a-net-client). Also see the relevant [Stack Overflow question](https://stackoverflow.com/questions/15705092/do-httpclient-and-httpclienthandler-have-to-be-disposed). – mason Jan 19 '18 at 21:36
  • @mason slow your role man, you're getting too worked up. The blog post claims that `using(HttpClient)` is wrong and that apps should use a globally reused `HttpClient`, but this is not a change without consequences -- this changes how `HttpClient` behaves. To not address this is a mistake, and to recommend all newbies switch over to this would be to misinform them. That is all I meant. – Cory Nelson Jan 19 '18 at 23:21
  • Current recommendation is to never depend on HttpClient directly, but to use IHttpClientFactory to get HttpClient objects as needed. https://learn.microsoft.com/en-us/dotnet/architecture/microservices/implement-resilient-applications/use-httpclientfactory-to-implement-resilient-http-requests – KwaXi Dec 14 '21 at 21:03

1 Answers1

5

See the StreamContent class:

HttpResponseMessage response =
    await httpClient.PostAsync("http://....", new StreamContent(streamToSend));

In your example, you are getting a response stream and trying to write to it. Instead, you must pass in your content for the request, as above.

The HttpCompletionOption.ResponseHeadersRead is to disable buffering of the response stream, but does not affect the request. You would typically use it if your response is large.

For posting multiple files of form data, use the MultipartFormDataContent:

var content = new MultipartFormDataContent();

content.Add(new StreamContent(stream1), "file1.jpg");
content.Add(new StreamContent(stream2), "file2.jpg");

HttpResponseMessage response =
    await httpClient.PostAsync("http://...", content);
Cory Nelson
  • 29,236
  • 5
  • 72
  • 110