Getting an UTF-8 response with httpclient in Windows Store apps

Question

I'm building a Windows Store app, but I'm stuck at getting a UTF-8 response from an API.

This is the code:

using (HttpClient client = new HttpClient())
{
    Uri url = new Uri(BaseUrl + "/me/lists");

    HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Get, url);
    request.Headers.Add("Accept", "application/json");
    HttpResponseMessage response = await client.SendRequestAsync(request);
    response.EnsureSuccessStatusCode();

    string responseString = await response.Content.ReadAsStringAsync();

    response.Dispose();
}

The reponseString always contains strange characters which should be accents like é, and I tried using a stream, but the API I found in some examples don't exist in Windows RT.

Edit: improved code, still same problem.

Answer 1

Instead of using response.Content.ReadAsStringAsync() directly you could use response.Content.ReadAsBufferAsync() pointed by @Kiewic as follows:

var buffer = await response.Content.ReadAsBufferAsync();
var byteArray = buffer.ToArray();
var responseString = Encoding.UTF8.GetString(byteArray, 0, byteArray.Length);

This is working in my case and I guess that using UTF8 should solve most of the issues. Now go figure why there is no way to do this using ReadAsStringAsync :)

Answer 2

Solved it like this:

using (HttpClient client = new HttpClient())
    {
        using (HttpResponseMessage response = await client.GetAsync(url))
            {
                var byteArray = await response.Content.ReadAsByteArrayAsync();
                var result = Encoding.UTF8.GetString(byteArray, 0, byteArray.Length);
                return result;
            }
    }

Answer 3

Perhaps the problem is that the response is zipped. If the content type is gzip, you will need decompress the response in to a string. Some servers do this to save bandwidth which is normally fine. In .NET Core and probably .NET Framework, this will automatically unzip the response. But this does not work in UWP. This seems like a glaring bug in UWP to me.

string responseString = await response.Content.ReadAsStringAsync();

This thread gives a clear example of how to decompress the response:

Compression/Decompression string with C#

Answer 4

I like El Marchewko's approach of using an extension, but the code did not work for me. This did:

using System.IO;
using System.Net.Http;
using System.Text;
using System.Threading.Tasks;

namespace WannaSport.Data.Integration
{
    public static class HttpContentExtension
    {
        public static async Task<string> ReadAsStringUTF8Async(this HttpContent content)
        {
            return await content.ReadAsStringAsync(Encoding.UTF8);
        }

        public static async Task<string> ReadAsStringAsync(this HttpContent content, Encoding encoding)
        {
            using (var reader = new StreamReader((await content.ReadAsStreamAsync()), encoding))
            {
                return reader.ReadToEnd();
            }
        }
    }
}

Answer 5

The HttpClient doesn't give you a lot of flexibility.

You can use a HttpWebRequest instead and get the raw bytes from the response using HttpWebResponse.GetResponseStream().

Answer 6

Can't comment yet, so I'll have to add my thoughts here.

You could try to use _client.GetStringAsync(url) as @cremor suggested, and set your authentication headers using the _client.DefaultRequestHeaders property. Alternatively, you could also try to use the ReadAsByteArrayAsync method on the response.Content object and use System.Text.Encoding to decode that byte array to a UTF-8 string.

Answer 7

My approach using an Extension:

    using System;
    using System.Collections.Generic;
    using System.IO;
    using System.Linq;
    using System.Text;
    using System.Threading.Tasks;
    using Windows.Web.Http;

    namespace yourfancyNamespace
    {
        public static class IHttpContentExtension
        {
            public static async Task<string> ReadAsStringUTF8Async(this IHttpContent content)
            {
                return await content.ReadAsStringAsync(Encoding.UTF8);
            }
            public static async Task<string> ReadAsStringAsync(this IHttpContent content, Encoding encoding)
            {
                using (TextReader reader = new StreamReader((await content.ReadAsInputStreamAsync()).AsStreamForRead(), encoding))
                {
                    return reader.ReadToEnd();
                }
            }
        }
    }