JToken.WriteToAsync does not write to JsonWriter

Question

I'm trying to create a middleware that changes the request in a certain way. I am able to read it and change the content but I cannot figure out how to correctly setup the stream writers to create a new body. When I call normalized.WriteToAsync(jsonWriter) the MemoryStream remains empty and consequently I receive the A non-empty request body is required. exception. What am I missing here? This is what I have so far:

public async Task Invoke(HttpContext context)
{
    if (context.Request.ContentType == "application/json" && context.Request.ContentLength > 0)
    {
        using var scope = _logger.BeginScope("NormalizeJson");
        try
        {
            using var requestReader = new HttpRequestStreamReader(context.Request.Body, Encoding.UTF8);
            using var jsonReader = new JsonTextReader(requestReader);

            var json = await JToken.LoadAsync(jsonReader);
            var normalized = _normalize.Visit(json); // <-- Modify json and return JToken

            // Create new Body
            var memoryStream = new MemoryStream();
            var requestWriter = new StreamWriter(memoryStream);
            var jsonWriter = new JsonTextWriter(requestWriter);
            await normalized.WriteToAsync(jsonWriter); // <-- At this point the MemoryStream has still 0 length.

            var content = new StreamContent(memoryStream.Rewind()); // <-- Use helper extension to Seek.Begin = 0                 
            context.Request.Body = await content.ReadAsStreamAsync();
        }
        catch (Exception e)
        {
            _logger.Scope().Exceptions.Push(e);
        }
    }

    await _next(context);
}

Demo for LINQPad etc.:

async Task Main()
{
    var token = JToken.FromObject(new User { Name = "Bob" });

    var memoryStream = new MemoryStream();
    var requestWriter = new StreamWriter(memoryStream);
    var jsonWriter = new JsonTextWriter(requestWriter);
    await token.WriteToAsync(jsonWriter); 
    memoryStream.Length.Dump(); // <-- MemoryStream.Length = 0
}

public class User
{
    public string Name { get; set; }
}

Since you're writing to a `MemoryStream`, why use `async` at all? — dbc, Mar 29 '20 at 15:01
@dbc I don't have any clever answer to that but consistency. Would you use something else instead? — t3chb0t, Mar 29 '20 at 15:04
Yes, I'd just do [`WriteTo()`](https://www.newtonsoft.com/json/help/html/M_Newtonsoft_Json_Linq_JToken_WriteTo.htm), declare victory and move on? — dbc, Mar 29 '20 at 15:09
@dbc I've just tested it and unfortunately the `MemoryStream` is still empty :\ There must be some piece of magic missing here... — t3chb0t, Mar 29 '20 at 15:11
@dbc See the _demo_ below the actual code. This is the minimal example that reproduces this issue. — t3chb0t, Mar 29 '20 at 15:14
@t3chb0t When I call `Flush()` on `jsonWriter` the memory stream get data, so you probably need to wrap the writer in a `using` — Nkosi, Mar 29 '20 at 15:31

dbc · Accepted Answer · 2021-05-11T15:06:53.233

You need to properly flush and close your JsonTextWriter and StreamWriter in order to fully populate the memoryStream, like so:

var memoryStream = new MemoryStream();
// StreamWriter implements IAsyncDisposable
// Leave the underlying stream open
await using (var requestWriter = new StreamWriter(memoryStream, leaveOpen: true)) 
{
    var jsonWriter = new JsonTextWriter(requestWriter); // But JsonTextWriter does not implement IAsyncDisposable, only IDisposable!
    try
    {
        await token.WriteToAsync(jsonWriter); 
    }
    finally
    {
        await jsonWriter.CloseAsync();
    }
}

Demo fiddle #1 here.

Or, since you're writing to a MemoryStream, there's really no nead to use async at all, and instead you can do:

var memoryStream = new MemoryStream();
using (var requestWriter = new StreamWriter(memoryStream, leaveOpen: true)) // Leave the underlying stream open
using (var jsonWriter = new JsonTextWriter(requestWriter))
{
    token.WriteTo(jsonWriter); 
}

Demo fiddle #2 here.

Notes:

Note the use of await using for the StreamWriter. This syntax guarantees that the StreamWriter will be flushed and closed asynchronously, and can be used on any object that implements IAsyncDisposable. (This only really matters if you were writing to a file stream or other non-memory stream.)
It seems that neither JsonTextWriter nor the base class JsonWriter implement IAsyncDisposable, so I had to asynchronously close the JSON writer manually rather than via a using statement. The outer await using should ensure that the underlying StreamWriter is not left open in the event of an exception.
JSON RFC 8259 specifies that Implementations MUST NOT add a byte order mark (U+FEFF) to the beginning of a networked-transmitted JSON text. Thus, when constructing a StreamWriter, it is recommended to pass an encoding such as new UTF8Encoding(false) that does not prepend a BOM. Alternatively, if you just want UTF-8, the StreamWriter constructors will create a StreamWriter with UTF-8 encoding without a Byte-Order Mark (BOM) if you do not specify one yourself and leave a default value for that parameter as is shown in the code above.

What a combo of awaits and close and usings and leave-open! :-O — t3chb0t, Mar 29 '20 at 16:02

JToken.WriteToAsync does not write to JsonWriter

1 Answers1

Linked