How to Asynchronously read MemoryStream

Question

I am writing an application which involves IronRuby. I would like to update a UI element when I receive output from my ruby script, so I have set up the following:

var ScriptEngine engine = IronRuby.Ruby.CreateEngine();
MemoryStream outputStream = new MemoryStream();
engine.Runtime.IO.SetOutput(outputStream, Encoding.Default);

This redirects the output of IronRuby to a custom stream called outputStream. However, I cannot seem to figure out how to call a block of code once the stream receives new information. How could I do the equivalent of the following?

outputStream.DataReceived += (sender, e) =>
{
    // assumes I passed in the Func `processing` to my method
    processing(e.Value);
};

Thanks!

You generally set a buffer up, `byte[2048] buffer` and do a `while(outputStream.Read())` which will continuously loop reading `2048` bytes at a time and will continue to do so while the stream is open. See this [answer](http://stackoverflow.com/questions/221925/creating-a-byte-array-from-a-stream) — Callum Linington, May 12 '16 at 08:09
Perhaps if I used a `BackgroundWorker` I can just run it in a new thread? — Alexander Lozada, May 12 '16 at 08:14
http://stackoverflow.com/questions/20805396/asynchronous-memory-streaming-approach-which-of-the-following — Mohammad Sayadi, May 12 '16 at 08:17
What do you mean asynchronous, runs in another thread or uses `async/await` because either can be done with streams seeing as they have `ReadAsync()` — Callum Linington, May 12 '16 at 08:18
"Run in another thread" is **not** asynchronous. That other thread will still block. `ReadAsync` won't block any thread though — Panagiotis Kanavos, May 12 '16 at 08:19
Apologies! I'm somewhat new to programming asynchronous stuff. Thanks for the clarity. — Alexander Lozada, May 12 '16 at 08:22
Why do you want an asynchronous MemoryStream operation? You are only copying bytes from one location to another — Panagiotis Kanavos, May 12 '16 at 08:26

Panagiotis Kanavos · Answer 1 · 2016-05-12T08:37:53.220

2

The easiest way to read asynchronously from any stream is to use ReadAsync in the same way that you would use Read :

async Task MyMethod()
{
    ...
    byteArray = new byte[1024];
    int count = await memStream.ReadAsync(byteArray, 0, 1024);
    ...
}

Typically, ReadAsync is a truly asynchronous call, ie it doesn't use a separate thread that blocks while waiting for a result. It takes advantage of asynchronous I/O completion ports to hand off the operation to the operating system and start processing again only when the OS returns some results.

MemoryStream may have a simpler implementation though as no I/O is actually involved. In fact, an asyncrhonous operation doesn't make much sense as Read will simply copy bytes from the stream to a buffer.

UPDATE

After checking the source for MemoryStream.ReadAsync it seems that ReadAsync just calls Read directly and returns a Task<int> with the number of bytes read. That means that byte copying is still done synchronously but at least MemoryStream can still be used in asynchronous methods. For example, it can be used in a form method to avoid blocking the UI thread while copying a large buffer.

This makes sense as simply copying the bytes will be faster than setting up an asynchronous operation.

edited May 12 '16 at 08:37

answered May 12 '16 at 08:24

Panagiotis Kanavos

120,703
13
188
236

The problem comes when another thread is writing the stream while you're reading it - in that case, `ReadAsync()` could return 0 (indicating end of stream) even though the other thread is still writing to it. That mean you can end up in the horrible situation of looping and calling `ReadAsync()` repeatedly while waiting for new data, with it repeatedly returning 0 for an effectively arbitrary length of time. – Matthew Watson May 12 '16 at 08:33
@MatthewWatson that has nothing to do with asynchronous operations. In fact, ReadAsync just calls Read. What you describe is a pub/sub problem though. Streams are *not* a pub/sub implementation and what you describe will be an issue for any stream, even if you use `Read`. The closest to what you describe are [channels](https://github.com/dotnet/corefxlab/blob/master/src/System.Threading.Tasks.Channels/README.md). Until they become part of Core CLR though, you'd have to use one of the other pub/sub mechanisms – Panagiotis Kanavos May 12 '16 at 08:41
@MatthewWatson in fact, a BlockingCollection or a ConcurrentQueue would be far better than using a stream if you simply had to pass a buffer from one thread to another. – Panagiotis Kanavos May 12 '16 at 08:43
Agreed - it would affect non-async too. I just wanted to mention it because it could affect the OP. Also agree that another approach would be better for the OP. – Matthew Watson May 12 '16 at 09:00
@MatthewWatson in this scenario a stream is the *wrong* tool for the job. The OP isn't asking about an advanced scenario though. – Panagiotis Kanavos May 12 '16 at 09:02
Thanks for your answer! Unfortunately, there is little documentation on what I am trying to do - the only example I found used the output stream. I have a few questions - where would I call this async function? If I were reading the script from the moment I clicked a button, to the moment I clicked it again, how would I start/stop reading? When does the async method actually respond with data? I apologize for asking such noobie questions! – Alexander Lozada May 12 '16 at 09:35

How to Asynchronously read MemoryStream

1 Answers1