Task running I/O with async-await for parallel producer-consumer

Question

Assume that I have a producer-consumer pattern like this. The post also explains why using TPL Dataflow might not be optimal in this specific case. That is why this question uses Tasks instead.

+----------+                          +----------------+                           +-----------+ 
|   Task   |                          |      Task      |                           |   Task    |
|Read files|-----BlockingCollection-->|Process values  |----BlockingCollection---->|Write files|
+----------+   |                      |of data logger 1|  |                        +-----------+ 
               |                      +----------------+  |             
               |                      +----------------+  |
               |                      |      Task      |  |
               |-BlockingCollection-->|Process values  |--|
               |                      |of data logger 2|  |
               |                      +----------------+  |
                ...                       (n Tasks)        ...

In this implementation the reading and writing needs to happen concurrently to the processing, so each use a Task for that purpose. If I would use a blocking function for reading and writing, this would be the way to go, but how about an asynchronous read/write? Now I would like to know if I understood the use of async-await correctly, in this specific case. As I need the parallelism, the read and write should still occur in a separate Task. I would not like to waste CPU cycles on waiting for I/O results so async-await seems to be the solution.

Take this pseudo-implementation as an example. The ReadAllLinesAsync would be an implementation like this

BlockingCollection<string []> queue = new BlockingCollection<string []>(100);
var pathsToFiles = files;
await Task.Run(async () => 
{
    //This is a dummy for the "Read files"-object's function call
    for(int i=0; i<pathsToFiles.Length; i++)
    {
        string[] file = await ReadAllLinesAsync(pathsToFiles[i]);
        queue.Add(file);
    }
    queue.CompleteAdding();
}

The first question is, does the example code use async-await properly?
The second question is, does this improve efficiency compared to the blocking I/O, by not blocking the CPU waiting for I/O?

I read the several articles and post on the async-await topic and have to say this is quite complicated with all the does and don'ts. Specifically I read the articles by Steven Cleary.

You may want to look in to [TPL Dataflow](https://msdn.microsoft.com/en-us/library/hh228603(v=vs.110).aspx) it makes complex async consumer/producer pipelines much easier. — Scott Chamberlain, Mar 25 '16 at 23:42
@ScottChamberlain I know you gave me the tip a couple days ago, thanks for that! The problem is that it does not fit very well for this specific case as far as I am aware. If you would be so kind have a look at the first link I posted. P.S: I was really hoping you would see this :) — , Mar 25 '16 at 23:47
@ScottChamberlain the reason I am not using TPL Dataflow right know, is that I need to process values from data loggers in parallel, but for each data logger in a different way. Within the processing for one data logger the processing has to be sequential (chronological). Also the data from the reading block needs to be distributed to specific blocks and conditional linking seems like an overhead. For a detailed description see my post here http://programmers.stackexchange.com/questions/313759/partially-parallel-producer-consumer-pattern-with-internal-state — , Mar 26 '16 at 10:32

score 1 · Answer 1 · answered Mar 26 '16 at 11:29

1

This usage of await to save a thread while doing IO is correct. Whether it will gain you anything depends on whether threads are a scarce resource for you.

If the thread pool is overloaded this can help a lot. On the other hand if the thread pool is overloaded you are probably oversubscribing the disk anyway. You could improve disk usage by limiting the number of outstanding IOs using something like await SemaphoreSlim.WaitAsync().

IO efficiency is not impacted by the call style (sync or async). Usually, async IO uses a tiny bit more CPU which is negligible here because the IO costs dominate.

Is this a GUI scenario? Otherwise I don't see the point of using await Task.Run.

answered Mar 26 '16 at 11:29

usr

168,620
35
240
369

As I am expecting about 200 data loggers and for each one a Task for the processing, I guess the thread pool is overloaded. I would just have one Task for reading and one Task for writing. I am not sure how the semaphore would improve disk usage. Do you mean I should only let either read or write? As far as I understood async and I/O it does improve efficiency by not blocking and through the usage of I/O completion ports. The I/O is a bottleneck in the producer-consumer as it will be the slowest part. It would be used in a Windows service so I guess you could drop that one await. – Mar 26 '16 at 12:14
The disk head does not move in a different way no matter how you initiate the IO. You will only see perf improvements by improving the motion of the disk head somehow. Many concurrent disk IOs have the effect of randomizing IO thereby destroying performance. That's why you often need to tame IO by limiting the DOP (often to 1 for magnetic disks). – usr Mar 26 '16 at 14:49
200 tasks are a lot and I can see a good case for async IO and waiting here. `improve efficiency by not blocking and through the usage of I/O completion ports` how das that improve efficiency? Frankly, this is too vague of an understanding. Too long to explain it here but you can basically forget about that. – usr Mar 26 '16 at 14:51
`The I/O is a bottleneck in the producer-consumer as it will be the slowest part` Correct. Therefore, any CPU-based optimization is pointless. All that matters here is improving physical IO patterns and not overloading the thread pool. – usr Mar 26 '16 at 14:53
I think it is a little too low level to optimize for disk head movement. Besides that this isn't even possible as long as you don't write your own firmware for the controller :) The efficiency would be improved because you don't spend CPU cycles idle waiting for I/O and as far as I understood with async you get the maximum performance for I/O fully utilizing I/O completion ports. You can't really get faster than that, as far as I am aware. The only things that you can tweak are the buffer size and the size of the blocking queue. – Mar 26 '16 at 19:37
I would not call CPU optimization pointless, it just doesn't have such a great impact. The question is, is there any way to get even more performance out of this, specifically for I/O? It's NOT like this is a high demand application with thousand of data loggers and enormous amounts of data. The service runs periodically, reads all available files and feeds the pipeline. It should be as fast as possible though by reasonable means. – Mar 26 '16 at 19:40
During an IO the CPU does nothing for that thread. It makes no sense to keep the CPU core busy. Another thread can take over. You are falling for some common async misconceptions. I have already explained that the physical IO that is executed is not changed by the way you initiate the call. – usr Mar 26 '16 at 21:09
Almost all IO throughput optimizations are disk head movement optimizations. For example you can turn random into sequential, or random into elevator ordered. If you issue many sequential IOs at once the (stupid) OS turns them into random. This is basically a Windows kernel defect if you ask me. You can see that using procmon.exe if you want to dig in deep. – usr Mar 26 '16 at 21:11
Yes of course it does not make sense to keep the CPU core busy and that is why we use async, right? While it is true that the actual physical process does not change and cannot be magically speed up, it does change how the os deals with it, right? Pretty sure that optimizing on head movement level is over kill for me. The actual mapping to sectors and such is usually just pretend from the controller, they may be at a totally different area on the physical disk. Combine that with the specialty of SSDs and you have a hard time to generally get a better performance. – Mar 26 '16 at 22:16
So do you have a specific suggestion on what I should do? The only reasonable optimization that comes to my mind, is to make sure the read and write is never happen at the same time. Possibly even let each process a larger chunk before the other gets the turn. – Mar 26 '16 at 22:22
The best way to optimize IO is to experiment with different degrees of parallelism. Read/write does not matter. Try locking IO with a `SemaphoreSlim` with different DOP levels. This kind of optimization works because it optimized disk head movement. Sometimes, the best DOP is 1 to make it sequential. Sometimes it is "as high as possible" to get it do do elevator random IO. On SSD intermediate DOP levels might make sense. – usr Mar 27 '16 at 21:13
`that is why we use async, right` no, as explained the call style does not change anything about the IO. It does not significantly change what happens on physical CPUs. I think you don't understand what async IO is. Something to research. CPU cores are not blocked when threads are blocked. If that was the case a single core machine would freeze during any IO. – usr Mar 27 '16 at 21:14
Alright I looked into the usage of SemaphoreSlim. It would make sense if I created multiple Tasks to read and write like [here](http://stackoverflow.com/questions/10806951/how-to-limit-the-amount-of-concurrent-async-i-o-operations), but in my case I definitely only have one Task. The Task would be created and iterate through all files. – Mar 28 '16 at 10:23
Regarding async, it definitely does change what happens otherwise we wouldn't get any benefit at all. Maybe we are talking past each other. [This](https://channel9.msdn.com/Series/Three-Essential-Tips-for-Async/Tip-2-Distinguish-CPU-Bound-work-from-IO-bound-work) is what I am talking about. – Mar 28 '16 at 10:25
I never claimed it did nothing. I can only recommend that you understand in detail what async changes and under what circumstances that's helpful. Most people don't understand that including Microsoft employees. In a UI scenario it is very convenient to use it but this does not apply here. In a server/background work scenario it saves threads, that's all. We have written server apps without async for 15 years in .NET now and it went just fine. They did "scale".; I'm all for using async here because you have 200 of them. Just don't expect IO improvements. – usr Mar 28 '16 at 10:51
I would like to let this comment thread rest now. I feel you need to do some research on your own. All the statements that I made are true. If you disagree with them pick them apart and find out what others are saying. – usr Mar 28 '16 at 10:53
Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/107533/discussion-between-john-and-usr). – Mar 28 '16 at 11:12
Sure. I created a chat just in case you want to write about this in the future. – Mar 28 '16 at 11:30

Task running I/O with async-await for parallel producer-consumer

1 Answers1