0

I already have multiple async actors polling remote servers for data, and need this data persisted to a database. It seems like a classic consumer-producer pattern, with multiple Poller producers and a single Persister consumer. IProducerConsumerCollection<T> seems a great option here, and it seems to suit my purposes better that T is a simple chunk of data DataUnit, not a unit of work - I want the consumer to decide what work is done on the data, not the producer. i.e. I am favoring a data-queue not a task-queue.

So, using BlockingCollection<DataUnit> would give a trivial implementation along the lines of while(...){ DoStuff(blockingCollection.Take(); } (checks omitted for simplicity) but BlockingCollection.Take() obviously commits a thread which blocks idly waiting for data, which though not a huge problem would be nice to avoid.

My first question is simply: why is there no TakeAsync method? Does the fact it's not there imply I've misunderstood the designed use?

Another option could be to use ConcurrentQueue<DataUnit> directly:

while(...)//checks and stuff
{
 DataItem item;
 if(concurrentQueue.TryTake(out item);
  DoStuff(item);
 else
  await Task.Delay(50);
}

I dislike hard-coding a delay of 50ms but is this a reasonable approach? If not, is there an obvious alternative?

Mr. Boy
  • 60,845
  • 93
  • 320
  • 589
  • 3
    For Q1, try taking a look at [Is there anything like asynchronous BlockingCollection?](https://stackoverflow.com/questions/21225361/is-there-anything-like-asynchronous-blockingcollectiont). Q2 looks reasonable for me as I've seen different projects with the same approach mostly on background services and azure worker roles. – jegtugado May 20 '20 at 13:07
  • The reason there is no `TakeAsync` is because the `BlockingCollection` class is blocking by nature. It is named **Blocking**Collection for a reason. – Theodor Zoulias May 20 '20 at 13:26
  • @TheodorZoulias yeah but then why isn't there an equivalent on non-clocking concurrent collections is the obvious question - it seems like a really obvious thing which makes me think there is a good reason not to add it. – Mr. Boy May 20 '20 at 13:37
  • @jegtugado that seems a nice implementation (https://stackoverflow.com/a/48963786/197229 is the direct link). The other options seem like overkill. I'm not sure if this can be considered a dupe or the same answer to a different question? If people think this is a good direction for my usem that is. – Mr. Boy May 20 '20 at 13:46
  • 1
    Currently the best available async queue is the [`Channel`](https://devblogs.microsoft.com/dotnet/an-introduction-to-system-threading-channels/). It is even built-in the .NET Core. You only need the [package](https://www.nuget.org/packages/System.Threading.Channels) in .NET Framework platform. – Theodor Zoulias May 20 '20 at 14:42
  • @TheodorZoulias yeah I've looked at this before but it seems a lot to take on in my case. Definitely the way to go if things get more complex. Interestingly, the prelude in that article is almost exactly the code presented in the linked answer. – Mr. Boy May 20 '20 at 14:50
  • I'd say go with the [`Channel`](https://learn.microsoft.com/en-us/dotnet/api/system.threading.channels.channel). It is simple and sweet, and highly efficient and heavily tested, unlike any code you'll find here and there in SO questions. :-) The learning curve is almost nothing. It's just one class with a `Reader` and a `Writer` property. – Theodor Zoulias May 20 '20 at 15:12
  • @jegtugado did you realise you already posted this - you're the first comment above!? – Mr. Boy May 20 '20 at 16:15
  • 1
    The last comment is automated after voting to close this question. Cheers. – jegtugado May 20 '20 at 16:17

0 Answers0