How to group data logging in multithreading?

Question

I have two methods. One method contains ThreadPool and ThreadPool runs second method.
Code snippet below:

    static void Main()
    {
        var processes = GetProcessCounter();

        foreach (var process in processes)
        {
            var tmp = process;
            ThreadPool.QueueUserWorkItem(WriteCounter, tmp); 
            Console.Write("\n\n");
        }

        Console.WriteLine("Any key to exit...");
        Console.Read();
    }

    static void WriteCounter(object obj)
    {
        var process = (ProcessInstance) obj;
        Console.WriteLine(
            $"Thread ID: {Thread.CurrentThread.ManagedThreadId}, " +
            $"Process Name: {process.ProcessName}, " +
            $"Process ID {process.ProcessId} ");
        Console.Write("\n");

        foreach (var cnt in process.Counters)
        {
            cnt.NextValue();
            Console.WriteLine(
                $"ThreadId: {Thread.CurrentThread.ManagedThreadId} | " +
                $"Group: {cnt.CategoryName} | " +
                $"Process: {cnt.InstanceName} | " +
                $"Name: {cnt.CounterName} | " +
                $"Value: {cnt.GetCalculatedValue(cnt.CounterName)}");

            cnt.Close();
        }
    }

I see that it is working properly. As I'm counting child cycle will run in a single thread with the parent cycle. But in console i see folowins. All mixed:

Thread ID: 15, Process Name: SearchFilterHost, Process ID 1552

ThreadId: 12 | Group: Process | Process: explorer | Name: % User Time | Value: 0
ThreadId: 13 | Group: Process | Process: mqsvc | Name: % User Time | Value: 0
ThreadId: 14 | Group: Process | Process: SearchProtocolHost | Name: % Processor Time | Value: 0
ThreadId: 4 | Group: Process | Process: RtWLan | Name: % Privileged Time | Value: 0,599904
ThreadId: 15 | Group: Process | Process: SearchFilterHost | Name: % Processor Time | Value: 0
ThreadId: 11 | Group: Process | Process: VsHub | Name: % Privileged Time | Value: 0
Thread ID: 16, Process Name: AAM Updates Notifier, Process ID 1280

ThreadId: 12 | Group: Process | Process: explorer | Name: % Privileged Time | Value: 0,142615
ThreadId: 15 | Group: Process | Process: SearchFilterHost | Name: % User Time | Value: 0
ThreadId: 14 | Group: Process | Process: SearchProtocolHost | Name: % User Time | Value: 0
ThreadId: 13 | Group: Process | Process: mqsvc | Name: % Privileged Time | Value: 0
ThreadId: 4 | Group: Process | Process: RtWLan | Name: Virtual Bytes Peak | Value: 1,957683E+08
ThreadId: 16 | Group: Process | Process: AAM Updates Notifier | Name: % Processor Time | Value: 0

What should I do for grouped output by ThreadID?

score 1 · Answer 1 · edited May 23 '17 at 10:34

You have a few options:

Use Console.WriteLine only once per thread. Console.WriteLine is thread-safe, so if you use thread to build your output string and call Console.WriteLine at the end it should be enough.
Use some thread-synchronization construct explicitly over all your Console.WriteLine calls - like lock (consoleLock) { Console.WriteLine(...); for (...) Console.WriteLine(...)}. Though building the entire output string at once, like described in previous option, is a better idea - you should lock for the shortest possible interval, because if you put your entire thread start delegate in a lock it won't be much faster than a single-threaded solution, and with a lot of thread pool threads waiting for a shared resource it would be unwise and even harmful from a performance point of view.
If you want to do something fancier, then you can try some consumer-producer implementation (like in Synchronizing events from different threads in console application) where only one thread writes messages from other threads. It is not much different from options 1 and 2, because it still would be WriteCounter's responsibility to send data only once, so

If you want to overengineer it even further, you can combine consumer-producer with per-thread buffering in some

public interface IBatchTextWriter : 
{
    TextWriter TextWriter { get; }
    /// <summary>
    /// Ends the batch (everything written to the <see cref="TextWriter"> will be written to the actual stream).
    void EndBatch();
}

And use it like:

var process = (ProcessInstance) obj;
batchWriter.TextWriter.WriteLine(
    $"Thread ID: {Thread.CurrentThread.ManagedThreadId}, " +
    $"Process Name: {process.ProcessName}, " +
    $"Process ID {process.ProcessId} ");
batchWriter.TextWriter.Write("\n");

foreach (var cnt in process.Counters)
{
    cnt.NextValue();
    batchWriter.TextWriter.WriteLine(
        $"ThreadId: {Thread.CurrentThread.ManagedThreadId} | " +
        $"Group: {cnt.CategoryName} | " +
        $"Process: {cnt.InstanceName} | " +
        $"Name: {cnt.CounterName} | " +
        $"Value: {cnt.GetCalculatedValue(cnt.CounterName)}");

    cnt.Close();
}

batchWriter.EndBatch();

P.S.: Recommended reading about ThreadPool and Tasks - C# - ThreadPool vs Tasks.

How to group data logging in multithreading?

1 Answers1