Task.Delay().Wait(); sometimes causing a 15ms delay in messaging system

Question

I have 3 programs talking to each other. One of my programs is giving me a strange issue, which is causing my latency measurements to be off.

All 3 applications use the same Custom communications library...

In the problematic application, when the program starts, a new task is started to continuously watch a queue for the arrival of a message:

public void Init()
{
    var ip = "127.0.0.1";
    CommunicationsManager.UdpManager.Initialize(ip, (int)UDPPort.FromUnityToPredictor, (int)UDPPort.FromPredictorToUnity);
    CommunicationsManager.UdpManager.UdpMessageReceived += OnMessageReceived;
    CommunicationsManager.UdpManager.StartListening();

    var task = Task.Run(() =>
    {
        ProcessMessageQueueThread();
    });

    //  Thread thread = new Thread(ProcessMessageQueueThread);
    //  thread.IsBackground = true;
    //  thread.Start();
}

private void ProcessMessageQueueThread()
{
    while (true)
    {
            if (MessageQueue.Count > 0)
            {
                ProcessMessage();
            }
    }
}

I a have a function subscribed to an event listener, which fires upon the arrival of a new UDP datagram:

private void OnMessageReceived(object sender, UDPMessage message)
{
    MessageQueue.Enqueue(message);
    //Task.Run(() =>
    //{
    //ProcessMessage();
    //});
}

Upon the function firing, the message is added to a BlockingCollection queue.

The message is then processed by ProcessMessage:

static Stopwatch sw = new Stopwatch();

private void ProcessMessage()
{
    var message = MessageQueue.Dequeue();
    messagesReceived++;
    sw.Restart();

    //if server side prediction, delay before reading...
    if (!message.ClientSidePrediction)
    {
        // Thread.Sleep(message.UpDelay);
        Task.Delay(message.UpDelay).Wait();
    }

    //If prediction must not be used...
    if (!message.UsePrediction)
    {
        message.IsAPredictedMessage = false;
        CommunicationsManager.UdpManager.Send(message);
        messagesSent++;
        return;
    }

    if (message.UsePrediction&& messagesReceived == 1)
    {

        message.IsAPredictedMessage = false;
        CommunicationsManager.UdpManager.Send(message);
        logger.Info("First message sent:" + sw.ElapsedMilliseconds);
        sw.Restart();
        messagesSent++;
    }

    message.IsAPredictedMessage = true;
    CommunicationsManager.UdpManager.Send(message);
    logger.Info("second message sent:" + sw.ElapsedMilliseconds);
    messagesSent++;

    // Console.WriteLine("Sent:" + messagesSent + ", Received: " + messagesReceived);

}

As can be seen, I am simulating a network delay in `ProcessMessage':

Task.Delay(message.UpDelay).Wait();

If I set message.UpDelay to 30, I will see a delay between sending messages of 45ms or 46ms... if I set the message.UpDelay to 1, I will see a sending delay of 15ms or 16ms...

However, most of the time the delay is correct... ie, 30ms/31 ms, or 1ms/2ms. Is seems that at some random (at least, to me it appears random) time, the delay takes an additional 15ms/16ms to complete...

What is going on? It is very important that my delays are consistent and accurate. I have tried using Thread.Sleep and creating threads rather than using tasks (can be seen in the code comments), but I do not see a difference.

Have you tried adding `.ConfigureAwait(false)` to your `Task.Delay` call? Pure speculation on my part, but I wonder if tracking and restoring the synchronization context is adding the delay that you're seeing... — Jesse Squire, Jan 24 '17 at 14:09
You're seeing an artifact of the Windows interrupt rate, which is (by default) approx every 15ms. Thus if you ask for 1-15ms, you'll get an approx 15ms delay. ~16-30 will yield 30ms... so on. — Glorin Oakenfoot, Jan 24 '17 at 14:10
@JesseSquire Just tried adding `await` before `Task`, made my `ProcessMessage` `async` and added what you suggested, but the results are the same. Did I do that correctly? — pookie, Jan 24 '17 at 14:17
@pookie Unfortunately, no. There's a big difference between `await Task.Delay(TimeSpan.FromMilliseconds(15));` and `Task.Delay(TimeSpan.FromMilliseconds(15)).ConfigureAwait(false)`. That said, Glorin's comment above is probably more salient than mine - if the interrupt rate is only accurate to 15ms, which seems plausible, then it's unlikely that you're going get more accurate without first tuning that (if that is possible) — Jesse Squire, Jan 24 '17 at 14:20
@JesseSquire What I mean is that I did `await Task.Delay(TimeSpan.FromMilliseconds(15)).ConfigureAwait(fal‌se)` — pookie, Jan 24 '17 at 14:27
@GlorinOakenfoot The problem is that I do not always get the 15ms extra delay. It appears to arise at random times...So, one message might be delayed an extra 15ms, but the next 7 messages or whatever may take exactly the time I required. — pookie, Jan 24 '17 at 14:29
1) Your `while (true)` loop is burning your CPU for no reason. Replace it with `BlockingCollection1.Take()` call that will block the current thread until new item is available. Threads that are blocked do not consume CPU resources. 2) Your `Task.Delay(message.UpDelay)` makes no sense if it is not asynchronous. Replace it with `Thread.Sleep`. 3) Windows is not a real-time OS, thus unpredictable delays when using `Sleep` are unavoidable. — Yarik, Jan 24 '17 at 14:31
@Yarik Thank you for your suggestions, I will try the recommended changes! — pookie, Jan 24 '17 at 14:33
@Servy Thank you for the duplicate question tag. While my question is different (not about sleeping for less than 1ms -- I gave 1ms as an example), the answer posted (http://stackoverflow.com/a/22223245/596841) did prove very useful. — pookie, Jan 24 '17 at 17:17
@pookie You specifically mentioned in your question that you get the same behavior when sleeping. The answer is the same for both cases; they have the same underlying cause. Note that even that answer still won't necessarily work, as the accepted answer states. You need to use an actual real time OS to *reliably* have timings to that level of precision. — Servy, Jan 24 '17 at 17:18
@Servy I'm not referring to the use of `Thread.Sleep`; I'm referring to the duration & result. While the underlying cause is the same, the questions are not. The linked question asks about waiting for less than 1ms and is aware of a limitation and is therefore asking for a workaround, whereas I was asking what the *cause* was. Regardless, thank you for your help. — pookie, Jan 24 '17 at 17:25
@Yarik I know this comment was quite a while ago, but it's worth noting that your suggestion to block the current thread instead of using a while loop really runs counter to what the TAP framework tries to accomplish, which is to *avoid* blocking threads. This is a particularly big deal in a server context where a blocked thread is a resource that cannot be utilized. A while(true) loop with an appropriate Task.Delay(1) or Task.Yield() statement in it will not burn CPU, but will make it possible for the thread to be fully utilized, rather than blocked. — Craig Tullis, Mar 14 '22 at 23:31
In general (there are always exceptions, maybe), you don't want to mix Thread.Sleep and Tasks. Use Task.Delay and/or Task.Yield instead. At the very least consider using Thread.Yield so that the thread is not blocked and unable to do useful work. — Craig Tullis, Mar 14 '22 at 23:33

Task.Delay().Wait(); sometimes causing a 15ms delay in messaging system

0 Answers0