0

I have a unit test which randomly fails. I would expect this unit test to always work. This is the test code (as you can see I'm using NUnit as the testing framework):

using NUnit.Framework;

[TestFixture]
public class TestClass
{
    [Test]
    public async Task Test()
    {
        var start = DateTimeOffset.UtcNow;
        await Task.Delay(TimeSpan.FromMilliseconds(500)).ConfigureAwait(false);
        var end = DateTimeOffset.UtcNow;

        Assert.GreaterOrEqual((end - start), TimeSpan.FromMilliseconds(500));
    }
}

This is the csproj file of the test project:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net6.0</TargetFramework>
      <IsPackable>false</IsPackable>
    <AnalysisMode>All</AnalysisMode>
    <Nullable>enable</Nullable>
    <ImplicitUsings>enable</ImplicitUsings>
  </PropertyGroup>
    <ItemGroup>
        <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.4.1" />
        <PackageReference Include="NUnit" Version="3.13.3" />
        <PackageReference Include="NUnit3TestAdapter" Version="4.3.1" />
    </ItemGroup>
</Project>

The failure happens on both Windows 10 development machine and Ubuntu 20.04.5 machine (executed via WSL2 on the Windows 10 host). There is a difference between the observed behavior: test failure happens much more frequently on Linux. Failures in Windows are unusual, but sometimes it happens on Windows too.

Does anyone have an explanation for this weird behavior ? Based on my knowledge and understanding, this doesn't make sense.

Enrico Massone
  • 6,464
  • 1
  • 28
  • 56
  • Lol, why you think that they would be equal? Windows and Linux are not RTOS – Selvin Feb 03 '23 at 17:26
  • 1
    @Selvin from the fairness standpoint OP was expecting the elapsed time to be greater or equal though. – Guru Stron Feb 03 '23 at 17:33
  • 1
    It seems that you are struggling with clock precision. `DateTime` and `DateTimeOffset` are not an appropriate tool for benchmarking/measuring small periods of time. – Guru Stron Feb 03 '23 at 17:37
  • 1
    Philosophically, this would be better fit for statistics SE - you take 3 know to be imprecise values and try to confirm particular relationship between those. Some good discussion can be had to see how often that condition should pass... (For practical purposes, giving a range of couple smallest timslices - 16.6ms on Windows, not sure if WSL gets different value - would do... or just +/-100ms should do for most cases) – Alexei Levenkov Feb 03 '23 at 17:52
  • @AlexeiLevenkov I put the very same code in a for loop in order to repeat it many times. I executed the loop in a console app. I never noticed an unexpected result by doing that. It seems that being under the test framework does influence this randomness – Enrico Massone Feb 03 '23 at 17:59
  • 1
    Anyway I got the point. I'm using the wrong tools for my purpose. To be honest, this is somewhat documented in Microsoft docs, so that's clear why it is not matching my expectations. Thanks for all the comments and for reporting the duplicate. – Enrico Massone Feb 03 '23 at 18:01

0 Answers0