Why do async unit tests fail when the async/await keywords aren't used?

Question

According to this discussion, there should be no difference between the following two methods:

public async Task Foo()
{
    await DoSomethingAsync();
}

public Task Foo()
{
    return DoSomethingAsync();
}

Actually, it would seem that for very simple methods, the invocation without the async/await keywords would be preferred, as they remove some overhead.

However, this apparently doesn't always work in unit tests.

MSTest

[TestClass]
public class AsyncTest
{
    [TestMethod]
    public async Task Test1()
    {
        await Task.Delay(0);
    }

    [TestMethod]
    public Task Test2()
    {
        return Task.Delay(0);
    }
}

NUnit

[TestFixture]
public class AsyncTest
{
    [Test]
    public async Task Test1()
    {
        await Task.Delay(0);
    }

    [Test]
    public Task Test2()
    {
        return Task.Delay(0);
    }
}

XUnit

public class AsyncTest
{
    [Fact]
    public async Task Test1()
    {
        await Task.Delay(0);
    }

    [Fact]
    public Task Test2()
    {
        return Task.Delay(0);
    }
}

• In all cases, Test1 passes.
• In MSTest, Test2 shows up in the test runner, but it doesn't run.

• In NUnit, Test2 is ignored, with the message:

  Test method has non-void return type, but no result is expected

• In XUnit, Test2 passes.

Since the tasks are still awaitable in all cases, what is it about the async keyword that affects the NUnit and MSTest test runners? Perhaps some reflection issue?

Answer 1

It sounds like those test runners may be using reflection to check whether the method returning Task really is an async method. That doesn't mean the method would behave differently if they were run - but they're just not being run.

It's like saying that:

public string Name { get; set; }

is equivalent to:

private string name;
public Name { get { return name; } set { name = value; } }

They're logically the same in terms of behaviour, but if you try hard enough with reflection, you can tell the difference. In this particular case there are other more subtle differences, but the same general principle applies.

It looks like in the current NUnit code (at the time of this writing) the detection is in AsyncInvocationRegion.cs.

Admittedly it's at least unusual to write a unit test returning a Task but without using an async method - but far from impossible.