How do I unit test my threaded vb.net application?

Question

I'm sure once the test is written, it'll point out some things that will need to be changed in order to get everything working with the UI, but basically I have this:

frmMain:

Dim totalFinished as Integer = 0
reporter as Func(Of Object) = Function()
                                totalFinished += 1
                                return Nothing
                              End Function
classWithAsync.setReporter(reporter)
classWithAsync.BeginCalculation() ' ignore that this is a private method call
' there is another method I'm leaving out (the public one) which just does
' some set up stuff and then calls BeginCalculation
' Note: this notice isn't actually in my code

ClassWithAsync:

Private Async Sub BeginCalculation()
  ' some logic here
  If synchronous Then
    CalculatorCalculate(calculator)
  Else
    Await calculatorAsyncCalculate(calculator)
  End If 
End Sub

Private Async Function calculatorAsyncCalculate(ByVal calculator as Calculatable) as Hashtable
  Dim result as Hashtable

  result = Await Tasks.Task.Run(Function() as Hashtable
                                 return calculator.Calculate()
                                EndFunction)

  reporter.Invoke()
  return result
End Function

Private Function CalculatorCalculate(ByVal calculator as Calculatable) as Hashtable
  Dim result as Hashtable
  result = calculator.Calculate()

  reporter.Invoke()
  return result
End Function

The intention here is to have reporter invoke twice. Which I have working in a synchronous version.. but the threaded test acts as if the thread didn't even run (which might be because the execution is continuing, and the assert is evaluating before the threads are finished?)

Here are my tests:

Synchronous: (Passes)

  <TestMethod> Public Sub UpdatesForEachSynchronousProduct() 
    Dim data As Hashtable = TestData.GenerateGroupData("LTD,WDL") 

    CreateMockCalculators(data, privateTarget) 

    privateTarget.SetFieldOrProperty("reporter", reporter) 

    privateTarget.SetFieldOrProperty("synchronous", True) 
    privateTarget.Invoke("BeginCalculation") 

    Assert.AreEqual(2, totalCalculated) 
  End Sub

Async: (Doesn't Pass)

<TestMethod> Public Sub UpdatesForEachAsyncProduct() 
    Dim data As Hashtable = TestData.GenerateGroupData("LTD,WDL") 

    CreateMockCalculators(data, privateTarget) 

    privateTarget.SetFieldOrProperty("reporter", reporter) 

    privateTarget.SetFieldOrProperty("synchronous", False) 
    privateTarget.Invoke("BeginCalculation") 

    Assert.AreEqual(2, totalCalculated) 
  End Sub

The error for this one is that it expected 2, but totalCalculated was 0.

So, is there a way to make Assert wait until the threads are done?

You might find this helpful: http://stackoverflow.com/q/12159/951890 — Vaughn Cato, Mar 19 '15 at 14:10

score 0 · Accepted Answer · answered Mar 19 '15 at 16:07

Figured it out.
Well, I figured something out.
Though, this kind of a hacky way to do it. At this moment, I'm unsure if it's still async anymore, Maybe someone with more async experience than me can comment on that. I'm pretty sure it's still Async. I'll know more when I get further in my implementation of my overall project.

Had to change the BeginCalculation() method:

  Private Sub BeginCalculation() 
    For Each calculator As Calculatable In calculatables 
      If synchronous Then 
        CalculatorCalculate(calculator) 
      Else 
        ' Call this without Await so that the for each loop 
        ' may continue spawning calculation tasks 
        ' Calling Await halts the execution of this method 
        CalculatorAsyncCalculate(calculator) 
      End If 
    Next 

  End Sub

Before, I was using Await on CalculatorAsyncCalculate, which halted execution of the loop until the thread finished - which is undesired. Visual Studio complains about this (as a warning) and suggests that I add Await and add Async to the method header. But.. we can ignore visual studio here.

Here is my async method:

 Private Async Function CalculatorAsyncCalculate(ByVal calculator As Calculatable) As Tasks.Task(Of Hashtable) 
    Dim result As Hashtable 

    Dim task As New Tasks.Task(Function() As Hashtable 
                                 calculator.Calculate() 

                                 ' Update the reporter 
                                 If Not reporter Is Nothing Then 
                                   reporter.Invoke() 
                                 End If 

                                 Return calculator.GetCalculatedData() 
                               End Function) 

    calculationTasks.Add(task) 

    result = Await Tasks.Task.Run(Function() 
                                    ' Begin calculations 
                                    task.Start() 

                                    ' Wait until the calculations are done 
                                    ' before returning the results 
                                    task.Wait() 

                                    Return calculator.GetCalculatedData() 
                                  End Function) 

    Return result 
  End Function

Instead of just running what I need to asynchronously, I need to put that code in to a Task object, so that I can add it to an ArrayList which keeps track of the tasks. The purpose of that array is to have something that I can pass in to Threading.Tasks.Task.WaitAll(...) in my test. Kind of convoluted, but I guess it gives more flexibility with what to do with the individual tasks in the future.

And then my Test:

<TestMethod> Public Sub UpdatesForEachAsyncProduct() 
    Dim data As Hashtable = TestData.GenerateGroupData("LTD,WDL") 

    CreateMockCalculators(data, privateTarget) 

    privateTarget.SetFieldOrProperty("reporter", reporter) 

    privateTarget.SetFieldOrProperty("synchronous", False) 
    privateTarget.Invoke("BeginCalculation") 

    Dim calcTasks As ArrayList = privateTarget.GetFieldOrProperty("calculationTasks") 
    Dim cTasks(calcTasks.Count - 1) As System.Threading.Tasks.Task 
    Dim array() = calcTasks.ToArray() 
    array.CopyTo(cTasks, 0) 

    Assert.AreEqual(2, cTasks.Count) 

    System.Threading.Tasks.Task.WaitAll(cTasks) 

    Assert.AreEqual(2, totalCalculated) 
  End Sub

privateTarget is a PrivateObject(classWithAsync) so I can get at private properties and methods.

Before calling wait all, I'm just making sure that I have the correct number of tasks, to ensure that BeginCalculation has iterated over both objects, and spawned two threads.

The WaitAll(cTasks) call is the magical bit here. Without I get a failure saying that Assert expected 2 but received 1.

score -1 · Answer 2 · answered Mar 19 '15 at 14:29

-1

One of the fundamental features of a good unit test is that it should test a Unit; that is it should test a single, self-contained, independent feature. Multiple threads, by their very nature, are not independent of each other, and can interact in ways that are not easily reproducible for testing.

Based on that I would argue that you should be testing the actual calculation (calculator.Calculate()) outside of the threading.

Trying to test the calculation in the presence of threading is trying to test too many things at the same time, IMHO.

answered Mar 19 '15 at 14:29

Bradley Uffner

16,641
3
39
76

I have a test for the `Calculate()` method. I just want to test that I set up threading correctly, and that any changes to how the threading is implemented doesn't affect the end result. Tests should be able to cover all aspects of an application. Maybe a more appropriate term would be integration test. – NullVoxPopuli Mar 19 '15 at 14:33

How do I unit test my threaded vb.net application?

2 Answers2