vb.net multithreading with multiple threads in a For..Next Loop

Question

Hi folks I have the code below which i wanted to use all my processors cores. Originally, the code in the sub chekFileDupe was all within the For..Next loop in SearchForDupes sub and worked just fine on one thread. I'm a bit stuck as to create 4 threads that execute the checkFileDupe code so that i can check 4 items at a time, then wait till all 4 threads have completed. Then carry on with the next iteration of the loop.

This is the original code

Private Sub SearchForDupes()

    Me.Refresh()
    Dim totalrecords As Integer = dvSortedSearchResults.Count
    Label1.Text = "Searching for duplicates"
    Label1.Refresh()
    dvSortedSearchResults.Sort = DataSortOrder
    For i = 0 To totalrecords - 6 Step 4
        Dim TrackDifference As Integer
        Dim originalTrack, originalArtist, trackToCompare, artistToCompare As String
        If i / 10 = CInt(i / 10) Then
            Label2.Text = i.ToString + " of " + totalrecords.ToString
            Label2.Refresh()
        End If
        originalArtist = mp3record(i, "Artist")
        originalTrack = mp3record(i, "Track")
        Dim ii As Integer = i + 1
        While ii < totalrecords - 2
            artistToCompare = mp3record(ii, "Artist")
            trackToCompare = mp3record(ii, "Track")
            TrackDifference = Difference(originalTrack, trackToCompare)
            dvSortedSearchResults(ii).Item("Difference") = TrackDifference
            'dgvSearchResults.Rows(ii).Cells("Difference").Value = trackdiff

            If Difference(originalArtist, artistToCompare) < 6 Then
                TrackDifference = Difference(originalTrack, trackToCompare)
                If TrackDifference < 4 Then
                    dvSortedSearchResults(i).Item("Difference") = 999
                    dvSortedSearchResults(ii).Item("Difference") = TrackDifference
                    dvSortedSearchResults(ii).Item("chkdupe") = True
                End If
            Else
                Exit While
            End If
            ii = ii + 1
        End While
    Next
    Label2.Text = ""
    Label1.Text = ""
End Sub

This is the first attempt at multithreading - probably naive but hey - everyone's new at something

Private Sub SearchForDupes()
        'Dim query = (From record In dvSortedSearchResults Where record.Artist = "Abba" Select record).ToList

        Me.Refresh()
        Dim totalrecords As Integer = dvSortedSearchResults.Count
        Label1.Text = "Searching for duplicates"
        Label1.Refresh()
        dvSortedSearchResults.Sort = DataSortOrder
        Dim params(2) As Integer
        For i = 0 To totalrecords - 6 Step 4
            params(2) = totalrecords
            params(1) = i
            thread1 = New System.Threading.Thread(Sub() checkFileDupe(params))
            thread1.Start()
            params(1) = i + 1
            thread2 = New System.Threading.Thread(Sub() checkFileDupe(params))
            thread2.Start()
            params(1) = i + 2
            thread3 = New System.Threading.Thread(Sub() checkFileDupe(params))
            thread3.Start()
            params(1) = i + 3
            thread4 = New System.Threading.Thread(Sub() checkFileDupe(params))
        Next
        Label2.Text = ""
        Label1.Text = ""
    End Sub

    Private Sub checkFileDupe(params As Array)
        Dim i As Integer = params(1)
        Dim totalrecords As Integer = params(2)
        Dim TrackDifference As Integer
        Dim originalTrack, originalArtist, trackToCompare, artistToCompare As String
        If i / 10 = CInt(i / 10) Then
            Label2.Text = i.ToString + " of " + totalrecords.ToString
            Label2.Refresh()
        End If
        originalArtist = mp3record(i, "Artist")
        originalTrack = mp3record(i, "Track")
        Dim ii As Integer = i + 1
        While ii < totalrecords - 2
            artistToCompare = mp3record(ii, "Artist")
            trackToCompare = mp3record(ii, "Track")
            TrackDifference = Difference(originalTrack, trackToCompare)
            dvSortedSearchResults(ii).Item("Difference") = TrackDifference
            'dgvSearchResults.Rows(ii).Cells("Difference").Value = trackdiff

            If Difference(originalArtist, artistToCompare) < 6 Then
                TrackDifference = Difference(originalTrack, trackToCompare)
                If TrackDifference < 4 Then
                    dvSortedSearchResults(i).Item("Difference") = 999
                    dvSortedSearchResults(ii).Item("Difference") = TrackDifference
                    dvSortedSearchResults(ii).Item("chkdupe") = True
                End If
            ii = ii + 1
        End While
    End Sub     Else
                Exit While
            End If

You are starting, in total, `(totalrecords-6)/4 * 4` threads, namely 4 new threads per iteration of the `For` loop. Is that what you intended? — MicroVirus, Oct 15 '15 at 17:10
hehe - oops - no. Create 4 threads and destroy them before the next iteration i guess - tho this may be very inefficient — David Wilson, Oct 15 '15 at 17:12
The way you are dividing the work among the threads is not correct. You need to think of how you want to divide the task of checking every element against every other element. — MicroVirus, Oct 15 '15 at 17:26

MicroVirus · Accepted Answer · 2015-10-15T17:19:12.223

Actually, what I might advise you to do is forget about the code you have now and use a BackgroundWorker control. Start with just creating one thread to do the work and get that up and running. The advantage is that the background worker allows you to report progress back to the main UI thread. There should be plenty of tutorials out there.

I can try to tackle/indicate some of the problems in your current code, but to be honest there is a lot wrong with it.

You have a problem where you use the same underlying object, params(2), for each thread you create, so when you modify params you thereby modify the value that all the threads see. What you need to do is create a new array for each time you want to pass an argument to a new thread. It's also extra confusing because you are using closures Sub () checkFileDupe(params), rather than using a correct signature for checkFileDupe and using Thread.Start to pass the argument.

I'd advise you to create a Structure to hold the arguments to your thread:

Private Structure FileDupeArguments
    Public StartIndex As Integer
    Public TotalRecords As Integer
End Structure

Then you can create a new thread via:

Dim params As FileDupeArguments
...
thread1 = New System.Threading.Thread(AddressOf checkFileDupe)
params = New FileDupeArguments With {.StartIndex = i, .TotalRecords = totalrecords}
thread1.Start(params)

And then declaring checkFileDupe as:

Sub checkFileDupe(argObj as Object)
    Dim args As FileDupeArguments = CType(argObj, FileDupeArguments)
    Dim i As Integer = args.StartIndex
    Dim totalRecords As Integer = args.TotalRecords
    ...
End Sub

The important part here is that you send a new copy of FileDupeArguments to each thread. Also, no closures are needed.

There is an issue with you accessing controls from the threads you created. For instance,

Label2.Text = i.ToString + " of " + totalrecords.ToString
Label2.Refresh()

will not work on a background thread and will give you errors. I'd advise you to not do direct progress reports from your worker threads. Like I mentioned, a BackgroundWorker will allow you to report back progress using events.

All the code that accesses dvSortedSearchResults suffers from the same or similar problems. If you access something from multiple threads, you need to apply locks. This is already more advanced and beyond the scope of this answer to explain.

Note that my VB.NET is a bit rusty, so it's possible that I have made a small mistake in the code snippet. Please let me know. — MicroVirus, Oct 15 '15 at 17:13
@DavidWilson I've updated my answer. I suggest you start with one thread using a background worker control. Get that up and running first. The current code has too many issues to address. — MicroVirus, Oct 15 '15 at 17:19
Thanks - yeah it is messy code. I wrote it quite a while ago before i learned more - Maybe i should sort the original code out first :) Just as a final thing for the moment.. do i need to somehow destroy the threads at the end of each iteration or do they get destroyed as (i think maybe) they go out of scope as the loop iterates? — David Wilson, Oct 15 '15 at 17:24
@DavidWilson You don't need to destroy them, they'll clean-up once they are done. See [Do we need to dispose or terminate a thread in C# after usage?](http://stackoverflow.com/questions/10698107/do-we-need-to-dispose-or-terminate-a-thread-in-c-sharp-after-usage) — MicroVirus, Oct 15 '15 at 17:29
After digging around about backgroundworkers, I found out about parallel.For - with a bit of code alteration to synclock the dv, it worked like a charm. Thanks for the time and effort :) — David Wilson, Oct 17 '15 at 08:04

vb.net multithreading with multiple threads in a For..Next Loop

1 Answers1