ConcurrentDictionary AddOrUpdate method throwing IndexOutOfRangeException

Question

Jobs are getting added to the HashSet by different threads and throwing this error. Is there any solution for this ?

ConcurrentDictionary<myKey, HashSet<Job>> _dictKeyJob;

_dictKeyJob.AddOrUpdate(myKey, key =>
{
    return new HashSet<Job>({ Job };
}, (key, hashJobs) =>
{
    if (Job.Status == eStatus.Cancelled)
    {
        hashJobs.Remove(Job);
    }
    else
    {
        hashJobs.Add(Job);
    }
    return hashJobs;
});

Exception:

System.IndexOutOfRangeException: Index was outside the bounds of the array.
   at System.Collections.Generic.HashSet`1.SetCapacity(Int32 newSize, Boolean forceNewHashCodes)
   at System.Collections.Generic.HashSet`1.AddIfNotPresent(T value)
   at Raj.OPS.Common.Test.<>c__DisplayClass38_0.<SetOrAddKey>b__1(mKey key, HashSet`1 hashJobs) in 
   at System.Collections.Concurrent.ConcurrentDictionary`2.**AddOrUpdate**(TKey key, Func`2 addValueFactory, Func`3 updateValueFactory)

It would be awesome if you could provide a [mcve]. My crystal ball suggests that you are storing a `HashSet` as the values inside a `ConcurrentDictionary`. A `ConcurrentDictionary` does not magically make a `HashSet` thread-safe. — mjwills, Nov 08 '19 at 22:43
https://stackoverflow.com/questions/4307131/indexoutofrangeexception-when-adding-to-hashsett is likely the appropriate duplicate. — mjwills, Nov 08 '19 at 22:44

score 1 · Answer 1 · answered Nov 14 '19 at 23:50

From the documentation of the ConcurrentDictionary.AddOrUpdate method:

For modifications and write operations to the dictionary, ConcurrentDictionary<TKey,TValue> uses fine-grained locking to ensure thread safety. (Read operations on the dictionary are performed in a lock-free manner.) However, the addValueFactory and updateValueFactory delegates are called outside the locks to avoid the problems that can arise from executing unknown code under a lock. Therefore, AddOrUpdate is not atomic with regards to all other operations on the ConcurrentDictionary<TKey,TValue> class.

(emphasis added)

So you can't use HashSet as the value of the ConcurrentDictionary, and update it from multiple threads without protection. It will become corrupted, and start throwing random exceptions like the ones you observe. You should either protect it with a lock (using a different locking object for each HashSet to reduce contention), or use a concurrent HashSet (there is no ConcurrentHashSet class so you must use a nested ConcurrentDictionary).

Regarding the first option, the one involving a lock, you should use the same locking object everywhere you access the same HashSet, not only inside the callback of the AddOrUpdate method.

It is possible though that all this synchronization that adds overhead to your application could be removed by using a workflow approach (the approach favored by the TPL Dataflow library). Or it may not be possible. It depends on the specifics of what you are doing.

ConcurrentDictionary AddOrUpdate method throwing IndexOutOfRangeException

1 Answers1