How get current object in operation parallel Thread

Question

I am List of Object for send to api .i used parallel thread below.
code

  List<object> data  ;//contain data

                result =new  Dictionary<decimal, object>();
                

                var threadCount = 4;
                if (data.Count < threadCount)
                {
                    threadCount = data.Count;
                }

                var pageSize = threadCount > 0 ? Convert.ToInt32(Math.Ceiling((Convert.ToDecimal(data.Count) / threadCount))) : 0;
                var pageCount = threadCount;

                

                for (int j = 0; j < threadCount; j++)
                {

                    var temp = data.Skip(j * pageSize).Take(pageSize).ToList();
                    var tempLength = temp?.Count;
                    Parallel.ForEach(temp, item =>
                    {
                        result.Add(item.ID, null);

                       //call Api and get resultApi

                        if (resultApi != null && resultApi.Result != null)
                        {
                            
                            result[item.ID] = resultApi.Result.id;
                        }
                        else if (resultApi != null && resultApi .Message != null)
                        {
                            
                            result[item.ID] = null;
                        }
                        else
                        {
                            result[item.ID] = null;
                        }
                    });
                 }

problem
in end operation in top when check result i see some items are not related to their ID and have been moved.If there is no shift when I exit the parallel mode, all the identifiers are set correctly how resolve problem?

Where to start. `Dictionary` is not threadsafe. So you would need to change that to a datastructure that is threadsafe (`ConcurrentDictionary` seems like a good fit). That said how you use that datastructure also needs to be threadsafe and it isn't either. Better might be to first determine what you want to enter into the dictionary locally and then in 1 action at the end write that to the `ConcurrentDictionary`. Hard to say though without knowing what exactly is going on and what you want to achieve — , Mar 15 '21 at 11:13
@Knoop thank's a lot. I want to have a list of objects to be stored in the database as a bulk at the end of the operation — alireza, Mar 15 '21 at 11:26
`//call Api and get resultApi` -> Is this an asynchronous call? — Johnathan Barclay, Mar 15 '21 at 11:57

score 1 · Accepted Answer · answered Mar 15 '21 at 12:56

My suggestion is to use PLINQ instead of the Parallel class. It is a safer tool to use, for entry-level multithreading. The PLINQ is like LINQ, but it starts with .AsParallel(). It includes almost all of the familiar LINQ operators like Select, Where, Take, ToList etc.

Dictionary<decimal, object> dictionary = data
    .AsParallel()
    .WithDegreeOfParallelism(4)
    .Cast<Item>()
    .Select(item => (item.ID, CallAPI(item).Result))
    .Where(entry => entry.Result != null)
    .ToDictionary(entry => entry.ID, entry => (object)entry.Result.Message);

The CallAPI method is assumed to have this signature: Task<APIResult> CallAPI(Item item);

This PLINQ query will process your data with a concurrency level of 4. This means that 4 operations will be concurrently in flight, and when one item completes the next one will start automatically.

The IDs are expected to be unique, otherwise the ToDictionary operator will throw an exception.

This approach is suggested for its simplicity, not for its efficiency. The PLINQ is not really intended for processing I/O bound workloads, and will needlessly block ThreadPool threads while doing so. You can look here for more efficient ways to throttle asynchronous I/O bound operations.

Thank's a lot.please check [answer](https://stackoverflow.com/questions/66636528/how-get-current-object-in-operation-parallel-thread#answer-66651130) and express your opinion. — alireza, Mar 16 '21 at 08:05

score 0 · Answer 2 · answered Mar 16 '21 at 08:02

I have reached an answer. Which is as follows.
Code

List<object> data  ;//contain data

                result =new  Dictionary<decimal, object>();
                

                var threadCount = 4;
                if (data.Count < threadCount)
                {
                    threadCount = data.Count;
                }

                var pageSize = threadCount > 0 ? Convert.ToInt32(Math.Ceiling((Convert.ToDecimal(data.Count) / threadCount))) : 0;
                var pageCount = threadCount;

                

                for (int j = 0; j < threadCount; j++)
                {

                    var temp = data.Skip(j * pageSize).Take(pageSize).ToList();
                    var tempLength = temp?.Count;
                    Parallel.ForEach(temp, item =>
                    {
                      lock (result)
                        {
                           var temp = callapi(item.ID);
                           result.Add(temp.Item1, temp.Item2);
                        }
                    });
                 }
     private (decimal, object) callapi(decimal id){
                        //call Api and get resultApi

                        if (resultApi != null && resultApi.Result != null)
                        {
                            
                            result[item.ID] = resultApi.Result.id;
                            return (id,result);
                        }
                        else if (resultApi != null && resultApi .Message != null)
                        {
                            
                            result[item.ID] = null;
                            return (id,result);
                        }
                        else
                        {
                            result[item.ID] = null;
                            return (id,result);
                        }
}

By invoking the `callapi` inside the lock you are serializing the calls, essentially killing the parallelism. You should protect with the lock only the `result.Add` invocation. Other than that, your solution invokes multiple `Parallel.ForEach` loops sequentially, in a way that looks to me like an attempt to implement a custom partitioning scheme. The Task Parallel Library includes a dedicated `Partitioner` class, so implementing your own scheme seems like not a wise thing to do. It is unlikely that your custom partitioner will be better than the built-in one. — Theodor Zoulias, Mar 16 '21 at 09:02

How get current object in operation parallel Thread

2 Answers2