0

Am I missing something here? I've tried putting a set value such as 1 or 4 in the "MaxDegreeOfParallelism =" option but in the example below it is being assigned by and Up/down selector.

Is there a reason my MaxDegreeOfParallelism Parallel option isn't working? Right now the program seems to use as many thread as it wants. I also have a memory leak big time and I am assuming this is due to the Parallel.ForEach loop because the leak wasn't there before.

private void NumericUpDown1_ValueChanged(object sender, EventArgs e)
       {

           threadCount = threadCountUD.Value;
           threadC = Decimal.ToInt32(threadCount);
       }




       private void MainWork()
       {
           var maxThread = new ParallelOptions
           {
               MaxDegreeOfParallelism = threadC,

           };


           var cookies = new CookieContainer();
           proxies = File.ReadAllLines(proxPath);
           fileInfo = File.ReadAllLines(path);
           var proxAndNames = proxies.Zip(fileInfo, (n, w) => new { Proxies = n, Name = w });
           bool repeat = true;

           Parallel.ForEach(proxAndNames.AsParallel().ToArray(),maxThread,  async (var) =>
         { while (repeat)
             {


                   if (stopBtn is true)
                 {
                     break;
                 }


                 var httpClientHandler = new HttpClientHandler
                 {
                     Proxy = new WebProxy(var.Proxies),
                     AllowAutoRedirect = true,
                     CookieContainer = cookies,
                     UseCookies = false,
                     UseDefaultCredentials = false,
                     AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate,
                     UseProxy = true
                 };

                 using (var client = new HttpClient(httpClientHandler, false))
                 {
client.DefaultRequestHeaders.Add("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:70.0) Gecko/20100101 Firefox/70.0");
                     client.BaseAddress = new Uri("https://www.google.com/");
                     var cts = new CancellationTokenSource();
                     try

             {
  HttpResponseMessage response = await client.GetAsync(var.Name.AsParallel() + "/");
                         response.EnsureSuccessStatusCode();

                         if (response.IsSuccessStatusCode)
                         {
                             string grabbed = await client.GetStringAsync(var.Name.AsParallel() + "/");

}

}

                     catch (TaskCanceledException)
                     {
                         if (!cts.Token.IsCancellationRequested)
                         {

                         }
                         else
                         {
                               // Cancelled for some other reason
                           }
                     }

                     catch (HttpRequestException ex)

                     {
                         proxBox.Invoke(new Action(() => proxBox.Text += ex.Message + System.Environment.NewLine));

                     }
                     finally
                     {

                        client.Dispose();
                        cts.Dispose();
                     }

                 }


             }

         });


       }
Jack Jones
  • 45
  • 6
  • 2
    Please review [MCVE] guidance on posting code. In particular "//get requests going on here" seem to be the most interesting place where you are mixing tasks (`async`/`await`) with `Parallel.ForEach`... – Alexei Levenkov Dec 18 '19 at 06:07
  • @AlexeiLevenkov Thank you for your reply. I will edit and fix the issue now :) – Jack Jones Dec 18 '19 at 06:11
  • Its pretty clear that you have huge memory usage of your code.` Parallel` making his own tasks over and over again. making `Task` is pretty costly in memory – michasaucer Dec 18 '19 at 06:20
  • 4
    You cannot use `Parallel.ForEach` with an async method. Use `Task.WhenAll`. See one of those Q&A's: https://stackoverflow.com/questions/15136542/parallel-foreach-with-asynchronous-lambda, https://stackoverflow.com/questions/53497077/passing-async-method-into-parallel-foreach, https://stackoverflow.com/questions/23137393/parallel-foreach-and-async-await. Since you are actually starting some fire and forget actions the `MaxDegreeOfParallelism` won't do what you expect since the body `Action` is immediately finished. It cannot and will not be awaited. – Peter Bons Dec 18 '19 at 07:10
  • Using the `Parallel` class for I/O-bound workloads is highly problematic. Also take a look at [this](https://learn.microsoft.com/en-us/dotnet/api/system.net.http.httpclient#remarks): *`HttpClient` is intended to be instantiated once and re-used throughout the life of an application. Instantiating an `HttpClient` class for every request will exhaust the number of sockets available under heavy loads.* – Theodor Zoulias Dec 18 '19 at 09:21

0 Answers0