Is there a way to runtime-conditionally Parallelise a query that produces anonymous types?

Question

Edit: mostly solved below. However, one other thing I'd be curious about (to improve google-fu in the future is - how would your phrase this question? I'm aware my title is a bit weird but could not think of a more concise way to summarise it. Suggestions in comments are welcome.

Suppose my workflow involves a potentially expensive selection. At runtime I'd like to be able to choose whether or not to make this parallel.

To explain better. Consider this query (broken code - do-what-i-want-not-what-i-say):

void Main()
{
    bool runTimeParallelOption = false; // assume this option comes from e.g. user/config

    var j = Enumerable.Range(1,5);

    if (runTimeParallelOption)
        j = j.AsParallel();

    var results = j.Select(x => 
    {
        Thread.Sleep(1000);
        return new { Number = x, Square = x * x };
    });

    var sw = Stopwatch.StartNew();
    foreach (var result in results)
        Console.WriteLine(result.Square);

        Console.WriteLine("Elapsed: : " + sw.Elapsed);
}

The hope here is that if runTimeParallelOption is set, it would make the query parallel. However obviously (in hindsight!) this doesn't work because the Type of j means I am still executing the IEnumerable extension method.

The fix for this is trivial if you don't mind refactoring to make { Number = x, Square = x * x } a first-class Type, but this isn't very appealling to me. Is there a way to have my cake and eat it?

svick · Accepted Answer · 2014-04-28T01:38:40.957

1

I think the simplest option here it to always use AsParallel(). Then, if you don't want to parallelize it, add WithDegreeOfParallelism(1).

edited Apr 28 '14 at 01:38

answered Apr 08 '14 at 17:22

svick

236,525
50
385
514

score 0 · Answer 2 · edited May 23 '17 at 12:28

Inspired somewhat by this, the best I've come up with is this:

void Main()
{
    bool runTimeParallelOption = true; // assume this option comes from e.g. user/config

    var j = Enumerable.Range(1,5);

    var results = Enumerable.Empty<int>().Select(x => new { Number = (int)0, Square = (int)0 }).Take(0);

    var mutate = InferredProjection(1, results.FirstOrDefault(),  x => { Thread.Sleep(1000); return new { Number = x, Square = x * x }; });

    var sw = Stopwatch.StartNew();
    if (runTimeParallelOption)
        results = j.AsParallel().Select(mutate);
    else
        results = j.Select(mutate);
    sw.Stop();

    foreach (var result in results)
        Console.WriteLine(result.Square);

    Console.WriteLine("Elapsed: : " + sw.Elapsed);
}


static Func<T, U> InferredProjection<T, U>(T infer1, U infer, Func<T, U> project)
{
    return project;
}

Is there a way to runtime-conditionally Parallelise a query that produces anonymous types?

2 Answers2