What determines if a variable is closed over?

Question

//var sample= new { bottom=2, top=5,count=4};
var sample= new [] {2,3,4,5,6};
var q = from x in new int[]{0} //force desired behavior
    from i in sample
    from j in sample
    select Math.Pow(i,j);


q.Distinct().Count().Dump();
sample = new[]{2,3,4,5};
//q = from i in Enumerable.Range(sample.bottom, sample.top-sample.bottom+1)
//  from j in Enumerable.Range(sample.bottom, sample.top-sample.bottom+1)
//  select checked(System.Numerics.BigInteger.Pow((BigInteger)i,j));
q.Distinct().Count().Dump();

2nd answer is incorrect every time if the from x line is not there, or the resetting of the q variable is not done.(shown here commented out)

Original sample was an anonymous variable, but the array did it also.

var sample = new { bottom =2, top=5};

Is it related to this Scope of Linq Expressions defined in a loop - At issue: closing over loop variable ?

Why did putting a 1 item array on the top fix the closure?

can't it be both? in a sense it is the lazy evaluation wherein one part is being eagerly evaluated, the other lazily. which to me is one closing and one not. — Maslow, Aug 04 '13 at 00:59
This seems to agree with both of us. http://stackoverflow.com/questions/428617/what-are-closures-in-net — Maslow, Aug 04 '13 at 01:06

ta.speot.is · Answer 1 · 2013-08-04T01:42:24.157

This code closes over sample twice.

from x in new int[]{0}
from i in sample
from j in sample
select Math.Pow(i,j);

The actual method calls this evaluates to are:

new int[] {0}
    .SelectMany(x => sample, (x, i) => new {x, i})
    .SelectMany(@t => sample, (@t, j) => Math.Pow(@t.i, j));

This code closes over sample once.

from i in sample
from j in sample
select Math.Pow(i,j);

The actual method calls this evaluates to are:

sample.SelectMany(i => sample, (i, j) => Math.Pow(i, j));

So if you change sample, only the i => sample lambda will "see" the "new" value for sample; sample.SelectMany has already run with the "old" value.

Both examples cause R# to issue an Access to modified closure warning, which is usually a big red flag. Consider keeping a private copy of sample for each of your queries, or have a function parameter do it for you:

IEnumerable<int> SelectManyPow(int[] sample)
{
    return from i in sample
           from j in sample
           select Math.Pow(i, j);
}

score 2 · Accepted Answer · answered Aug 04 '13 at 01:45

The first step to understanding what is going on here is to understand how a query expression is translated. Per the specification:

from x1 in e1
from x2 in e2
select v

is translated into

e1.SelectMany(x1 => e2, (x1, x2) => v)

This is incredibly important to understand. The result of from x1 in e1 from x2 in e2 select v is a query on e1, whatever e1 is; it is fixed, period.

Now, here's the issue. Let's look at your query.

var q = from i in sample
        from j in sample
        select Math.Pow(i, j)

The value of sample at at the time the query is constructed is a reference to the result of the array-creation expression new[] { 2, 3, 4, 5, 6 }. Let e1 be a reference to the result of that array-creation expression. Then the query becomes:

e1.SelectMany(i => sample, (i, j) => Math.Pow(i, j));

This is fixed, period, no matter what you reassign to sample. That is, you have to think of

var sample = new[] { 2, 3, 4, 5, 6 }
var q = from i in sample
        from j in sample
        select Math.Pow(i, j)

as

var sample = new[] { 2, 3, 4, 5, 6 };
var e1 = sample;
var q = e1.SelectMany(i => sample, (i, j) => Math.Pow(i, j));

Now, sample is closed over, and that's why after you assign a new value to sample, you see a different query result. But no matter what, e1 stays fixed as the result of the array-creation expression new[] { 2, 3, 4, 5, 6 }.

However, when you start with the query

var q = from x in new int[] { 0 }
        from i in sample
        from j in sample
        select Math.Pow(i, j)

now the query is translated to

int[] e1 = new int[] { 0 };
var q = e1.SelectMany(
    x1 => sample.SelectMany(i => sample, (i, j) => Math.Pow(i, j)
);

and now sample is captured as the sequence for both i and j but e1 is fixed.

+1 Your translation of the first query in OP's question might be more accurate than mine. — ta.speot.is, Aug 04 '13 at 01:47
Is there a way to write this such that the entire query is closed over properly without introducing a fake 1 item array but it still works as desired? — Maslow, Aug 05 '13 at 12:53

What determines if a variable is closed over?

2 Answers2