Closure and Linq

Question

IEnumerable<char> charQuery1 = "Not what you might expect";
string vowels = "aeiou";
for (int i = 0; i < vowels.Length; i++)
   charQuery1 = charQuery1.Where(c => c != vowels[i]);
foreach (char c in charQuery1) Console.Write(c);

This is will throw IndexOutofRange Exception. Compiler scopes the for loop variable "i" as like declared outside. Can someone dissect this and make me more comfortable to understand? Debugging did not help me much here. I am trying to understand the compiler role here.

Answer 1

Linq functions such as Where create an IEnumerable that will be lazily evaluated.

This means the result of the Where function will not be produced when the Where function is being invoked. The result of the Where function is being produced when the IEnumerable returned by Where is being iterated.

In your example code, the results of the Where functions are iterated in the foreach loop at the end of your code:

foreach (char c in charQuery1) Console.Write(c);

This is where the enumerabled returned by the Where functions are being iterated, causing the predicate delegates provided to the Where functions to be executed.

In your case, those predicate delegate are anonymous functions in the form of c => c != vowels[i], which close over the variable i.

Once again, those anonymous functions c => c != vowels[i] are invoked only when the foreach loop at the end of your code is being executed.

So, what is the value of i when the foreach loop is being executed? It's the last value it has when the for loop exits. In other words, the value of i is equal to vowels.Length.

That leads to the predicate delegate to be equivalent to c => c != vowels[vowels.Length]. Obviously, vowels[vowels.Length] will cause an IndexOutOfRangeException.

How to fix it?

The naive approach would be to not let your predicates close over the i counter variable of the for loop. Instead, let it close over a variable that is scoped inside the for body:

for (int i = 0; i < vowels.Length; i++)
{
    var local_i = i;
    charQuery1 = charQuery1.Where(c => c != vowels[local_i]);
}

But as the comment by Tim Schmelter points out, the more elegant and much shorter solution would be to simply use Linq's Except method:

var charQueryWithoutVowels = charQuery1.Except(vowels);

Answer 2

The predicate you pass into Where (which is c != vowels[i]) is executed in deferred way when you enumerate charQuery1 in last part of the code (foreach (char c in charQuery1)). By that moment variable i has the value which was the last one in the loop for (int i = 0; i < vowels.Length; i++) which is '5'. So you captured the value which is modified from inner closure. You must remember that LINQ expressions have deferred execution and don't execute when you declare them but when you enumerate them.

To avoid this problem you should capture loop index into temporary variable withing the loop scope like this.

IEnumerable<char> charQuery1 = "Not what you might expect";
string vowels = "aeiou";
for (int i = 0; i < vowels.Length; i++)
{
   int index = i; // Temporary index variable. Should be declared inside the scope!
   charQuery1 = charQuery1.Where(c => c != vowels[index]);
}
foreach (char c in charQuery1) Console.Write(c);

UPD: Fully deferred behavior:

IEnumerable<char> charQuery1 = "Not what you might expect";
string vowels = "aeiou";
Enumerable
    .Range(0, vowels.Length)
    .ForEach(i =>
    {
        charQuery1 = charQuery1.Where(c => c != vowels[i]);
    });
foreach (char c in charQuery1) Console.Write(c);

But even further simplification will give you this:

IEnumerable<char> charQuery1 = "Not what you might expect";
string vowels = "aeiou";

charQuery1 = Enumerable
    .Range(0, vowels.Length)
    .Aggregate(charQuery1, (cq, i) => cq.Where(c => c != vowels[i]));

foreach (char c in charQuery1) Console.Write(c);

Cool, isn't it?