Why can iterators in structs modify this?

Question

I discovered that iterator methods in value types are allowed to modify this.
However, due to limitations in the CLR, the modifications are not seen by the calling method. (this is passed by value)

Therefore, identical code in an iterator and a non-iterator produce different results:

static void Main() {
    Mutable m1 = new Mutable();
    m1.MutateWrong().ToArray();     //Force the iterator to execute
    Console.WriteLine("After MutateWrong(): " + m1.Value);

    Console.WriteLine();

    Mutable m2 = new Mutable();
    m2.MutateRight();
    Console.WriteLine("After MutateRight(): " + m2.Value);
}

struct Mutable {
    public int Value;

    public IEnumerable<int> MutateWrong() {
        Value = 7;
        Console.WriteLine("Inside MutateWrong(): " + Value);
        yield break;
    }
    public IEnumerable<int> MutateRight() {
        Value = 7;
        Console.WriteLine("Inside MutateRight(): " + Value);
        return new int[0];
    }
}

Output:

Inside MutateWrong(): 7
After MutateWrong(): 0

Inside MutateRight(): 7
After MutateRight(): 7

Why isn't it a compiler error (or at least warning) to mutate a struct in an iterator?
This behavior is a subtle trap which is not easily understood.

Anonymous methods, which share the same limitation, cannot use this at all.

Note: mutable structs are evil; this should never come up in practice.

Answer 1

In order to justify a warning, it should be in a situation where the programmer is likely to get unexpected results. According to Eric Lippert, "we try to reserve warnings for only those situations where we can say with almost certainty that the code is broken, misleading or useless." Here is an instance where the warning would be misleading.

Let's say you have this perfectly valid – if not terribly useful – object:

struct Number
{
    int value;
    public Number(int value) { this.value = value; }
    public int Value { get { return value; } }
    // iterator that mutates "this"
    public IEnumerable<int> UpTo(int max)
    {
        for (; value <= max; value++)
            yield return value;
    }
}

And you have this loop:

var num = new Number(1);
foreach (var x in num.UpTo(4))
    Console.WriteLine(num.Value);

You'd expect this loop to print 1,1,1,1, not 1,2,3,4, right? So the class works exactly as you expect. This is an instance where the warning would be unjustified.

Since this is clearly not a situation where the code is broken, misleading, or useless, how would you propose that the compiler generate an error or warning?

Answer 2

To cite yourself "mutable structs are evil" :) The same thing as you experienced happens if you implement an extension method for a struct. If you try to modify the struct within the extension method you still will have your original struct unchanged. It is a bit less surprising since the extension method signature looks like:

static void DoSideEffects(this MyStruct x) { x.foo = ...

Looking at it we realize that something like parameter passing happens and therefore the struct is copied. But when you use the extension it looks like:

x.DoSideEffects()

and you will be surprised not to have any effects on your variable x. I suppose that behind the scenes your yield constructs do something similar to extensions. I would phrase the starting sentence harder: "structs are evil" .. in general ;)

Answer 3

I had a similar thought to what Gabe said. It seems at least theoretically possible that one might opt to use a struct to behave kind of like a method, encapsulating that method's local variables as instance fields:

struct Evens
{
    int _current;

    public IEnumerable<int> Go()
    {
        while (true)
        {
            yield return _current;
            _current += 2;
        }
    }
}

I mean, it's kind of weird, obviously. It kind of reminds me of ideas I've encountered in the past, though, where developers have concocted ever-stranger ways of calling methods, such as wrapping a method's parameters into an object and letting that object call the method—going backwards, in a sense. I'd say that's roughly what this is.

I'm not saying this would be a wise thing to do, but it is at least a way of using this in the way you are describing that might be intentional, and would technically exhibit correct behavior.

Answer 4

It's not really clear what should happen, though I think .net is deficient in not requiring a special attribute for methods which modify 'this'; such an attribute could be useful applied to immutable class types as well as mutable structs. Methods which are tagged with such an attribute should only be usable on structure variables, fields, and parameters, and not on temporary values.

I don't think there's any way to avoid having the iterator capture the struct by value. It's entirely possible that by the time an iterator is used, the original struct upon which it was based may not exist anymore. On the other hand, if the struct implemented an interface that inherited IEnumerable<int>, but also included a function to return Value, casting the struct to that interface type before using the enumerator could in theory allow the iterator to keep a reference to the struct without having to recopy its value; I wouldn't be at all surprised if the enumerator copies the struct by value even in that case, though.