Trouble understanding yield in C#

Question

I'm hoping to get some clarification on a snippet that I've recently stepped through in the debugger, but simply cannot really understand.

I'm taking a C# course on PluralSight and the current topic is on yield and returning a IEnumerable<T> with the keyword.

I've got this overly basic function that returns an IEnumerable collection of Vendors (A simple class with Id, CompanyName and Email):

public IEnumerable<Vendor> RetrieveWithIterator()
{
    this.Retrieve(); // <-- I've got a breakpoint here
    foreach(var vendor in _vendors)
    {
        Debug.WriteLine($"Vendor Id: {vendor.VendorId}");
        yield return vendor;
    }
}

And I've got this code in a unit test that I'm using to test the function:

var vendorIterator = repository.RetrieveWithIterator(); // <-- Why don't it enter function?
foreach (var item in vendorIterator) // <-- But starts here?
{
    Debug.WriteLine(item);
}
var actual = vendorIterator.ToList();

What I really can't seem to understand, and I'm sure a lot of beginners are having the same trouble, is why the initial call to RetrieveWithIterator doesn't initiate the function, but it rather starts when we start iterating through its returned IEnumerable collection (see the comments).

You missed the [Deferred execution](https://www.kenneth-truyers.net/2016/05/12/yield-return-in-c/) — aloisdg, Jun 28 '17 at 09:44
Cheers. Do you have any good, easy-to-understand sources that I can use to read up on it? I'd love to edit the question with it, as I'm sure other people might be having the same troubles as I have — geostocker, Jun 28 '17 at 09:44
[Another example](https://stackoverflow.com/a/37856965/4137916) of an iterator getting dissected, with references to the standard. — Jeroen Mostert, Jun 28 '17 at 09:49
we call this Deferred and lazy execution, see this https://stackoverflow.com/questions/2515796/deferred-execution-and-eager-evaluation — Ghassen, Jun 28 '17 at 09:56

InBetween · Accepted Answer · 2017-06-28T09:50:09.153

20

This is called deferred execution, yield is lazy and will only work as much as it needs to.

This has great many advantages, one of which being that you can create seemingly infinite enumerations:

public IEnumerable<int> InfiniteOnes()
{
     while (true)
         yield 1;
}

Now imagine that the following:

var infiniteOnes = InfiniteOnes();

Would execute eagerly, you'd have a StackOverflow exception coming your way quite happily.

On the other hand, because its lazy, you can do the following:

var infiniteOnes = InfiniteOnes();
//.... some code
foreach (var one in infiniteOnes.Take(100)) { ... }

And later,

foreach (var one in infiniteOnes.Take(10000)) { ... }

Iterator blocks will run only when they need to; when the enumeration is iterated, not before, not after.

edited Jun 28 '17 at 09:50

answered Jun 28 '17 at 09:47

InBetween

32,319
3
50
90

Ah, interesting stuff. So the mentioned loops would basically return 100 and 1000 ones? – geostocker Jun 28 '17 at 09:49
@geostocker yes, that is correct. – InBetween Jun 28 '17 at 09:50
@geostocker Also, do realize that all `System.Linq` is built lazy. `infiniteOnes.Take(100)` is also lazy, so the resulting enumeration of the first 100 ones will only really execute once its enumerated in the `foreach`. The lazy chain can go on endlessly (`infiniteOnes.Take(100).Skip(10)`, etc.), it will only really start doing work when iterated. – InBetween Jun 28 '17 at 10:47

score 5 · Answer 2 · answered Jun 28 '17 at 09:50

From msdn:

Deferred Execution

Deferred execution means that the evaluation of an expression is delayed until its realized value is actually required. Deferred execution can greatly improve performance when you have to manipulate large data collections, especially in programs that contain a series of chained queries or manipulations. In the best case, deferred execution enables only a single iteration through the source collection.

Deferred execution is supported directly in the C# language by the yield keyword (in the form of the yield-return statement) when used within an iterator block. Such an iterator must return a collection of type IEnumerator or IEnumerator<T> (or a derived type).

var vendorIterator = repository.RetrieveWithIterator(); // <-- Lets deferred the execution
foreach (var item in vendorIterator) // <-- execute it because we need it
{
    Debug.WriteLine(item);
}
var actual = vendorIterator.ToList();

Eager vs. Lazy Evaluation

When you write a method that implements deferred execution, you also have to decide whether to implement the method using lazy evaluation or eager evaluation.

In lazy evaluation, a single element of the source collection is processed during each call to the iterator. This is the typical way in which iterators are implemented.
In eager evaluation, the first call to the iterator will result in the entire collection being processed. A temporary copy of the source collection might also be required.

Lazy evaluation usually yields better performance because it distributes overhead processing evenly throughout the evaluation of the collection and minimizes the use of temporary data. Of course, for some operations, there is no other option than to materialize intermediate results.

source

score 2 · Answer 3 · answered Jun 28 '17 at 09:48

It will get the items when you loop over them when needed. This way say you only need the first 4 results and then you break, it won't yield anything more and you just saved some processing power!

From MS Docs:

You use a yield return statement to return each element one at a time. You consume an iterator method by using a foreach statement or LINQ query. Each iteration of the foreach loop calls the iterator method. When a yield return statement is reached in the iterator method, expression is returned, and the current location in code is retained. Execution is restarted from that location the next time that the iterator function is called. You can use a yield break statement to end the iteration.

Note - If you .ToList() on the result of a method that yields, it will work as if you returned a single list, thus defeating the purpose of yield.

Trouble understanding yield in C#

3 Answers3

Deferred Execution

Eager vs. Lazy Evaluation