5

How can an iterator over a non-empty sequence, with no filtering and no aggregation (sum(), etc.), yield nothing?

Consider a simple example:

sequence = ['a', 'b', 'c']
list((el, ord(el)) for el in sequence)

This yields [('a', 97), ('b', 98), ('c', 99)] as expected.

Now, just swap the ord(el) out for an expression that takes the first value out of some generator using (...).next() — forgive the contrived example:

def odd_integers_up_to_length(str):
    return (x for x in xrange(len(str)) if x%2==1)

list((el, odd_integers_up_to_length(el).next()) for el in sequence)

This yields []. Yeah, empty list. No ('a',stuff) tuples. Nothing.

But we're not filtering or aggregating or reducing. A generator expression over n objects without filtering or aggregation must yield n objects, right? What's going on?

Gunnlaugur Briem
  • 2,684
  • 2
  • 23
  • 24
  • Wow... you related to hekevintran (http://stackoverflow.com/users/84952/hekevintran) at all? Ask a question. Answer it yourself almost immediately. – Jarret Hardie Apr 01 '09 at 00:15
  • Is that a Bad Thing To Do? I was just sharing my observation in question form, as per the FAQ: “It's also perfectly fine to ask and answer your own programming question, but pretend you're on Jeopardy: phrase it in the form of a question.” – Gunnlaugur Briem Apr 01 '09 at 00:41
  • My n00b warts are showing :) ... that link should be http://stackoverflow.com/faq of course – Gunnlaugur Briem Apr 01 '09 at 00:42
  • There's some debate over that FAQ entry. Yes, by all means, answer your own question. But if you routinely have a set of pre-baked questions and answers, then this site just becomes a blog, rather than an exchange. You have the misfortune of picking a day when someone abused the spirit of that FAQ. – Jarret Hardie Apr 01 '09 at 00:57
  • 1
    Jarret: ah, didn't know, thanks. No such routine intended. S. Lott: can't tell if you're being curt or just concise. If the former, save it for the FAQ authors. I was acting in good faith here. – Gunnlaugur Briem Apr 01 '09 at 01:58

4 Answers4

13

odd_integers_up_to_length(el).next() will raise StopIteration, which isn't caught there, but is caught for the generator expression within it, stopping it without ever yielding anything.

look at the first iteration, when the value is 'a':

>>> odd_integers_up_to_length('a').next()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration
Devin Jeanpierre
  • 92,913
  • 4
  • 55
  • 79
4

What happens is that the next() call raises a StopIteration exception, which bubbles up the stack to the outer generator expression and stops that iteration.

A StopIteration is the normal way for an iterator to signal that it's done. Generally we don't see it, because generally the next() call occurs within a construct that consumes the iterator, e.g. for x in iterator or sum(iterator). But when we call next() directly, we are the ones responsible for catching the StopIteration. Not doing so springs a leak in the abstraction, which here leads to unexpected behavior in the outer iteration.

The lesson, I suppose: be careful about direct calls to next().

Gunnlaugur Briem
  • 2,684
  • 2
  • 23
  • 24
0

str is a reserved keword, you should name your variable differently

I was also to advise about the next

Martin
  • 5,954
  • 5
  • 30
  • 46
0
>>> seq=['a','b','c']
>>> list((el,4) for el in seq)
[('a',4), ('b',4), ('c',4)]

So it's not list giving you trouble here...

dwc
  • 24,196
  • 7
  • 44
  • 55