ValueError raised before StopIteration with map(next, iterables)

Question

I am writing a loop over 4 list-iterators like this :

it0 = iter(['foo','bar'])
it1 = iter(['bar','foo'])
it2 = iter(['foo','foo'])
it3 = iter(['bar','bar'])

try:
    a, b, c, d = map(next, (it0, it1, it2, it3))
    while True:
        #some operations that can raise ValueError
        if a+b+c+d == 'foobar'*2:
            a, b, c, d = map(next, (it0, it1, it2, it3))
        elif a+b == 'foobar':
            c, d = map(next,(it2, it3))
        else:
            a, b = map(next,(it0, it1))
except StopIteration:
    pass

But when I run this code in Python3, I get a ValueError: not enough values to unpack (expected 2, got 0) raised before the expected StopIteration.

I do not want to catch ValueError in the except statement because some operations in the while loop could also result in a ValueError that should stop the program.

How is it that next doesn't raise its exception before the assignement ?

In Python2.7, StopIteration is raised first.

You should probably fix the error but FYI you can catch more than one type of exception without catching everything: `except StopIteration, ValueError: do_something()` — jDo, Apr 27 '16 at 14:48
Cool cool.. Your question just led me to believe that you didn't know this. It's written as if your only choices are to catch *either* a `StopIteration` or a `ValueError`; not both. Anyway, I think Mr. Pieters might have solved the *actual* problem below. — jDo, Apr 27 '16 at 15:07
@jDo, I just want to catch StopIteration to stop the while loop but I want the program to stop if there is a ValueError. If both are caught and ValueError can mean StopIteration, it is confusing. As you said problem solved. — Jacques Gaudin, Apr 27 '16 at 15:18
@JacquesGaudin It appears that you are attempting to provide solutions to your question in the question itself. That is what *answers* are for. It is completely fine, even encouraged, to answer your own question with what worked for you. — Justin, Apr 27 '16 at 18:10
Ok, I didn't want to be rude to Martijn Pieters who did the most of the job... Will create another answer. — Jacques Gaudin, Apr 27 '16 at 18:16
@Justin: not sure that that required a full roll-back however. — Martijn Pieters, Apr 27 '16 at 18:21

Martijn Pieters · Accepted Answer · 2016-04-27T18:24:30.563

The StopIteration is raised, but the tuple assignment swallows this as it sees the StopIteration as a signal from the map() iterator that it is done producing values:

>>> i0, i1 = iter(['foo']), iter([])
>>> m = map(next, (i0, i1))
>>> next(m)
'foo'
>>> next(m)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration
>>> i0, i1 = iter(['foo']), iter([])
>>> m = map(next, (i0, i1))
>>> a, b = m
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: not enough values to unpack (expected 2, got 1)

This is normal behaviour; Python built-ins that expect an iterator as input always use StopIteration as the 'iteration is done' signal, and tuple unpacking must iterate here.

Either convert the map() output to a list first and test the length, use next() on each iterable separately and not use map(), or catch ValueError locally.

Testing the length would have to re-raise StopIteration:

values = list(map(next, (it0, it1, it2, it3)))
if len(values) < 4:
    raise StopIteration
a, b, c, d = values

Note that here list() has swallowed the StopIteration exception.

Catching the ValueError just for the map() operation:

try:    
    a, b, c, d = map(next, (it0, it1, it2, it3))
except ValueError:
    raise StopIteration

Not using map() at all by calling next() on each iterator individually:

a, b, c, d = next(it0), next(it1), next(it2), next(it3)

or by using a list comprehension:

a, b, c, d = [next(i) for i in (it0, it1, it2, it3)]

both of which ensure that next() is called before assignment takes place, rather than during assignment itself.

Thanks, it makes more sense now. May I ask why Python2.7 behaviour is different ? — Jacques Gaudin, Apr 27 '16 at 15:04
@JacquesGaudin: `map()` returns a list in Python 2.7. This means it has to first *produce* that list before tuple assignment can take place, so the `StopIteration` exception is raised outside of tuple assignment iteration. — Martijn Pieters, Apr 27 '16 at 15:06
That makes even more sense ! Many thanks for a prompt answer. — Jacques Gaudin, Apr 27 '16 at 15:11

score 0 · Answer 2 · edited May 23 '17 at 10:28

In the light of Martijn Pieters answer, using list comprehension directly works :

a, b, c, d = [next(it) for it in (it0, it1, it2, it3)]

This post clarifies why for catches StopIteration on the iterable but not in the body of the loop.

Another possible way, taking advantage of the default parameter of next :

it0 = iter(['foo','bar'])
it1 = iter(['bar','foo'])
it2 = iter(['foo','foo'])
it3 = iter(['bar','bar'])

try:
    a, b, c, d = map(next, (it0, it1, it2, it3), (None,)*4)
    while all(_ is not None for _ in (a,b,c,d)):
        #some operations that can raise ValueError
        if a+b+c+d == 'foobar'*2:
            a, b, c, d = map(next, (it0, it1, it2, it3), (None,)*4)
        elif a+b == 'foobar':
            c, d = map(next,(it2, it3), (None,)*2)
        else:
            a, b = map(next,(it0, it1), (None,)*2)
except StopIteration:
    pass

I've incorporated that approach into my answer too; it is basically the same approach as using `next()` on individual iterators first, the point being that both avoid using a generator to postpone using `next()` until iteration for assignments. — Martijn Pieters, Apr 27 '16 at 18:25
Absolutely, it is worth noting that the `StopIteration` in the body of the `for` loop isn't swallowed though. — Jacques Gaudin, Apr 27 '16 at 18:28

ValueError raised before StopIteration with map(next, iterables)

2 Answers2