Order of for statements in a list comprehension

Question

In python2.7, I am trying to prepend every item in a list of strings with another item (eg. add item 'a' before every item in the list ['b', 'c']). From How to add list of lists in a list comprehension, I have determined the correct command, which boils down to:

>>> [i for x in ['b', 'c'] for i in ['a', x]]
['a', 'b', 'a', 'c']

Based purely on the temporary i and x variables, the version below seems more readable. However, it gives a completely different result. Why does this not give the same result?

>>> [i for i in ['a', x] for x in ['b', 'c']]
['a', 'a', 'c', 'c']

Even more curious, what happened to the 'b' entry?

Hint: what happens if you run the second code in a brand new interpreter window, where no code had previously been run? — Kevin, Sep 11 '15 at 19:33
The `for`s in a nested list comprehension appear in the same order as they do in a regular nested `for` loop - outermost first. You've just been confused by leakage of `x` from an earlier list comp (this has been fixed in 3.x). — jonrsharpe, Sep 11 '15 at 19:37
@Kevin, I get: NameError: name 'x' is not defined. By that logic, though, all list comprehensions use a variable before it's defined. In '[x for x in ...' the first x is what receives the value, and hasn't been defined yet. My understanding is you read from right to left when parsing list comprehensions, but this example doesn't seem to fit that axiom. — petiepooo, Sep 11 '15 at 19:42
You list comp is basically `for i in ['a', x]:;for x in ['b', 'c']:` — Padraic Cunningham, Sep 11 '15 at 19:47

Martijn Pieters · Accepted Answer · 2015-09-11T20:42:13.457

The for loops in list comprehensions are always listed in nesting order. You can write out both of your comprehensions as regular loops using the same order to nest; remember that only the expression before the first for produces the final values, so put that inside the loops.

So [i for x in ['b', 'c'] for i in ['a', x]] becomes:

for x in ['b', 'c']:
    for i in ['a', x]:
        i  # added to the final list

and [i for i in ['a', x] for x in ['b', 'c']] becomes:

for i in ['a', x]:
    for x in ['b', 'c']:
        i

As you can see, the second version would not be able to run without first defining x outside of your list comprehension, because otherwise the ['a', x] list could not be created. Also note that the x for the inner loop for x in ['b', 'c'] is otherwise ignored. All you get is i repeated. It doesn't matter what the values are in that list in the inner loop, only the length of the loop matters anymore.

In your case, your output would be explained by setting x = 'c' first; then you get for i in ['a', 'c'] for the outer loop, the inner loop iterates twice so 'a' is added twice, then i = 'c' is set and you get 'c' added twice.

As it happens, in Python 2, the variables using in a list comprehension 'leak', just like the variables used in a regular for loop leak; after using for x in ['b', 'c']: pass, x would remain available and bound to 'c'. This is where your x = 'c' comes from:

>>> [i for x in ['b', 'c'] for i in ['a', x]]
['a', 'b', 'a', 'c']
>>> i
'c'
>>> x
'c'
>>> [i for i in ['a', x] for x in ['b', 'c']]
['a', 'a', 'c', 'c']

i and x reflect what they were last bound to, so running the next list comprehension works as the first (outer) loop iterates over ['a', 'c'].

Remove x from your globals and the second list comprehension simply fails to run:

>>> del x
>>> [i for i in ['a', x] for x in ['b', 'c']]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'x' is not defined

The same happens to the full regular for loop versions above:

>>> for i in ['a', x]:
...     for x in ['b', 'c']:
...         i
... 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'x' is not defined
>>> x = 'foo'
>>> for i in ['a', x]:
...     for x in ['b', 'c']:
...         i
... 
'a'
'a'
'foo'
'foo'

In Python 3, list comprehensions are executed in a new scope (just like generator expressions, dict comprehensions and set comprehensions already do in Python 2).

Order of for statements in a list comprehension

1 Answers1

Related