Modifying a list iterator in Python not allowed?

Question

Simple example:

myList = [1, 2, 3, 4, 5]
for obj in myList:
  obj += 1
print myList

prints

[1, 2, 3, 4, 5]
[1, 2, 3, 4, 5]

while:

myList = [1, 2, 3, 4, 5]
for index in range(0,len(myList)):
  myList[index] += 1
print myList

prints

[1, 2, 3, 4, 5]
[2, 3, 4, 5, 6]

Conclusion:

Lists can be modified in place using global list access Lists can
List items can NOT be modified in place using the iterator object

All example code I can find uses the global list accessors to modify the list inplace. Is it so evil to modify a list iterator?

As-is, this code would produce a `NameError` because `a` isn't defined. — senderle, Feb 23 '12 at 13:58
your code examples are bad. Where do the first to prints come from? Your printing 'a' which doesn't exist and there are two outputs. — Matt Alcock, Feb 23 '12 at 13:59
[Somewhat related](http://stackoverflow.com/questions/1135335/increment-int-object) — Lauritz V. Thaulow, Feb 23 '12 at 14:45
Sorry guys, 'a' should obviously be 'myList'. Fixed in the post. — RobW, Feb 29 '12 at 09:35

NPE · Answer 1 · 2012-02-23T14:03:42.173

7

The reason obj += 1 does not do what you expect is that this statement does not modify obj in-place. Instead, it computes the new value, and rebinds the variable obj to point to the new value. This means that the contents of the list remain unchanged.

In general it is possible to modify the list while iterating over it using for obj in myList. For example:

myList = [[1], [2], [3], [4], [5]]
for obj in myList:
  obj[0] += 1
print(myList)

This prints out:

[[2], [3], [4], [5], [6]]

The difference between this and your first example is that here, the list contains mutable objects, and the code modifies those objects in-place.

Note that one could also write the loop using a list comprehension:

myList = [val+1 for val in myList]

edited Feb 23 '12 at 14:03

answered Feb 23 '12 at 13:58

NPE

486,780
108
951
1,012

Thanks for your answer. I already concluded that the `obj` iterator is not an iterator as in C++, and modifying the object does not change the list. What you say is that `obj` is bound again to the new object that will be lost in the next iteration. Clear! – RobW Feb 29 '12 at 09:41
Great answer! It's not about 'shallow copy' talked about by the accepted answer. – Oct 28 '15 at 23:08

score 7 · Answer 2 · edited May 23 '17 at 11:51

I think you've misunderstood what an "iterator object" is. A for loop is not an iterator object. For all intents and purposes, a for loop like this:

myList = [0, 1, 2, 3, 4]
for x in myList:
    print x

does this (but more efficiently and less verbosely):

i = 0
while i < len(myList)
    x = myList[i]
    print x
    i += 1

So you see, any changes made to x are lost as soon as the next loop starts, because the value of x is overwritten by the value of the next item in the list.

As others have observed, it is possible to alter the value of a list while iterating over it. (But don't change its length! That's where you get into trouble.) One elegant way to do so is as follows:

for i, x in enumerate(myList):
    myList[i] = some_func(x)

Update: It's also important to understand that no copying goes on in a for loop. In the above example, i and x -- like all variables in Python -- are more like pointers in C/C++. As the for loop progresses, obj points at myList[0], myList[1], etc, in turn. And like a C/C++ pointer, the properties of the object pointed to are not changed when the pointer is changed. But also like a C pointer, you can directly modify the thing pointed at, because it's not a copy. In C, this is done by dereferencing the pointer; in Python, this is done by using a mutable object. That's why NPE's answer works. If i and x were even shallow copies, it wouldn't be possible to do what he does.

The reason you can't directly change ints the way you can change lists (as in NPE's answer), is that ints aren't mutable. Once a 5 object is created, nothing can change its value. That's why passing around a pointer to 5 is safe in Python -- no side-effects can occur, because the thing pointed to is immutable.

Obviously, `print a` should be `print myList`, I fixed this. Thanks for the pointer to `enumerate`, very useful, avoids to maintain your own code for the list index! — RobW, Feb 29 '12 at 09:45
To be even more pythonic & concise, you could do either `[some_func(x) for x in myList]` or `map(some_func, myList)`, although those make copies of the initial list. — Ryan Madsen, Feb 26 '13 at 01:34

score 6 · Accepted Answer · answered Feb 23 '12 at 13:59

6

in for obj in myList:, in every iteration, obj is a (shallow) copy of the element in myList. So the change on the obj does nothing to myList's elements.

It's different with the Perl for my $obj (@myList) {}

answered Feb 23 '12 at 13:59

Ade YU

2,292
3
18
28

Exactly, I understand that now. `obj` is not an iterator as in C++, it basically is a copy of the object, which is ignored in the next iteration. Thus, modification can only be done using global/direct list access. – RobW Feb 29 '12 at 09:38
1

@RobW, `obj` is not a copy, and this answer is misleading. obj -- like all variables in Python -- is more like a pointer in C/C++. See the edit to [my answer](http://stackoverflow.com/a/9414582/577088). – senderle Feb 29 '12 at 14:27
Ade YU, I gather when you say "copy" you mean "copy of the address of the element" rather than "copy of the element itself." You might clarify that. – senderle Feb 29 '12 at 14:29
@senderle Yes. In Python, *variables* are just references to *objects*. Only the *variables* are copied, but the *objects*. It's just like `obj = myList[index]` happened at the very beginning of each iteration. So `obj += 1` doesn't affect the `myList[index]`. Thank you. I will improve my answer statement. – Ade YU Feb 29 '12 at 14:51
might not be a shallow copy if it is a list of lists – Ninja420 Apr 14 '17 at 01:29

Marcin · Answer 4 · 2012-02-29T10:29:47.623

1

You are confused. Consider your first snippet:

myList = [1, 2, 3, 4, 5]
for obj in myList:
  obj += 1
print a

obj is not some kind of magical pointer into the list. It is a variable which holds a reference to an object which happens to also be in myList. obj += 1 has the effect of increasing the value stored in obj. Your code then does nothing with that value.

To be clear: There are no copies in this code example. obj is a variable, which holds an object in the list. That is all.

edited Feb 29 '12 at 10:29

answered Feb 23 '12 at 13:58

Marcin

48,559
18
128
201

If `obj` would be an iterator as in C++, then the list object would be updated. But apparently, `obj` is a copy and changes are ignored after each iteration. Apparently, Python does the latter, which is my question. – RobW Feb 29 '12 at 09:44
@RobW: Repeat after me: There are no copies in this code example. `obj` is a variable, which holds an object in the list. That is all. – Marcin Feb 29 '12 at 10:29

score 0 · Answer 5 · answered Feb 23 '12 at 13:57

0

In the first example the integer is copied into obj which is increased by 1. The list is not changed.

If you would use a class instance and perform operations on it, it would be changed.

answered Feb 23 '12 at 13:57

Michel Keijzers

15,025
28
93
119

score 0 · Answer 6 · answered Feb 23 '12 at 14:00

0

Modification in list is allowed. Your code examples arbove are pretty garbled...

myList = [1, 2, 3, 4, 5]
for index in range(0,len(myList)):
   myList[index] += 1
 print myList

This works.

answered Feb 23 '12 at 14:00

Matt Alcock

12,399
14
45
61

That's indeed what I posted and does not answer anything. – RobW Feb 29 '12 at 09:42

Modifying a list iterator in Python not allowed?

6 Answers6

Linked

Related