Why is list.append(x) more efficient than list += l[x]?

Question

Appending to list can be possibly done in two ways:

1)

mat = []
for i in range(10):
    mat.append(i)

or

2)

mat = []
for i in range(10):
    mat += [i]

From the official Python documentation:

The method append() shown in the example is defined for list objects; it adds a new element at the end of the list. In this example it is equivalent to result = result + [a], but more efficient.

The documentation suggests that approach 1 is more efficient. Why is it so?

yes. that is also true. so which among the possible approaches is more efficient? — prashanth, Dec 15 '15 at 13:25
Or even just `mat = list(range(10))`. A list comprehension is generally preferred over an explicit loop, but some situations (complicated filtering, or building multiple lists at a time) favor a loop. — chepner, Dec 15 '15 at 13:25
I think this is primarily opinion-based. Although I prefer `.append` looks more readable to me — JDurstberger, Dec 15 '15 at 13:26
@Altoyr: It's not at all, you can see the produced bytecode to know which method is more efficient, which is the question. — Vincent Savard, Dec 15 '15 at 13:29
@chepner No the question was not in the generation of a list, but in appending to a list. — prashanth, Dec 15 '15 at 13:34
@apt-getinstallhappyness You're starting with an empty list; it's the same thing. In any case, you can always generate the extension using a list comprehension, then add that to the original list with `extend`. There are *many* options, and which (if any) is superior depends other factors. — chepner, Dec 15 '15 at 13:45
@Altoyr Nope. This is not POB. If the question was *Should I use this or this*, then it would have been. But this is asking *Why is this faster than that*. I hope you got the point :-) — Bhargav Rao, Dec 15 '15 at 14:45

PM 2Ring · Accepted Answer · 2015-12-15T14:14:09.407

Even though using .append requires a method call it's actually slightly more efficient than using the augmented assignment operator, +=.

But there's an even better reason to use .append: you can do it when the list you're appending to isn't in the local scope, since it's just calling the method of an object in an outer scope, whereas you cannot perform assignments on objects that aren't in the local scope unless you declare them as global (or nonlocal), a practice which is generally best to avoid.

Here's an example:

mat = []

def test_append():
    for i in range(10):
        #mat += [i]
        mat.append(i)

def test_iadd():
    for i in range(10):
        mat += [i]

test_append()
print(mat)
test_iadd()
print(mat)

output

[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
Traceback (most recent call last):
  File "./qtest.py", line 29, in <module>
    test_iadd()
  File "./qtest.py", line 25, in test_iadd
    mat += [i]
UnboundLocalError: local variable 'mat' referenced before assignment

Of course, we can pass mat as an argument to the function:

mat = []

def test_append():
    for i in range(10):
        #mat += [i]
        mat.append(i)

def test_iadd2(mat):
    for i in range(10):
        mat += [i]

test_append()
print(mat)
test_iadd2(mat)
print(mat)

output

[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

One reason that test_iadd2 is slower is that it has to construct a new [i] list on every loop.

FWIW, here's the bytecode:

test_append
 21           0 SETUP_LOOP              33 (to 36)
              3 LOAD_GLOBAL              0 (range)
              6 LOAD_CONST               1 (10)
              9 CALL_FUNCTION            1
             12 GET_ITER            
        >>   13 FOR_ITER                19 (to 35)
             16 STORE_FAST               0 (i)

 23          19 LOAD_GLOBAL              1 (mat)
             22 LOAD_ATTR                2 (append)
             25 LOAD_FAST                0 (i)
             28 CALL_FUNCTION            1
             31 POP_TOP             
             32 JUMP_ABSOLUTE           13
        >>   35 POP_BLOCK           
        >>   36 LOAD_CONST               0 (None)
             39 RETURN_VALUE        

test_iadd2
 26           0 SETUP_LOOP              33 (to 36)
              3 LOAD_GLOBAL              0 (range)
              6 LOAD_CONST               1 (10)
              9 CALL_FUNCTION            1
             12 GET_ITER            
        >>   13 FOR_ITER                19 (to 35)
             16 STORE_FAST               1 (i)

 27          19 LOAD_FAST                0 (mat)
             22 LOAD_FAST                1 (i)
             25 BUILD_LIST               1
             28 INPLACE_ADD         
             29 STORE_FAST               0 (mat)
             32 JUMP_ABSOLUTE           13
        >>   35 POP_BLOCK           
        >>   36 LOAD_CONST               0 (None)
             39 RETURN_VALUE

The above bytecode dump was produced using the dis module:

from dis import dis

mat = []

def test_append():
    for i in range(10):
        #mat += [i]
        mat.append(i)

def test_iadd2(mat):
    for i in range(10):
        mat += [i]

print('test_append')
dis(test_append)

print('\ntest_iadd2')
dis(test_iadd2)

"Even though using `.append` requires a method call it's actually slightly more efficient than using the augmented assignment operator, `+=`." The question is _why_ is `append` more efficient then `+=`. Provide sources that support this claim. — Vincent Savard, Dec 15 '15 at 13:40
@VincentSavard: Fair comment, although the main focus of my answer is that there's another reason _apart from efficiency_ why the use of `.append` is to be preferred. But I hope the additions to my answer have addressed your concerns, to a degree. — PM 2Ring, Dec 15 '15 at 13:57
It did, I think it's a much better answer in the context of the question. I upvoted you. — Vincent Savard, Dec 15 '15 at 13:58

score 1 · Answer 2 · answered Dec 15 '15 at 13:42

1


import timeit
timeit.timeit('for i in range(10): mat.append(i)', 'mat = []')
1.798893928527832
timeit.timeit('for i in range(10): mat += [i]', 'mat = []')
2.547478199005127

method append is faster, because it does't use any type conversion.

answered Dec 15 '15 at 13:42

mrvol

2,575
18
21

1

`+=` doesn't use any type conversion either. – pacholik Dec 15 '15 at 13:47
Shouldn't you invert the two statements called by `timeit`? First you create an empty list and then you append elements to it... – toti08 Dec 15 '15 at 13:48
@pacholik mat += [i] it's equivalent mat.extend(list(i)). You can check it to try run `mat += i` – mrvol Dec 15 '15 at 13:59
@toti08 No, it isn't. `timeit` create a new empty scope, which defined by second statement `'mat = []'`. If i understand you correctly... – mrvol Dec 15 '15 at 14:05
@mrvol And where is the type conversion in that? – pacholik Dec 15 '15 at 14:07
@pacholik, `mat += [i]` is type conversion. `i` has type integer before assign, and in moment assign its type is list, because `[i]` is equivalent `list(i)` – mrvol Dec 15 '15 at 14:12
@mrvol type `type(i)` before and after. – pacholik Dec 15 '15 at 14:15
@pacholik I've done. Why do you do it? http://www.codeskulptor.org/#user40_tkEl1qUw9e_0.py – mrvol Dec 15 '15 at 14:25
Ok, time to end this conversation, I'm clearly wasting my time here... – pacholik Dec 15 '15 at 14:39

score 0 · Answer 3 · answered Dec 15 '15 at 13:49

0

The second is slower because it creates a new list (which is somewhat slow operation).

answered Dec 15 '15 at 13:49

pacholik

8,607
9
43
55

Why is list.append(x) more efficient than list += l[x]?

3 Answers3