Iterate two lists in the same loop

Question

How do I iterate through two lists in Python? I want exactly the result of

list1, list2, list3 = [0, 1], ['a', 'b'], [2, 3] # example data

for item in list1 + list2 + list3:
    print(item)

But I think adding lists and then iterating through them is not efficient. I used for loop to iterate through the lists:

for list_ in (list1, list2, list3):
    for item in list_:
        print(item)

But when I checked this with timeit execution time of both was very similar.

Is it possible to make it faster?

Answer 1

For small data sets you will not find much difference. But normally if you want to chain and iterate multiple iterables, then you can use itertools.chain, like this

>>> list1, list2, list3 = [0, 1], ['a', 'b'], [2, 3]
>>> from itertools import chain
>>> for item in chain(list1, list2, list3):
...     print(item)
0
1
a
b
2
3

This doesn't create any intermediate data structures and iterate each of the iterables one by one. The value returned by chain is an iterator. So that also doesn't create a container with all the items in it and it is very memory efficient if the iterables are going to be very big.

And itertools.chain is effectively the same as your second approach. Quoting the equivalent implementation from the official docs

def chain(*iterables):
    # chain('ABC', 'DEF') --> A B C D E F
    for it in iterables:
        for element in it:
            yield element

---

If we look at the byte code generated for the first program you have shown, with this

from dis import dis
list1, list2, list3 = [0, 1], ['a', 'b'], [2, 3]


def func():
    for item in list1 + list2 + list3:
        print(item)

dis(func)

it would be something like

  6           0 SETUP_LOOP              27 (to 30)
              3 LOAD_GLOBAL              0 (list1)
              6 LOAD_GLOBAL              1 (list2)
              9 BINARY_ADD          
             10 LOAD_GLOBAL              2 (list3)
             13 BINARY_ADD          
             14 GET_ITER            
        >>   15 FOR_ITER                11 (to 29)
             18 STORE_FAST               0 (item)

  7          21 LOAD_FAST                0 (item)
             24 PRINT_ITEM          
             25 PRINT_NEWLINE       
             26 JUMP_ABSOLUTE           15
        >>   29 POP_BLOCK           
        >>   30 LOAD_CONST               0 (None)
             33 RETURN_VALUE        

As you can see, the BINARY_ADD code is used twice. It means that list1 and list2 are added first and a temporary list is created and that is again added with list3. This will be highly inefficient if any of the lists is very big.

Answer 2

Use itertools.chain:

for item in itertools.chain(list1, list2, list2):
    print(item)

Answer 3

You can use the method chain from itertools module to chain the three lists into single one:

from itertools import chain:

for item in chain(list1, list2, list3):
    print(item)

OR:

for item in chain.from_iterable([list1,list2,list3]):
    print(item)

Answer 4

If you're more of a numbers person, here is a test case providing evidence that it is inefficient to add the lists together before iterating through.

Note that hugelists.py contains three lists each containing 10,000 random four-digit numbers.

from hugelists import list_one, list_two, list_three
from itertools import chain
from datetime import datetime

def method_one():
    start_time = datetime.now()
    for item in list_one + list_two + list_three:
        pass
    stop_time = datetime.now()
    return stop_time - start_time

def method_two():
    start_time = datetime.now()
    for item in chain(list_one, list_two, list_three):
        pass
    stop_time = datetime.now()
    return stop_time - start_time

if __name__ == "__main__":
    print method_one()
    print method_two()

Results:

> python test.py
0:00:00.001720
0:00:00.001014
> python test.py
0:00:00.001865
0:00:00.000997
> python test.py
0:00:00.001603
0:00:00.000833

Answer 5

You can try the builtin zip() function.

>>> for m, n, k in zip([1,2,3,4,5], list('abcde'), [2,11,25,102,53]):
        print(m, n, k)


1 a 2
2 b 11
3 c 25
4 d 102
5 e 53

Notice you can put other iterables into it.

for m, n, k in zip(range(1,6), 'abcde', [2,11,25,102,53]):
    print(m, n, k)


1 a 2
2 b 11
3 c 25
4 d 102
5 e 53