28

I have python code like this:

newlist =[[52, None, None], [129, None, None], [56, None, None], [111, None, None],  
          [22, None, None], [33, None, None], [28, None, None], [52, None, None],  
          [52, None, None], [52, None, None], [129, None, None], [56, None, None],  
          [111, None, None], [22, None, None], [33, None, None], [28, None, None]]

I want the newlist like:

newlist =[52, None, None,129, None, None,56, None, None,111, None, None,22, 
          None, None,33, None, None,28, None, None,52, None, None,52, None,  
          None,52, None, None,129, None, None,56, None, None, 111, None,  
          None,22, None, None,33, None, None,28, None, None]

Is there any way to work around ?

thefourtheye
  • 233,700
  • 52
  • 457
  • 497
user2879220
  • 297
  • 1
  • 3
  • 5

4 Answers4

122

What you are trying to do is called flattening the list. And according to the Zen of Python, you are trying to do the right thing. Quoting from that

Flat is better than nested.

  1. So you can use list comprehension like this

     newlist = [item for items in newlist for item in items]

  2. Or you can use chain from itertools like this

     from itertools import chain
 newlist = list(chain(*newlist))

  3. Or you can use chain.from_iterable, where unpacking of the list is not necessary

     from itertools import chain
 newlist = list(chain.from_iterable(newlist))

  4. Using sum function

     newlist = sum(newlist, [])

  5. Using reduce function

     newlist = reduce(lambda x,y: x+y, newlist)

  6. Using operator.add. This will be faster than the reduce with lambda version.

     import operator
 from functools import reduce
 newlist = reduce(operator.add, newlist)

Edit: For the sake of completeness, included the answers found in How do I make a flat list out of a list of lists? as well.

I tried to time all of them in Python 2.7, like this

from timeit import timeit
print(timeit("[item for items in newlist for item in items]", "from __main__ import newlist"))
print(timeit("sum(newlist, [])", "from __main__ import newlist"))
print(timeit("reduce(lambda x,y: x+y, newlist)", "from __main__ import newlist"))
print(timeit("reduce(add, newlist)", "from __main__ import newlist; from operator import add"))
print(timeit("list(chain(*newlist))", "from __main__ import newlist; from itertools import chain"))
print(timeit("list(chain.from_iterable(newlist))", "from __main__ import newlist; from itertools import chain"))

Output on my machine

2.26074504852
2.45047688484
3.50180387497
2.56596302986
1.78825688362
1.61612296104

So, the most efficient way to do this is to use list(chain.from_iterable(newlist)), in Python 2.7. Ran the same test in Python 3.3

from timeit import timeit
print(timeit("[item for items in newlist for item in items]", "from __main__ import newlist"))
print(timeit("sum(newlist, [])", "from __main__ import newlist"))
print(timeit("reduce(lambda x,y: x+y, newlist)", "from __main__ import newlist; from functools import reduce"))
print(timeit("reduce(add, newlist)", "from __main__ import newlist; from operator import add; from functools import reduce"))
print(timeit("list(chain(*newlist))", "from __main__ import newlist; from itertools import chain"))
print(timeit("list(chain.from_iterable(newlist))", "from __main__ import newlist; from itertools import chain"))

Output on my machine

2.26074504852
2.45047688484
3.50180387497
2.56596302986
1.78825688362
1.61612296104

So, be it Python 2.7 or 3.3, use list(chain.from_iterable(newlist)) to flatten the nested lists.

Botond
  • 2,640
  • 6
  • 28
  • 44
thefourtheye
  • 233,700
  • 52
  • 457
  • 497
  • Got a question. Why does this work `[i for j in l for i in j]` and this `[i for i in j for j in l]` doesn't? – tmj Nov 21 '13 at 04:59
  • @tMJ In the second form, the sublists are being comprehended again, not the individual elements in the sublists. Just break the comprehension and print the values of `i` in both the cases. – thefourtheye Nov 21 '13 at 05:01
  • 2
    @tMJ: think about the equivalent for loops. The first is `for j in l: for i in j: newlist.append(i)`, and the second is `for i in j: for j in l: newlist.append(i)`. – DSM Nov 21 '13 at 05:01
  • @DSM i thought it that way, equating it to a nested lopp structure, but the issue is that if the outer loop is `for i in j:` j is not initialized yet. So, what do I make of that. Even though the structure `[i for i in j for j in l]` would assign some value to j, IDK as to what it would be. – tmj Nov 21 '13 at 05:04
  • @tMJ: It wouldn't assign any value to `j`; you'd get a `NameError` if it weren't defined. – DSM Nov 21 '13 at 05:06
  • @DSM The code works, `>>> l = [[1,2,3],[4,5,6]] >>> [i for j in l for i in j] [1, 2, 3, 4, 5, 6] >>> [i for i in j for j in l] [4, 4, 5, 5, 6, 6]` Explain this. – tmj Nov 21 '13 at 05:08
  • 1
    @tMJ: In Python 2, list comprehensions leak the loop variables. Try the same thing again, but `del j` between the two, and you'll see `NameError: name 'j' is not defined`. – DSM Nov 21 '13 at 05:11
  • @DSM aaah! Okay! But I can't still comprehend the output. Maybe you can help. – tmj Nov 21 '13 at 05:14
  • 1
    An extremely detailed answer, the best i've seen in a while. +1 – aIKid Nov 21 '13 at 05:36
  • The quoted timings are equal up to the last digit for Python 2.7 and Python 3.3: I suspect that these really come from the same run... – rerx Jan 22 '18 at 17:08
19

Just the easiest one:

newlist = sum(newlist, [])
print newlist
lennon310
  • 12,503
  • 11
  • 43
  • 61
6

Try:

newlist = [j for i in newlist for j in i]
Christian Tapia
  • 33,620
  • 7
  • 56
  • 73
2
temp = []
for small_list in newlist:
    temp += small_list
newlist = temp

This should do it.

tmj
  • 1,750
  • 2
  • 19
  • 34