For loop for second items in list of lists only. (python)

Question

What I have

is something like this

def mymethod():
    return [[1,2,3,4],
            [1,2,3,4],
            [1,2,3,4],
            [1,2,3,4]]

mylist = mymethod()

for _, thing, _, _ in mylist:
    print thing

# this bit is meant to be outside the for loop, 
# I mean it to represent the last value thing was in the for
if thing:
    print thing

What I want

what I want to do is avoid the dummy variables, is there a smarter way to do this than

for thing in mylist:
    print thing[1]

because then i would have to use thing[1] any other time I needed it, without assigning it to a new variable and then things are just getting messy.

newish to python so sorry if I'm missing something obvious

Hmmm... not sure I understand your problem; can you clarify what you want to do? — cfedermann, Apr 13 '12 at 11:34

score 7 · Accepted Answer · answered Apr 13 '12 at 11:35

7

You could hack a generator expression

def mymethod():
    return [[1,2,3,4],
            [1,2,3,4],
            [1,2,3,4],
            [1,2,3,4]]

mylist = mymethod()

for thing in (i[1] for i in mylist):
    print thing

# this bit is meant to be outside the for loop, 
# I mean it to represent the last value thing was in the for
if thing:
    print thing

answered Apr 13 '12 at 11:35

Kien Truong

11,179
2
30
36

+1, I wouldn't describe this as a hack - it's clear and works. – Gareth Latty Apr 13 '12 at 11:44
will it make a difference if i use a generator expression/comprehension, from what i understand a generator expression is more efficient for large numbers, is that accurate? – Jacxel Apr 13 '12 at 12:58
If you use list comprehension in this instance, you'll have to traverse the length of the first list twice: one to generate the smaller list and one to traverse the smaller list (both have the same length). Generator expression only traverse the first list once. – Kien Truong Apr 13 '12 at 14:10

Li-aung Yip · Answer 2 · 2012-04-13T13:02:27.703

3

If you want to get the second column of an array, you could use a list comprehension, like so:

a = [ [ 1, 2, 3, 4 ],
      [ 5, 6, 7, 8 ],
      [ 9,10,11,12 ],
      [13,14,15,16 ] ]


second_column = [ row[1] for row in a ]
# you get [2, 6, 10, 14]

You can wrap this up in a function:

def get_column ( array, column_number ):
    try:
        return [row[column_number] for row in array]
    except IndexError:
        print ("Not enough columns!")
        raise # Raise the exception again as we haven't dealt with the issue.

fourth_column = get_column(a,3)
# you get [4, 8, 12, 16]

tenth_column = get_column(a,9)
# You requested the tenth column of a 4-column array, so you get the "not enough columns!" message.

Though really, if you're working with rectangular arrays of numbers, you want to be using numpy arrays, not lists of lists of numbers.

Or, by Lattyware's implied request, a generator version:

def column_iterator ( array, column_number ):
    try:
        for row in array:
            yield row[column_number]
    except IndexError:
        print ("Not enough columns!")
        raise

Usage is just like a normal list:

>>> for item in column_iterator(a,1):
...    print(item)
... 
2
6
10
14
>>>

The generator-nature is evident by:

>>> b = column_iterator(a,1)
>>> b.next()
2
>>> b.next()
6
>>> b.next()
10
>>> b.next()
14

edited Apr 13 '12 at 13:02

answered Apr 13 '12 at 11:51

Li-aung Yip

12,320
5
34
49

This is basically the same thing as Dikei's generator expression, except less efficient as the list has to be generated beforehand. Also, it may have been for example here, but it's really bad practice to print an error and return None silently, instead of just throwing the exception. – Gareth Latty Apr 13 '12 at 11:52
Edited to include a generator version that makes me feel warm and fuzzy. (I had to stop and think about Dikei's.) – Li-aung Yip Apr 13 '12 at 11:58
@Lattyware: agreed this is a routine you don't want silently consuming exceptions - it's probably better to explode loudly here. Feel free to edit my answer to re-throw exceptions - I'm actually not sure how to re-throw while maintaining the traceback, and I'd appreciate a demo. ;) – Li-aung Yip Apr 13 '12 at 11:59
It's literally as simple as a blank raise statement, it'll automatically throw the last one. Alternatively, do ``catch IndexError, e: ... raise e`` where you can do things with e. If you want to deal with the tracebacks, see [the ``traceback`` module](http://docs.python.org/library/traceback.html). – Gareth Latty Apr 13 '12 at 12:34
Ah, I thought [some chicanery with `sys.exc_info()` was required to get the real culprit in the traceback](http://nedbatchelder.com/blog/200711/rethrowing_exceptions_in_python.html). Is that an artifact of old-Python? – Li-aung Yip Apr 13 '12 at 13:01
I believe that's not true in Python 3.x, and excuse my earlier comment, I have clearly done too much Java recently, ``except IndexError as e: ... raise e``, not ``catch``. – Gareth Latty Apr 13 '12 at 13:21

Abhijit · Answer 3 · 2012-04-13T11:50:17.057

2

Definitely, when would itertools.chain and slicing come into help?

for thing in itertools.islice(itertools.chain(*mylist),1,None,len(mylist)):
    print(thing)

Numpy is also helpful for column slicing. Here is another example in numpy

for thing in numpy.array(mylist)[:,1]:
    print(thing)

edited Apr 13 '12 at 11:50

answered Apr 13 '12 at 11:39

Abhijit

62,056
18
131
204

While these work, the first example is pretty unclear and hard to read, while adding numpy as a dependency just for this is overkill unless you are already using it. – Gareth Latty Apr 13 '12 at 11:49
@Lattyware: Not sure about the readability part as its a relative term. And yeah, unless you are using numpy this is an overkill. But I always love it the way it can manipulate arrays – Abhijit Apr 13 '12 at 11:52
It is definitely a powerful tool, and worth knowing about. If the OP was already using numpy, or could benefit from using it throughout his code, I would say it's definitely a good choice. – Gareth Latty Apr 13 '12 at 11:54
thanks for the suggestions but numpy is not an option as im bringing a frame work of existing code up to various required standards while avoiding making large changes (like a new dependency) – Jacxel Apr 13 '12 at 13:05

score 1 · Answer 4 · edited May 23 '17 at 12:02

While I like Dikei's answer for clarity and terseness, I still believe that a good option is simply:

for sublist in mylist:
    item = sublist[1]
    ...
    do_stuff(item)
    ...
    do_other_stuff(item)
    ...

It remains clear, can be expanded to do more easily, and is probably the fastest.

Here are some quick tests - I'm not sure about how accurate they will be thanks to doing nothing in the loop, but they probably give an idea:

python -m timeit -s "mylist = [range(1,8) for _ in range(1,8)]" 'for thing in mylist:' '    item=thing[1]' '    pass'
1000000 loops, best of 3: 1.25 usec per loop

python -m timeit -s "mylist = [range(1,8) for _ in range(1,8)]" 'for thing in (i[1] for i in mylist):' '    pass'
100000 loops, best of 3: 2.37 usec per loop

python -m timeit -s "mylist = [range(1,8) for _ in range(1,8)]" 'for thing in itertools.islice(itertools.chain(*mylist),1,None,len(mylist)):' '    pass'
1000000 loops, best of 3: 2.21 usec per loop

python -m timeit -s "import numpy" -s "mylist = numpy.array([range(1,8) for _ in range(1,8)])" 'for thing in mylist[:,1]:' '    pass' 
1000000 loops, best of 3: 1.7 usec per loop

python -m timeit -s "import numpy" -s "mylist = [range(1,8) for _ in range(1,8)]" 'for thing in numpy.array(mylist)[:,1]:' '    pass'
10000 loops, best of 3: 63.8 usec per loop

Note that numpy is fast if once generated, but very slow to generate on demand for a single operation.

On large lists:

python -m timeit -s "mylist = [range(1,100) for _ in range(1,100)]" 'for thing in mylist:' '    item=thing[1]' '    pass'
100000 loops, best of 3: 16.3 usec per loop

python -m timeit -s "mylist = [range(1,100) for _ in range(1,100)]" 'for thing in (i[1] for i in mylist):' '    pass'
10000 loops, best of 3: 27 usec per loop

python -m timeit -s "mylist = [range(1,100) for _ in range(1,100)]" 'for thing in itertools.islice(itertools.chain(*mylist),1,None,len(mylist)):' '    pass'
10000 loops, best of 3: 101 usec per loop

python -m timeit -s "import numpy" -s "mylist = numpy.array([range(1,100) for _ in range(1,100)])" 'for thing in mylist[:,1]:' '    pass'
100000 loops, best of 3: 8.47 usec per loop

python -m timeit -s "import numpy" -s "mylist = [range(1,100) for _ in range(1,100)]" 'for thing in numpy.array(mylist)[:,1]:' '    pass'
100 loops, best of 3: 3.82 msec per loop

Remember that speed should always come second to readability, unless you really need it.

score 1 · Answer 5 · answered Apr 13 '12 at 13:03

1

The method itemgetter() can be used to solve this:

from operator import itemgetter

def mymethod():
    return [[1,2,3,4],
            [1,2,3,4],
            [1,2,3,4],
            [1,2,3,4]]

mylist = mymethod()

row = map(itemgetter(2), mylist)
print("row %s" % row)

thing = row[-1]

# this bit is meant to be outside the for loop, 
# I mean it to represent the last value thing was in the for
if thing:
    print thing

The output is:

row [3, 3, 3, 3]
3

answered Apr 13 '12 at 13:03

Andreas Florath

4,418
22
32

Why bother using an itemgetter and map when the much better option of the generator expression allows you to express it in terms clear to any normal Python user - ``(sublist[2] for sublist in list)``? – Gareth Latty Apr 13 '12 at 13:06
Please explain 'better option' and 'normal Python user'? IMHO the `map` expression is much clearer - especially when the user has some background in lisp / scheme [link] (http://c2.com/cgi/wiki?MapFunction). – Andreas Florath Apr 13 '12 at 13:11
This is exactly the thing - map is normal for users of functional languages, which is not the average user. List comps, generator expressions and co are very clear as they use existing Python syntax, and will be faster thanks to not making an itemgetter. In Python 2.x, map is also not a generator, and slower on that count. In general, the use of ``map`` is discouraged unless it's extremely simple (``map(int, some_list)`` for example). – Gareth Latty Apr 13 '12 at 13:25
Sorry - but you did not answer my questions. You are arguing with words like 'average', 'better' or 'normal' but you don't explain what you are meaning. More questions concerning your last comment: What is a 'average user'? You? Who decides what is 'extremely simple'? You? – Andreas Florath Apr 14 '12 at 06:21
I'm talking about what the [official docs](http://docs.python.org/tutorial/datastructures.html#list-comprehensions) say. What [SO says](http://stackoverflow.com/a/1247490/722121) - what everyone says. List comprehensions are clearer. You might think otherwise, but you are a minority. – Gareth Latty Apr 14 '12 at 13:54

For loop for second items in list of lists only. (python)

What I have

What I want

5 Answers5