1

i am refreshing my python (2.7) and i am discovering iterators and generators. As i understood, they are an efficient way of navigating over values without consuming too much memory. So the following code do some kind of logical indexing on a list: removing the values of a list L that triggers a False conditional statement represented here by the function f.

I am not satisfied with my code because I feel this code is not optimal for three reasons:

  • I read somewhere that it is better to use a for loop than a while loop. However, in the usual for i in range(10), i can't modify the value of 'i' because it seems that the iteration doesn't care.

  • Logical indexing is pretty strong in matrix-oriented languages, and there should be a way to do the same in python (by hand granted, but maybe better than my code).

  • Third reason is just that i want to use generator/iterator on this example to help me understand.

Third reason is just that i want to use generator/iterator on this example to help me understand.

TL;DR : Is this code a good pythonic way to do logical indexing ?

#f string -> bool
def f(s):
    return 'c' in s

L=['','a','ab','abc','abcd','abcde','abde']  #example
length=len(L)
i=0
while i < length:
    if not f(L[i]): #f is a conditional statement (input string output bool)
        del L[i]
        length-=1 #cut and push leftwise

    else:
        i+=1
    print 'Updated list is :', L
    print length
rafaelc
  • 57,686
  • 15
  • 58
  • 82
Victor I let RAM
  • 13
  • 3
  • Please fix your indentation, in particular that `if` statement. What is the `else` associated with: the `if` or the `while``? – Daniel Roseman Feb 16 '16 at 15:14

5 Answers5

3

This code has a few problems, but the main one is that you must never modify a list you're iterating over. Rather, you create a new list from the elements that match your condition. This can be done simply in a for loop:

newlist = []
for item in L:
    if f(item):
        newlist.append(item)

which can be shortened to a simple list comprehension:

newlist = [item for item in L if f(item)]
Daniel Roseman
  • 588,541
  • 66
  • 880
  • 895
  • Okay. I was feeling like you could modify a list when iterating over it if you knew what you were doing. The list comprehension is the kind of code i was searching for. However, your solution does not mutate the original list L. I don't feel like just writing 'L=[item for item in L if f(item)]' does the job. Thanks for your answer – Victor I let RAM Feb 16 '16 at 15:27
  • Well, you could do `L[:] = [item for item in L if f(item)]`, which *does* mutate the list. – Daniel Roseman Feb 16 '16 at 15:37
1

It looks like filter() is what you're after:

newlist = filter(lambda x: not f(x), L)

filter() filters (...) an iterable and only keeps the items for which a predicate returns True. In your case f(..) is not quite the predicate but not f(...).

Simpler:

def f(s):
   return 'c' not in s

newlist = filter(f, L)

See: https://docs.python.org/2/library/functions.html#filter

rafaelc
  • 57,686
  • 15
  • 58
  • 82
emvee
  • 4,371
  • 23
  • 23
0

"The python way" to do it would be to use a generator expression:

# list comprehension
L = [l for l in L if f(l)]
# alternative generator comprehension
L = (l for l in L if f(l))

It depends on your context if a list or a generator is "better" (see e.g. this so question). Because your source data is coming from a list, there is no real benefit of using a generator here.

Community
  • 1
  • 1
syntonym
  • 7,134
  • 2
  • 32
  • 45
  • The generator is not quite the same; you don't get a list out of it, unless you explicitly call `list(L)`. But if you do that you might as well have used the list comprehension in the first place. – Daniel Roseman Feb 16 '16 at 15:17
  • While it's not the same you can iterate over a generator just like a list, so if you only want to do that a generator is "enough". Of course if you want/need a list the list comprehension is "better". – syntonym Feb 16 '16 at 15:23
0

Never modify a list with del, pop or other methods that mutate the length of the list while iterating over it. Read this for more information.

The "pythonic" way to filter a list is to use reassignment and either a list comprehension or the built-in filter function:

List comprehension:

>>> [item for item in L if f(item)]
['abc', 'abcd', 'abcde']

i want to use generator/iterator on this example to help me understand

The for item in L part is implicitly making use of the iterator protocol. Python lists are iterable, and iter(somelist) returns an iterator .

>>> from collections import Iterable, Iterator
>>> isinstance([], Iterable)
True
>>> isinstance([], Iterator)
False
>>> isinstance(iter([]), Iterator)
True

__iter__ is not only being called when using a traditional for-loop, but also when you use a list comprehension:

>>> class mylist(list):
...     def __iter__(self):
...         print('iter has been called')
...         return super(mylist, self).__iter__()
... 
>>> m = mylist([1,2,3])
>>> [x for x in m]
iter has been called
[1, 2, 3]

Filtering:

>>> filter(f, L)
['abc', 'abcd', 'abcde']

In Python3, use list(filter(f, L)) to get a list.

Of course, to filter a list, Python needs to iterate over it, too:

>>> filter(None, mylist())
iter has been called
[]
Community
  • 1
  • 1
timgeb
  • 76,762
  • 20
  • 123
  • 145
0

For simply deleting elements, especially if the original list is no longer needed, just iterate backwards:

Python 2.x:

for i in xrange(len(L) - 1, -1, -1):
    if not f(L[i]):
        del L[i]

Python 3.x:

for i in range(len(L) - 1, -1, -1):
    if not f(L[i]):
        del L[i]

By iterating from the end, the "next" index does not change after deletion and a for loop is possible. Note that you should use the xrange generator in Python 2, or the range generator in Python 3, to save memory*.

In cases where you must iterate forward, use your given solution above.

*Note that Python 2's xrange will break if there are >= 2 ** 32 - 1 elements. Python 3's range, as well as the less efficient Python 2's range do not have this limitation.

Pi Marillion
  • 4,465
  • 1
  • 19
  • 20