309

I have a list consisting of like 20000 lists. I use each list's 3rd element as a flag. I want to do some operations on this list as long as at least one element's flag is 0, it's like:

my_list = [["a", "b", 0], ["c", "d", 0], ["e", "f", 0], .....]

In the beginning, all flags are 0. I use a while loop to check if at least one element's flag is 0:

def check(list_):
    for item in list_:
        if item[2] == 0:
            return True
    return False

If check(my_list) returns True, then I continue working on my list:

while check(my_list):
    for item in my_list:
        if condition:
            item[2] = 1
        else:
            do_sth()

Actually, I wanted to remove an element in my_list as I iterated over it, but I'm not allowed to remove items as I iterate over it.

Original my_list didn't have flags:

my_list = [["a", "b"], ["c", "d"], ["e", "f"], .....]

Since I couldn't remove elements as I iterated over it, I invented these flags. But the my_list contains many items, and while loop reads all of them at each for loop, and it consumes lots of time! Do you have any suggestions?

Brian61354270
  • 8,690
  • 4
  • 21
  • 43
alwbtc
  • 28,057
  • 62
  • 134
  • 188
  • 4
    Looks like your data structure is not ideal for your problem. If you explained the context a little more maybe we could suggest something more appropriate. – uselpa May 19 '12 at 14:50
  • Maybe you could replace the items with `None` or `[]` as you iterate over the list instead of removing them. Checking the whole list with 'check()` iterating over all the items before each pass on the inner loop is a very slow approach. – martineau May 19 '12 at 20:08

5 Answers5

542

The best answer here is to use all(), which is the builtin for this situation. We combine this with a generator expression to produce the result you want cleanly and efficiently. For example:

>>> items = [[1, 2, 0], [1, 2, 0], [1, 2, 0]]
>>> all(flag == 0 for (_, _, flag) in items)
True
>>> items = [[1, 2, 0], [1, 2, 1], [1, 2, 0]]
>>> all(flag == 0 for (_, _, flag) in items)
False

Note that all(flag == 0 for (_, _, flag) in items) is directly equivalent to all(item[2] == 0 for item in items), it's just a little nicer to read in this case.

And, for the filter example, a list comprehension (of course, you could use a generator expression where appropriate):

>>> [x for x in items if x[2] == 0]
[[1, 2, 0], [1, 2, 0]]

If you want to check at least one element is 0, the better option is to use any() which is more readable:

>>> any(flag == 0 for (_, _, flag) in items)
True
Gareth Latty
  • 86,389
  • 17
  • 178
  • 183
  • My fault on the use of lambda, Python's all does not accept a function as the first argument like Haskell et. al., I changed my answer to a list comprehension as well. :) – Hampus Nilsson May 19 '12 at 15:11
  • 4
    @HampusNilsson A list comprehension is not the same as a generator expression. As `all()` and `any()` short circuit, if, for example, the first value on mine evaluates to `False`, `all()` will fail and not check any more values, returning `False`. Your example will do the same, except it will generate the entire list of comparisons first, meaning a lot of processing for nothing. – Gareth Latty May 19 '12 at 15:18
  • it also works if you do `all([ cond(i) for i in range (n) ])` – Charlie Parker May 10 '21 at 15:59
  • 2
    @CharlieParker Making it a list comprehension like that just makes things slower, as discussed in my previous comment. You want to use a generator expression (no square brackets) because that way it can short-circuit. – Gareth Latty May 10 '21 at 16:01
  • @GarethLatty ah, interesting point. Not sure why the optimization works but I didn't realize that `all` worked for lists as inputs which was the use case I cared about (e.g. debugging or lists that already exist). Thanks for the tip though! – Charlie Parker May 10 '21 at 16:09
  • `all` works for any iterable. – Karl Knechtel Sep 18 '22 at 08:08
28

If you want to check if any item in the list violates a condition use all:

if all([x[2] == 0 for x in lista]):
    # Will run if all elements in the list has x[2] = 0 (use not to invert if necessary)

To remove all elements not matching, use filter

# Will remove all elements where x[2] is 0
listb = filter(lambda x: x[2] != 0, listb)
Hampus Nilsson
  • 6,692
  • 1
  • 25
  • 29
  • 5
    You can remove `[...]` in `all(...)` since it can then create a generator instead of a list, which not only saves you two characters but also saves memory and time. By using generators, only one item will be calculated at a time (former results will be dropped since no longer used) and if any of them turns out `False`, the generator will stop calculating the rest. – InQβ May 22 '18 at 12:29
  • 1
    Note for python3; `filter()` returns an iterable, not a list. Thus the line would be `listb = list(filter(lamba x: x[2] != 0, listb))` – Meg Anderson Aug 10 '20 at 18:10
8

You could use itertools's takewhile like this, it will stop once a condition is met that fails your statement. The opposite method would be dropwhile

for x in itertools.takewhile(lambda x: x[2] == 0, list)
    print x
Hedde van der Heide
  • 21,841
  • 13
  • 71
  • 100
1

this way is a bit more flexible than using all():

my_list = [[1, 2, 0], [1, 2, 0], [1, 2, 0]]
all_zeros = False if False in [x[2] == 0 for x in my_list] else True
any_zeros = True if True in [x[2] == 0 for x in my_list] else False

or more succinctly:

all_zeros = not False in [x[2] == 0 for x in my_list]
any_zeros = 0 in [x[2] for x in my_list]
mulllhausen
  • 4,225
  • 7
  • 49
  • 71
  • Couldn't you simply say `all_zeros = False in [x[2] == 0 for x in my_list]` or even `0 in [x[2] for x in my_list]` and correspondingly for `any_zeros`? I don't really see any remarkable improvement over `all()`. – tripleee Jul 02 '19 at 10:59
  • no, your version - `all_zeros = False in [x[2] == 0 for x in my_list]` evaluates to `False`, while mine evaluates to `True`. If you change it to `all_zeros = not (False in [x[2] == 0 for x in my_list])` then it is equivalent to mine. And `0 in [x[2] for x in my_list]` is obviously only going to work for `any_zeros`. But I do like the succinctness of your idea, so I will update my answer – mulllhausen Jul 04 '19 at 07:41
0

Another way to use itertools.ifilter. This checks truthiness and process (using lambda)

Sample-

for x in itertools.ifilter(lambda x: x[2] == 0, my_list):
    print x
Learner
  • 5,192
  • 1
  • 24
  • 36