Python - if statement inside a list comprehension

Question

I'm writing a piece of code that should take a list with several numbers (like this for example: [2,3,5,6,6]) and create a new list, removing all duplicates (to end up with [2,3,5,6]).

The code I have is this:

first_list = [2,3,5,6,6]
second_list = []
second_list = [x for x in first_list if x not in second_list]

However, it makes second_list equal to first_list, and I don't understand why. How can I make it do what I explained above?

None of the `x` are in the second list when the comprehension is evaluated for the simple reason that the second list is empty. — John Coleman, Apr 09 '17 at 18:12
at the time your `second_list` would be empyt, it won't get updated during list comprehension. — bhansa, Apr 09 '17 at 18:12

James · Answer 1 · 2017-04-09T18:17:52.297

2

The assignment to second_list happens after the list comprehension has completed. It does not build second_list step by step. What you wrote is equivalent to doing:

# this does not give the answer you want:
temp = [x for x in first_list if x not in second_list]
second_list = temp

This is because list comprehension builds the entire list in memory before is assigns it to second_list. If you want to check the list as you build it, you can use a regular for loop with append.

second_list = []
for x in first_list:
    if x not in second_list:
        second_list.append(x)

For a faster method, use set:

second_list = list(set(first_list))

However, this does not necessarily preserve the order of first_list

edited Apr 09 '17 at 18:17

answered Apr 09 '17 at 18:12

James

32,991
4
47
70

It's not working... – J. C. Apr 09 '17 at 18:16
The first code block is explaining why it will not work. Use the second or third code block for the correct method. – James Apr 09 '17 at 18:17

score 1 · Answer 2 · answered Apr 09 '17 at 18:12

1

Just use sets instead of lists to remove duplicates:

first_list = [2,3,4,5,6,6]
first_set = set(first_list)
print(first_set)

sets cannot contain duplicates

answered Apr 09 '17 at 18:12

ma3oun

3,681
1
21
33

I'm a beginner, so I didn't even know about sets. This works fine, thanks :) – J. C. Apr 09 '17 at 18:13
This assumes that the order of the elements isn't relevant. – John Coleman Apr 09 '17 at 18:13
I just did `newList = list(first_set) newList.sort()` – J. C. Apr 09 '17 at 18:23

score 1 · Answer 3 · answered Apr 09 '17 at 18:14

To remove all duplicates, it is best to use set(), like this:

first_list = [2, 3, 5, 6, 6]
second_list = list(set(first_list))

Now, second_list contains all that you need

If you want to use for loops:

first_list = [2, 2, 3, 4, 6, 6]
second_list = []
final_list = []
for i in first_list:
   if i not in second_list:
       final_list.append(i)
       second_list.append(i)

print final_list

score 0 · Answer 4 · answered Apr 09 '17 at 18:18

If you need to maintain order, one way is to use an OrderedDict like:

Code:

second_list = OrderedDict([(x, None) for x in first_list]).keys()

Test code:

from collections import OrderedDict
first_list = [2, 3, 5, 6, 6]
second_list = OrderedDict([(x, None) for x in first_list]).keys()
print(second_list)

Results:

[2, 3, 5, 6]

Python - if statement inside a list comprehension

4 Answers4