Remove duplicate of a list via list matching in Python

Question

I have chosen a slightly different procedure to remove duplicates of a list. I want to keep a new list in parallel, in which each duplicate is added. Afterwards I check if the element is present in the "newly created list" in order to delete it.

The code looks like this:

# nums = [1,1,2] or [0,0,1,1,1,2,2,3,3,4]

t = []
nums_new = nums
for i in nums:
    if nums[i] not in t:
        t.append(nums[i])
    else:
        nums_new.remove(nums[i])
nums = nums_new
print(nums)

For the case when nums = [1,1,2] this works fine and returns [1,2].

However, for nums = [0,0,1,1,1,2,2,3,3,4] this case does not seem to work as I get the following output: [0, 1, 2, 2, 3, 3, 4].

Why is this? Can someone explain me the steps?

`for i in nums` makes `i` an element of `nums`, not an index. so instead of `nums[i]` you should just use `i` — Sembei Norimaki, Feb 13 '23 at 15:13
@SembeiNorimaki if i do so, i still get an incorrect list somehow... — 41 72 6c, Feb 13 '23 at 15:19
please, edit your question with the updated code, the input list you are using and the result you are getting — Sembei Norimaki, Feb 13 '23 at 15:20
Is there a motivation not to do `nums = list(set(nums))`? Note that removing elements from a list while iterating over is fraught with peril. — JonSG, Feb 13 '23 at 15:25
If only you hadn't chosen test cases where all the elements are valid indices, for instance `nums = ['1', '1', '2']`... (Doesn't what you want to do just leave you with two identical lists?) — molbdnilo, Feb 13 '23 at 15:36

Michael Currie · Accepted Answer · 2023-02-13T15:31:51.290

There are two issues with your code:

Since you are iterating over a list, for i in nums, i is your actual number, not the indexer, so you should just use i instead of nums[i].
nums_new = nums will not actually make a copy of nums, but instead it will make a second binding to the very same list. So you should write nums_new = nums.copy() to avoid changing your original list while you iterate over it.

With those two changes your code works as you wish:

nums = [0,0,1,1,1,2,2,3,3,4]

t = []
nums_new = nums.copy()
for i in nums:
    if i not in t:
        t.append(i)
    else:
        nums_new.remove(i)

nums = nums_new

print(nums)

Returns [0,1,2,3,4].

Of course this is an academic exercise of some kind, but note that the Pythonic way to remove duplicates from a list is:

list(dict.fromkeys(nums)) if you want to preserve the order, and
list(set(nums)) for a slightly faster method if you do not care about order.

i know this approach as well, but my usecase was such that i had to choose this approach. But I must have forgotten the `.copy()`. thanks for the explanation! — 41 72 6c, Feb 13 '23 at 15:37

Remove duplicate of a list via list matching in Python

1 Answers1