Finding Duplicate Values in a Dictionary

Question

I'm writing a script to find duplicated values in a dictionary. My dictionary duties has an integer key and a list as a value. My task is to make a new dictionary duties_without_duplicates without duplicate values.

duties = { 
    0: ["Submit an assignment","Submit an assignment"],
    1: ["Submit an assignment", "Finish a book review"]
}

duties_without_duplicates = {
    0: ["Submit an assignment"],
    1: ["Submit an assignment", "Finish a book review"]
}

Can anyone help me how to solve this problem?

Does this answer your question? [How do I remove duplicates from a list, while preserving order?](https://stackoverflow.com/questions/480214/how-do-i-remove-duplicates-from-a-list-while-preserving-order) — Kraigolas, Sep 14 '22 at 15:29

score 1 · Answer 1 · answered Sep 14 '22 at 15:29

If the order of items in your lists is not important, you can simply pass the lists to a set constructor -

duties = { 
    0: ["Submit an assignment","Submit an assignment"],
    1: ["Submit an assignment", "Finish a book review"]
}
duties_without_duplicates = {k: list(set(v)) for k, v in duties.items()}

Output

{0: ['Submit an assignment'],
 1: ['Submit an assignment', 'Finish a book review']}

ThePyGuy · Answer 2 · 2022-09-14T15:39:49.587

1

You can use grouby from itertools, and use the keys provided by groupby iterator for the unique values in the list, and you can implement a dictionary comprehension to achieve that:

>>> from itertools import groupby
>>> {key:[k for k,v in groupby(values)] for key,values in duties.items()}

{0: ['Submit an assignment'], 1: ['Submit an assignment', 'Finish a book review']}

As I've mentioned in comments as well, above solution will work only if the values list is sorted, otherwise, you can use following method:

result = {}
for key,values in duties.items():
    temp = []
    for v in values:
        if v not in temp:
            temp.append(v)
    result[key]=temp

# result: 
{0: ['Submit an assignment'], 1: ['Submit an assignment', 'Finish a book review']}

edited Sep 14 '22 at 15:39

answered Sep 14 '22 at 15:30

ThePyGuy

17,779
5
18
45

1

Depending on the lists - `itertools.groupby` might not remove all duplicates. – Mortz Sep 14 '22 at 15:31
1

@Mortz, you are right, it'll work only if the list is sorted else it will not work. But since the data mentioned by OP is sorted, so I didn't think in that direction. – ThePyGuy Sep 14 '22 at 15:36
Yes - just wanted to call it out as the results of `groupby` on an unsorted list might be surprising – Mortz Sep 14 '22 at 15:39
1

@Mortz, thanks for the comment, I've added alternate solution as well. – ThePyGuy Sep 14 '22 at 15:40

Finding Duplicate Values in a Dictionary

2 Answers2