inside list of dictionaries, merge lists based on key

Question

I have nested dictionaries in a list of dictionaries, I want to merge the lists based on 'id'

res = [{'i': ['1'], 'id': '123'},
       {'i': ['1'], 'id': '123'},
       {'i': ['1','2','3','4','5','6'],'id': '123'},
       {'i': ['1'], 'id': '234'},
       {'i': ['1','2','3','4','5'],'id': '234'}]

Desired output:

[{'i': [1, 1, 1, 2, 3, 4, 5, 6], 'id': '123'},
 {'i': [1, 1, 2, 3, 4, 5], 'id': '234'}]

I am trying to merge the nested dictionaries based on key "id". I couldn't figure out the best way out:

import collections
d = collections.defaultdict(list)
for i in res:
    for k, v in i.items():
        d[k].extend(v)

The above code is merging all the lists, but i wantto merge lists based on key "id".

Have you checked https://stackoverflow.com/questions/5501810/join-two-lists-of-dictionaries-on-a-single-key ? — CarlosSR, Apr 15 '20 at 07:38

score 3 · Accepted Answer · answered Apr 15 '20 at 07:38

3

Something like this should do the trick

from collections import defaultdict

merged = defaultdict(list)
for r in res:
    merged[r['id']].extend(r['i'])

output = [{'id': key, 'i': merged_list} for key, merged_list in merged.items()]

answered Apr 15 '20 at 07:38

michaeldel

2,204
1
13
19

score 1 · Answer 2 · answered Apr 15 '20 at 07:40

1

The following produces the desired output, using itertools.groupby:

from operator import itemgetter
from itertools import groupby

k = itemgetter('id')

[
    {'id': k, 'i': [x for d in g for x in d['i']]} 
    for k, g in groupby(sorted(res, key=k), key=k)
]

answered Apr 15 '20 at 07:40

user2390182

72,016
6
67
89

Thanks @schwobaseggl for the snippet! – louis Apr 15 '20 at 16:42

score 0 · Answer 3 · answered Apr 15 '20 at 07:42

I'm not sure what the expected behavior should be when there are duplicates -- for example, should the lists be:

treated like a set() ?
appended, and there could be multiple items, such as [1,1,2,3...] ?
doesn't matter -- just take any

Here would be one variation where we use a dict comprehension:

{item['id']: item for item in res}.values()
# [{'i': ['1', '2', '3', '4', '5'], 'id': '234'}, {'i': ['1', '2', '3', '4', '5', '6'], 'id': '123'}]

If you provide a bit more information in your question, I can update the answer accordingly.

@samuwelbrody1249 Thanks. I was looking to have duplicate elements in the list. — louis, Apr 15 '20 at 16:43

inside list of dictionaries, merge lists based on key

3 Answers3