Change one dict/list python structure to another

Question

I have the following data

data={
       None: [
               {'ne': '1', 'na': '1'}, 
               {'ne': '2', 'na': '2'}, 
               {'ne': '3', 'na': '3'}, 
               {'ne': '4', 'na': '4'}
             ], 
      'AO': [
               {'ne': '2', 'na': '2'}, 
               {'ne': '6', 'na': '6'}
            ], 
      'NZ': [
               {'ne': '1', 'na': '1'}
            ]
      }

and I want to have a list from it like this:

[
  {'ne': '1', 'na': '1', 'country': [None, 'NZ']},
  {'ne': '2', 'na': '2', 'country': [None, 'AO']},
  {'ne': '3', 'na': '3', 'country': [None]},
  {'ne': '4', 'na': '4', 'country': [None]},
  {'ne': '6', 'na': '6', 'country': ['AO']}
]

my code is doing it fine but it's far from being "pythonic" because I'm a newbie at python:

data = {None: [{'ne': '1', 'na': '1'}, {'ne': '2', 'na': '2'}, {'ne': '3', 'na': '3'}, {'ne': '4', 'na': '4'}], 'AO': [{'ne': '2', 'na': '2'}, {'ne': '6', 'na': '6'}], 'NZ': [{'ne': '1', 'na': '1'}]}
data_list = []
for k,d in data.items():
    for dd in d:
        dd['country'] = k
        data_list.append(dd)
help_dict = {}
for item in data_list:
    help_dict[item['ne']] = False
final_list = []
for idx, val in enumerate(data_list):
    if not help_dict[val['ne']]:
        val['country'] = [val['country']]
        for idx2, val2 in enumerate(data_list):
            if idx2 != idx and val['ne'] == val2['ne']:
                val['country'].append(val2['country'])
        help_dict[val['ne']] = True
        final_list.append(val)
print(final_list)

can someone help me with a better way to do this?

Please [edit] the question and make the title more descriptive. The current one is too vague. For reference, see [ask]. — wjandrea, Jan 05 '21 at 22:24
@PeptideWitch actually `na` is a translated field for `ne` so they are not exactly the same but I wrote it like this for better understandin, and if two dictionaries had the same `ne` they will have the same `na` for sure — Miriam Arbaji, Jan 06 '21 at 06:11

score 1 · Answer 1 · answered Jan 05 '21 at 22:56

This is a really naive approach to solve your problem, due to it requiring that the inner dictionaries are sorted in the same order for it to "match" earlier found dictionaries.

For more complex dictionaries inside of the country this might not give correct results:

data={
       None: [
               {'ne': '1', 'na': '1'},
               {'ne': '2', 'na': '2'},
               {'ne': '3', 'na': '3'},
               {'ne': '4', 'na': '4'}
             ],
      'AO': [
               {'ne': '2', 'na': '2'},
               {'ne': '6', 'na': '6'}
            ],
      'NZ': [
               {'ne': '1', 'na': '1'}
            ]
      }

d = {}
for country in data:
    for dictionary in data[country]:
        # Create a key that is a string of the dictionary, and value is dictionary plus country
        x = d.setdefault(str(dictionary), dictionary | {"country": []})
        # If you're using Python < 3.9, use this instead:
        # x = d.setdefault(str(dictionary), {**dictionary, "country": []})
        x["country"].append(country)

# pprint only used to represent data better
import pprint
pprint.pp(list(d.values()))

Output:

[{'ne': '1', 'na': '1', 'country': [None, 'NZ']},
 {'ne': '2', 'na': '2', 'country': [None, 'AO']},
 {'ne': '3', 'na': '3', 'country': [None]},
 {'ne': '4', 'na': '4', 'country': [None]},
 {'ne': '6', 'na': '6', 'country': ['AO']}]

To make it un-naive, replace `str(dictionary)` with `frozenset(dictionary.items())`. — wjandrea, Jan 05 '21 at 23:43
You can make it version-independent by breaking up the steps differently: `x = d.setdefault(..., dictionary.copy()); x.setdefault("country", []).append(country)` — wjandrea, Jan 05 '21 at 23:44

wjandrea · Answer 2 · 2021-01-05T23:21:50.013

Firstly, I'm assuming ne and na are always the same.

An optimal intermediate data structure is a dict with ne/na as keys and country lists as values:

{'1': [None, 'NZ'],
 '2': [None, 'AO'],
 '3': [None],
 '4': [None],
 '6': ['AO']}

Once you have that goal in mind, it's super simple to do it Pythonically:

inter = {}
for k, dicts in data.items():
    for d in dicts:
        inter.setdefault(d['ne'], []).append(k)

dict.setdefault() is used to get the value if it exists, or if not, set it to a default, which is an empty list here. It's functionally the same as this:
```
ne = d['ne']
if ne not in inter:
    inter[ne] = []
inter[ne].append(k)
```
You could also use collections.defaultdict(list) to do the same thing even more easily.

And once you have that dict, you just need to unpack it into a list of dicts:

result = [{'ne': ne, 'na': ne, 'country': c} for ne, c in inter.items()]

Which becomes:

[{'ne': '1', 'na': '1', 'country': [None, 'NZ']},
 {'ne': '2', 'na': '2', 'country': [None, 'AO']},
 {'ne': '3', 'na': '3', 'country': [None]},
 {'ne': '4', 'na': '4', 'country': [None]},
 {'ne': '6', 'na': '6', 'country': ['AO']}]

coderoftheday · Accepted Answer · 2021-01-06T10:26:17.700

new = [x for key,value in data.items() for x in value]

# remove duplicate dictionaries
new = [dict(t) for t in {tuple(d.items()) for d in new}]

for d in new:
    d['country'] = [key for key,data in data.items() if d in data]

print(new)

>>> [{'ne': '2', 'na': '2', 'country': [None, 'AO']}, 
     {'ne': '4', 'na': '4', 'country': [None]}, 
     {'ne': '1', 'na': '1', 'country': [None, 'NZ']}, 
     {'ne': '6', 'na': '6', 'country': ['AO']}, 
     {'ne': '3', 'na': '3', 'country': [None]}]

If you want to preserve the order

new = [x for n,(key,value) in enumerate(data.items()) for x in value]

seen = set()
new_l = []
for d in new:
    t = tuple(d.items())
    if t not in seen:
        seen.add(t)
        new_l.append(d)

for d in new_l:
    d['country'] = [key for key,data in data.items() if d in data]

print(new_l)

>>> [{'ne': '1', 'na': '1', 'country': [None, 'NZ']}, 
     {'ne': '2', 'na': '2', 'country': [None, 'AO']}, 
     {'ne': '3', 'na': '3', 'country': [None]}, 
     {'ne': '4', 'na': '4', 'country': [None]}, 
     {'ne': '6', 'na': '6', 'country': ['AO']}]

The enumerate was redundant, so I removed it – coderoftheday Jan 06 '21 at 10:26 — coderoftheday, Jan 06 '21 at 10:26

Change one dict/list python structure to another

3 Answers3

Linked