How to create a nested dictionary in a for loop (not using defaultdict)?

Question

my input looks like this:

test_file = [['ref1', 'test1', 2],
             ['ref2', 'test1', 3],
             ['ref3', 'test2', 4],
             ['ref1', 'test2', 4],
             ['ref2', 'test2', 1],
             ['ref1', 'test1', 4],
             ['ref1', 'test1', 5]]

And I'm trying to get a nested dictionary like this:

desired_output = {'ref1':{'test1':[2,5,4]}, 
                  'ref1':{'test2':[4]}, 
                  'ref2':{'test1':[3]}, 
                  'ref2':{'test2':[1]}, 
                  'ref3':{'test2':[4]}}

I tried to use defaultdict by appending the values to the second key, however, I got this error:

AttributeError: 'collections.defaultdict' object has no attribute 'append'

So, I tried this:

for entry in test_file:
    nest1 = {}
    try:
        nest1[entry[1]].append(entry[2])
    except KeyError:
        nest1[entry[1]] = [entry[2]]
    try:
        mynestdict[entry[0]].append(nest1)
    except KeyError:
        mynestdict[entry[0]] = [nest1]

print(dict(mynestdict))

But I'm getting this:

{'ref1': [{'test1': [2]}, {'test2': [4]}, {'test1': [4]}, {'test1': [5]}], 
'ref2': [{'test1': [3]}, {'test2': [1]}], 
'ref3': [{'test2': [4]}]}

I'm not familiar with nested dictionaries and I really would like to understand them, any suggestions?

Show your code with `defaultdict()` so we can help you fix it. — Barmar, Mar 31 '23 at 17:11
your desired output is not valid, you can't have duplicate `ref1` keys. — Barmar, Mar 31 '23 at 17:15
ditto dictionary has a duplicate key alternate can be '{"ref1":{"test1":[2,4,5],"test2":[4]},"ref2":{"test1":[3],"test2":[1]},"ref3":{"test2":[4]}}' — mks2192, Mar 31 '23 at 17:31

score 1 · Answer 1 · answered Mar 31 '23 at 17:16

Since you're creating nested dictionaries, you need nested defaultdict().

result = defaultdict(lambda: defaultdict(list))

for key1, key2, value in test_file:
    result[key1][key2].append(value)

pprint(default_to_regular(result))
# Output:
{'ref1': {'test1': [2, 4, 5], 'test2': [4]},
 'ref2': {'test1': [3], 'test2': [1]},
 'ref3': {'test2': [4]}}

Get default_to_regular() from How to convert defaultdict of defaultdicts [of defaultdicts] to dict of dicts [of dicts]?

Michael M. · Accepted Answer · 2023-03-31T17:34:54.337

I think the output you're looking for is this, combining the duplicate keys:

{'ref1': {'test1': [2, 4, 5], 'test2': [4]}, 'ref2': {'test1': [3], 'test2': [1]}, 'ref3': {'test2': [4]}}

To get that, just check if the nested keys exist, and if they don't, then create them. You asked for code that doesn't use collections.defaultdict and it doesn't make it too much more:

test_file = [['ref1', 'test1', 2],
             ['ref2', 'test1', 3],
             ['ref3', 'test2', 4],
             ['ref1', 'test2', 4],
             ['ref2', 'test2', 1],
             ['ref1', 'test1', 4],
             ['ref1', 'test1', 5]]

d = {}
for el in test_file:
    if el[0] not in d:
        d[el[0]] = {}
    if el[1] not in d[el[0]]:
        d[el[0]][el[1]] = []

    d[el[0]][el[1]].append(el[2])

print(d)

score 1 · Answer 3 · answered Mar 31 '23 at 17:27

test_file = [['ref1', 'test1', 2],
             ['ref2', 'test1', 3],
             ['ref3', 'test2', 4],
             ['ref1', 'test2', 4],
             ['ref2', 'test2', 1],
             ['ref1', 'test1', 4],
             ['ref1', 'test1', 5]]


df = pd.DataFrame(test_file)


df.groupby([0, 1])[2].apply(np.array).unstack().agg(lambda x: x.dropna().to_dict(), axis=1).to_json()


'{"ref1":{"test1":[2,4,5],"test2":[4]},"ref2":{"test1":[3],"test2":[1]},"ref3":{"test2":[4]}}'

How to create a nested dictionary in a for loop (not using defaultdict)?

3 Answers3