Recursively accessing paths and values of a nested dictionary

Question

In Python 2.7, how does one dynamically access and print out the keys and values of a nested dictionary? Here's a nonsensical example: https://jsoneditoronline.org/?id=da7a486dc2e24bf8b94add9f04c71b4d

Normally, I would do something like:

import json

json_sample = 'sample_dict.json'
json_file = open(json_sample, 'r')
json_data = json.load(json_file)

items = json_data['sample_dict']

for item in items:
    dict_id = item['dict_id']
    person = item['person']['person_id']
    family = item['family']['members']

    print dict_id
    print person
    print family

I can hard code it like this and it'll give me desirable results, but how would I access each of the keys and values dynamically so that:

The first row just prints the keys (dict_id, person['person_id'], person['name'], family['members']['father'])
The second row prints the values respectively (5, 15, "Martin", "Jose")

The end result should be in a CSV file.

do you know that the JSON data is a list of dicts? If so, you can just use `item.keys()` for the keys and `item.values()` for the values. — RishiG, Apr 27 '18 at 17:31
Oh yeah, so if I do `item.keys()`, it would just give me the parent dicts themselves, not including the child ones. `item.values()` would then give me those, but it'll include the keys, when I just want the values only. — cheebdragonite, Apr 27 '18 at 17:40

fferri · Accepted Answer · 2018-04-27T20:52:52.980

4

You can use a recursive visitor/generator which returns all the path/value pairs of the leaves:

def visit_dict(d, path=[]):
    for k, v in d.items():
        if not isinstance(v, dict):
            yield path + [k], v
        else:
            yield from visit_dict(v, path + [k])

(replace the yield from ... with the appropriate equivalent if using Python < 3.4)

Getting the keys:

>>> ','.join('/'.join(k) for k, v in visit_dict(json_data['sample_dict'][0]))
'dict_id,person/person_id,person/name,person/age,family/person_id,family/members/father,family/members/mother,family/members/son,family/family_id,items_id,furniture/type,furniture/color,furniture/size,furniture/purchases'

and the values:

>>> ','.join(str(v) for k, v in visit_dict(json_data['sample_dict'][0]))
'5,15,Martin,18,20,Jose,Maddie,Jerry,2,None,Chair,Brown,Large,[]'

edited Apr 27 '18 at 20:52

answered Apr 27 '18 at 17:40

fferri

18,285
5
46
95

Oh, this is a better solution! But now I'm wondering, is there a way to iterate through each dictionary (`dict_list` through `purchases`) within the whole dictionary of the sample_dict list? Rather than hardcoding `json_data['sample_dict'][0]`, is there a way to make the `[0]` part dynamic so that the rest of the dictionaries are also yielded in each row? – cheebdragonite Apr 28 '18 at 16:40
Yes, a `for` loop – fferri Apr 28 '18 at 16:42
To be clear: `for d in json_data['sample_dict']: csv_file.write(','.join(str(v) for k, v in visit_dict(d)))` – fferri Apr 28 '18 at 21:04
Yep! Initially, I did `for key in json_data['sample_dict']: csv_file.write(','.join('/'.join(k) for k, v in visit_dict(json_data['sample_dict'][key])))`, only to realize that I iterated through it incorrectly, and that it's just `...visit_dict(key)))`. I do have duplicate keys, however, but that's a different issue for now. – cheebdragonite Apr 28 '18 at 23:38

Recursively accessing paths and values of a nested dictionary

1 Answers1

Linked