Insert/Add a json object in a dictionary at nested level dynamically

Question

I am working on a script to generate some test data based on a json spec. The intention of this script is to construct a json object/python dict record

To simplify things, I am using a list items here that represents my source items, which also represent the path where the value should be inserted.

Here's my intended output -

{
    "access": {
        "device": {
            "java": {
                "version": "Test Data"
            },
            "python": {
                "version": "Test Data"
            }
        },
        "type": "Test Data"
    },
    "item1": 1,
    "item2": 0
}

I am able to build the nested objects but they are all getting inserted at first level of the dictionary instead.

How can I use dest_path to store the result in the intended location?

Source:

import json
import random

def get_nested_obj(items: list):
    """
    Construct a nested json object
    """
    
    res = 'Test Data'

    for item in items[::-1]:
        res = {item: res}

    return res

def get_dest_path(source_fields):
    """
    Construct dest path where result from `get_nested_obj` should go
    """

    dest_path = ''

    for x in source_fields:
        dest_path += f'[\'{x}\']'
    
    return 'record'+dest_path

record = {}
items = ['access.device.java.version', 'access.device.python.version', 'access.type', 'item1', 'item2']

for item in items:
    if '.' in item:
        source_fields = item.split('.')

        temp = record
        for i, source_field in enumerate(source_fields):
            if source_field in temp:
                temp = temp[source_field]
                continue

            res = get_nested_obj(source_fields[i+1:])

            dest_path = get_dest_path(source_fields[:i])
            print(dest_path)

            record[source_field] = res # Here's the problem. How to use dest_path here?
            break
    else:
        record[item] = random.randint(0, 1)
            
print(json.dumps(record))

My output:

{
    "access": {
        "device": {
            "java": {
                "version": "Test Data"
            }
        }
    },
    "python": {
        "version": "Test Data"
    },
    "type": "Test Data",
    "item1": 1,
    "item2": 0
}

So you just want to write helpers to get/set values in a nested dictionary using a string with the dot-separated list of keys? I'd suggest working with separate unit tests which each make one call of the function, instead of doing multiple things at once. — Kenny Ostrom, Sep 27 '22 at 16:56

score 1 · Accepted Answer · answered Sep 27 '22 at 17:08

To construct the record dictionary from the items list you can use next example:

import random

record = {}
items = [
    "access.device.java.version",
    "access.device.python.version",
    "access.type",
    "item1",
    "item2",
]

for i in items:
    i = i.split(".")

    if len(i) == 1:
        record[i[0]] = random.randint(0, 1)
    else:
        r = record
        for v in i[:-1]:
            r.setdefault(v, {})
            r = r[v]
        r[i[-1]] = "Test Data"

print(record)

Prints:

{
    "access": {
        "device": {
            "java": {"version": "Test Data"},
            "python": {"version": "Test Data"},
        },
        "type": "Test Data",
    },
    "item1": 1,
    "item2": 1,
}

JL Peyret · Answer 2 · 2022-09-27T18:17:56.053

Not very different from other answer, but recursive.

import json
items = ['access.device.java.version', 'access.device.python.version', 'access.type', 'item1', 'item2']

def populate(di, item):
    parts = item.split(".", maxsplit=1)
    key = parts[0]

    if len(parts) == 1:
        if key.startswith("item"):
            v = 1
        else:
            v = "Test Data"
        di[key] = v
    else:
        dikey = di.setdefault(key, {})
        populate(dikey,parts[1])

    return di

di = {}
for item in items:
    populate(di,item)

print(json.dumps(di, indent=4))

output:

{
    "access": {
        "device": {
            "java": {
                "version": "Test Data"
            },
            "python": {
                "version": "Test Data"
            }
        },
        "type": "Test Data"
    },
    "item1": 1,
    "item2": 1
}

And here's a version that directly specifies the data, which would probably be more useful (and that return di is also unnecessary in both cases):

import json
items = [('access.device.java.version',"Test Data"), ('access.device.python.version', "Test Data"), ('access.type', "type data"), ('item1',1), ('item2',2)]

def populate(di, item):

    parts = item[0].split(".", maxsplit=1)
    key = parts[0]
    v = item[1]

    if len(parts) == 1:
        di[key] = v
    else:
        dikey = di.setdefault(key, {})
        populate(dikey,(parts[1],v))
        

record = {}
for item in items:
    populate(record,item)

print(json.dumps(record, indent=4))

{
    "access": {
        "device": {
            "java": {
                "version": "Test Data"
            },
            "python": {
                "version": "Test Data"
            }
        },
        "type": "type data"
    },
    "item1": 1,
    "item2": 2
}

FWIW, tried collections.defaultdict for fun and that does not recurse itself.

Insert/Add a json object in a dictionary at nested level dynamically

2 Answers2

output: