how can I create nested dictionary keys and assign them values from a list of namespaced key value pairs?

Question

I have env vars that looks like this:

CONFIG-SOMEKEY-SOMEOTHERKEY = val345
CONFIG-SOMEKEY-SOMEOTHEROTHERKEY = val678
CONFIG-ANOTHERKEY = val222

I want to create a dictionary out of them that would look like:

{
  'SOMEKEY': {
    'SOMEOTHERKEY': 'val3242',
    'SOMEOTHEROTHERKEY': 'val678'
  }
  'ANOTHERKEY': 'val222'
}

"CONFIG-" is a prefix to denote which vars this should be done with- so I can filter them easily like this:

config_fields = [i for i in os.environ if i.startswith("CONFIG-")]

But I'm unsure of how to loop over the string, split on "-" and build a dict.

While looping I was thinking I could check if its the last item and assign the value but how would it know the full path of keys it's on?

I suspect this is a job for recursion I'm just now sure exactly how to implement it

thanks, that was a typo, fixed it – red888 Apr 19 '20 at 22:13 — red888, Apr 19 '20 at 22:13

gilch · Answer 1 · 2020-04-19T22:42:39.373

You can use the assoc_in function from toolz. Split the name on - and slice off the prefix.

import os

from toolz.dictoolz import assoc_in

CONFIG={}

for k, v in os.environ.items():
    if k.startswith("CONFIG-"):
        assoc_in(CONFIG, k.split('-')[1:], v)

If you don't want to add a dependency, you can see the implementation of assoc_in here. A simpler substitute might be something like

def assoc_in(d, ks, v):
    for k in ks[:-1]:
        d = d.setdefault(k, {})
    d[ks[-1]] = v

This uses the .setdefault() method to get the nested dicts, which will add a new one if it doesn't exist yet.

Dani Mesejo · Accepted Answer · 2020-04-19T22:45:56.850

2

You could do:

data = ['CONFIG-SOMEKEY-SOMEOTHERKEY = val345',
        'CONFIG-SOMEKEY-SOMEOTHEROTHERKEY = val678',
        'CONFIG-ANOTHERKEY = val222']

result = {}
for e in data:
    key, value = e.split(" = ")  # split into key and value
    path = key.split("-")  # split the key into parts
    ref = result
    for part in path[1:-1]:
        ref[part] = part in ref and ref[part] or {}
        ref = ref[part]
    ref[path[-1]] = value  # take the last part of key and set the value

print(result)

Output

{'SOMEKEY': {'SOMEOTHERKEY': 'val345', 'SOMEOTHEROTHERKEY': 'val678'}, 'ANOTHERKEY': 'val222'}

This part:

ref = result
for part in path[1:-1]:
    ref[part] = part in ref and ref[part] or {}
    ref = ref[part]
ref[path[-1]] = value

will create the nested dictionaries, is equivalent to:

for part in path[1:-1]:
    if part not in ref:
        ref[part] = {}
    ref = ref[part]

So if the part is in the dictionary you set ref as the value corresponding to part otherwise you create a new dictionary.

edited Apr 19 '20 at 22:45

answered Apr 19 '20 at 22:17

Dani Mesejo

61,499
6
49
76

`CONFIG-SOMEKEY-SOMEOTHERKEY = val345` isnt a string though the value of the CONFIG-SOMEKEY-SOMEOTHERKEY env var is val345 – red888 Apr 19 '20 at 22:21
Sorry, I do not understand your point? How is this different from the output required in your question? – Dani Mesejo Apr 19 '20 at 22:23
You could use `setdefault()` here? – NomadMonad Apr 19 '20 at 22:35
@NomadMonad I could but each call to setdefault will create a dictionary object, this way the expression is shortcircuited and does not create unnecesary objects. – Dani Mesejo Apr 19 '20 at 22:36
Could you step through the snippet that creates the nested dict in your answer a little more? Im still trying to fully understand how it works – red888 Apr 19 '20 at 22:40
@red888 Updated the answer – Dani Mesejo Apr 19 '20 at 22:46
@DaniMesejo dont mean to sound pedantic but `for part in path[1:-1]: ref = ref.setdefault(part, dict())` would save you one line. Good answer though, you got my upvote. – NomadMonad Apr 19 '20 at 22:47

Patrick Artner · Answer 3 · 2020-04-19T22:41:56.337

You can get your environment variables like so:

import os

text = [f"{k} = {v}" for k,v in os.environ.items() if k.startswith("CONFIG-")]
print(env)

(inspired by How to access environment variable values? - especially this answer)

Then you can use dicts to iterativly splitting your values:

text = """CONFIG-SOMEKEY-SOMEOTHERKEY = val345
CONFIG-SOMEKEY-SOMEOTHEROTHERKEY = val678
CONFIG-ANOTHERKEY = val222"""

text = text.split("\n")

d = {}

curr_d = d
for part in text:
    while "-" in part:
        a, b = part.split("-",1)
        if '-' in b:
            curr_d [a] = curr_d.get(a,{})
            curr_d = curr_d[a]
        part = b
    a, b = part.split("=",1)
    curr_d[a] = b

    curr_d = d

print(d)

Output:

{'CONFIG': {'SOMEOTHERKEY ': ' val345', 
            'SOMEOTHEROTHERKEY ': ' val678'}, 
 'ANOTHERKEY ': ' val222'}

how can I create nested dictionary keys and assign them values from a list of namespaced key value pairs?

3 Answers3