Chained, nested dict() get calls in python

Question

I'm interrogating a nested dictionary using the dict.get('keyword') method. Currently my syntax is...

M = cursor_object_results_of_db_query

for m in M:
    X = m.get("gparents").get("parent").get("child")
    for x in X:
        y = x.get("key")

However, sometimes one of the "parent" or "child" tags doesn't exist, and my script fails. I know using get() I can include a default in the case the key doesn't exist of the form...

get("parent", '') or
get("parent", 'orphan')

But if I include any Null, '', or empty I can think of, the chained .get("child") fails when called on ''.get("child") since "" has no method .get().

The way I'm solving this now is by using a bunch of sequential try-except around each .get("") call, but that seems foolish and unpython---is there a way to default return "skip" or "pass" or something that would still support chaining and fail intelligently, rather than deep-dive into keys that don't exist?

Ideally, I'd like this to be a list comprehension of the form:

[m.get("gparents").get("parent").get("child") for m in M]

but this is currently impossible when an absent parent causes the .get("child") call to terminate my program.

I don't see anything `requests` specific here. Are you talking about the JSON responses? Those are plain python dictionaries, and their use is separate from the `requests` library. — Martijn Pieters, Jan 23 '13 at 16:27
I hadn't realized that was plain python dict(). Apologies, updating. — Mittenchops, Jan 23 '13 at 16:30
Similar: http://stackoverflow.com/questions/9378380/python-dictionary-modify-getitem — Yuval Adam, Jan 23 '13 at 16:35
**See also (python dpath):** https://stackexchange.com/search?q=dpath%20python and also https://stackoverflow.com/a/16508328/42223 — dreftymac, Jul 02 '18 at 20:42
Are you asking about a type of null-coalescing operator for Python (C# comes to mind: https://kodify.net/csharp/operators/null-coalescing-operator/) — information_interchange, Dec 04 '20 at 06:26

score 103 · Accepted Answer · answered Jan 23 '13 at 16:32

Since these are all python dicts and you are calling the dict.get() method on them, you can use an empty dict to chain:

[m.get("gparents", {}).get("parent", {}).get("child") for m in M]

By leaving off the default for the last .get() you fall back to None. Now, if any of the intermediary keys is not found, the rest of the chain will use empty dictionaries to look things up, terminating in .get('child') returning None.

mgilson · Answer 2 · 2013-01-23T16:52:06.057

Another approach is to recognize that if the key isn't found, dict.get returns None. However, None doesn't have an attribute .get, so it will throw an AttributeError:

for m in M:
    try:
       X = m.get("gparents").get("parent").get("child")
    except AttributeError:
       continue

    for x in X:
        y = x.get("key")
        #do something with `y` probably???

Just like Martijn's answer, this doesn't guarantee that X is iterable (non-None). Although, you could fix that by making the last get in the chain default to returning an empty list:

 try:
    X = m.get("gparents").get("parent").get("child",[])
 except AttributeError:
    continue

Finally, I think that probably the best solution to this problem is to use reduce:

try:
    X = reduce(dict.__getitem__,["gparents","parent","child"],m)
except (KeyError,TypeError):
    pass
else:
    for x in X:
       #do something with x

The advantage here is that you know if any of the gets failed based on the type of exception that was raised. It's possible that a get returns the wrong type, then you get a TypeError. If the dictionary doesn't have the key however, it raises a KeyError. You can handle those separately or together. Whatever works best for your use case.

Oooh, I see what you did there. You got to use your `reduce()` sans `lambda` there! :-P — Martijn Pieters, Jan 23 '13 at 17:06
@MartijnPieters -- I've just been itching to use it ever since I cooked it up the other day ;-). — mgilson, Jan 23 '13 at 17:49

score 6 · Answer 3 · answered Jan 23 '13 at 16:41

6

How about using a small helper function?

def getn(d, path):
    for p in path:
        if p not in d:
            return None
        d = d[p]
    return d

and then

[getn(m, ["gparents", "parent", "child"]) for m in M]

answered Jan 23 '13 at 16:41

georg

211,518
52
313
390

score 5 · Answer 4 · edited Oct 17 '17 at 12:25

5

I realise I'm a bit late for the part but here's the solution I came up with when faced with a similar problem:

def get_nested(dict_, *keys, default=None):
    if not isinstance(dict_, dict):
        return default
    elem = dict_.get(keys[0], default)
    if len(keys) == 1:
        return elem
    return get_nested(elem, *keys[1:], default=default)

For example:

In [29]: a = {'b': {'c': 1}}
In [30]: get_nested(a, 'b', 'c')
Out[30]: 1
In [31]: get_nested(a, 'b', 'd') is None
Out[31]: True

edited Oct 17 '17 at 12:25

Никита Конин

135
2
9

answered Jan 23 '16 at 00:13

Kevin Dungs

1,555
1
13
18

BTW, this `def get_nested(dict_, *keys, default=None)` only works in python 3 (the keyword `default` arg after the `*keys` – Ran Jan 26 '16 at 09:06

Chained, nested dict() get calls in python

4 Answers4

Linked

Related