Python: Safe dictionary access with lists?

Question

Is there an exception free way to access values from a dictionary containing lists. For example, if I have:

data = {
    "object_1": {
        "object_2": {
            "list": [
                {
                    "property": "hello"
                }
            ]
        }
    }
}

How do I access the path data['object_1']['object_2']['list'][0]['property'] safely(i.e. return some default value if not possible to access without throwing error)? I am trying to avoid wrapping these in try-except's. I have seen the reduce based approach but it doesn't take into account having lists inside the dictionary.

In JS, I can write something like:

data.object_1?.object_2?.list[0]?.property ?? 'nothing_found'

Is there something similar in Python?

Answer 1

For dict you can use the get method. For lists you can just be careful with the index:

data.get('object_1', {}).get('object_2', {}).get('list', [{}])[0].get('property', default)

This is a bit awkward because it makes a new temporary dict or lost for each call get. It's also not super safe for lists, which don't have an equivalent method.

You can wrap the getter in a small routine to support lists too, but it's not really worth it. You're better off writing a one-off utility function that uses either exception handling or preliminary checking to handle the cases you want to react to:

def get(obj, *keys, default=None):
    for key in keys:
        try:
            obj = obj[key]
        except KeyError, IndexError:
            return default
    return obj

Exception handing has a couple of huge advantages over doing it the other way. For one thing, you don't have to do separate checks on the key depending on whether the object is a dict or list. For another, you can support almost any other reasonable type that supports __getitem__ indexing. To show what I mean, here is the asking for permission rather than forgiveness approach:

from collections.abc import Mapping, Sequence
from operator import index

def get(obj, *keys, default=None):
    for key in keys:
        if isinstance(obj, Mapping):
            if key not in obj:
                return default
        elif isinstance(obj, Sequence):
            try:
                idx = index(key)
            except TypeError:
                return default
            if len(obj) <= idx or len(obj) < -idx:
                 return default
        obj = obj[key]
    return obj

Observe how awkward and error-prone the checking is. Try passing in a custom object instead of a list, or a key that's not an integer. In Python, carefully used exceptions are your friend, and there's a reason it's pythonic to ask for forgiveness rather than for permission.

Answer 2

Uggh. Yeah, accessing such JSON data structures is just terrible, it's a bit awkward.

Glom to the rescue!

There's two ways to win:

1. You can just specify ... , default=None) to avoid exceptions, ..or..

2. Use Coalesce.

    print(glom(data, {'object_1.object_2.list': ['property']}, default=None))
   

Answer 3

In the below code, x will return None if 'object_1'/'object_2'/'list' key does not exist.

Also, if we are able to access 'list' key then we have x as Not None and we should ensure that the length of the list should be greater than zero and then we can search for 'property' key.

x = data.get('object_1', {}).get('object_2', {}).get('list')
    
if x is not None and len(x) > 0:
        print(x[0].get('property'))
else:
        print(None)

Answer 4

There is one way to do that, but it would involve the get method and would involve a lot of checking, or using temporary values.

One example lookup function would look like that:

def lookup(data):
    object_1 = data.get("object_1")
    if object_1 is None:
        # return your default
    object_2 = object_1.get('object_2')
    # and so on...

In Python 3.10 and above, there is also structural pattern matching that can help, in which case you would do something like this:

match data:
    case {'object_1': {'object_2': {'list': [{'property': x}]}}}:
        print(x) # should print 'hello'
    case _:
        print(<your_default>)

Please remember that this only works with the latest versions of Python (the online Python console on Python.org is still only on Python3.9, and the code above would cause a syntax error).