How to create a dataclass that handles nested dicts with different names?

Question

I am using the data from the League of Legends API to learn Python, JSON, and Data Classes. Using dacite, I have created parent and child classes that allow access to the data using this syntax: champs.data['Ahri']['key']. However, I wonder if there is a way to create a class that returns the keys as fields so one could access the data using this syntax: champs.data.Ahri.key.

Here is the working code:

from dataclasses import dataclass
from dacite import from_dict

j1 = {'type': 'champion',
      'data': {'Aatrox': {'id': 'Aatrox', 'key': '266', 'name': 'Aatrox'},
      'Ahri': {'id': 'Ahri', 'key': '103', 'name': 'Ahri'}}}

@dataclass
class C:
    type: str
    data: dict

@dataclass
class P:
    type: str
    data: dict

champs = from_dict(data_class=P, data=j1)
champs.data['Ahri']['key']

Answer 1

If it were me, I would probably leave/make champions a dictionary. Then access it like champions['Ahri'].key

Something like:

import dataclasses

@dataclasses.dataclass
class Champion:
    id: str
    key: str
    name: str

j1 = {
    'type': 'champion',
    'data': {
        'Aatrox': {'id': 'Aatrox', 'key': '266', 'name': 'Aatrox'},
        'Ahri': {'id': 'Ahri', 'key': '103', 'name': 'Ahri'}
    }
}

champions = {
    champion["id"]: Champion(**champion)
    for champion in j1["data"].values()
}

print(champions['Ahri'].key)

resulting in 103

However if you were really keen on champions.Ahri.key then you can implement Champions as an empty class and use setattr()

import dataclasses

@dataclasses.dataclass
class Champion:
    id: str
    key: str
    name: str

@dataclasses.dataclass
class Champions:
    pass

j1 = {
    'type': 'champion',
    'data': {
        'Aatrox': {'id': 'Aatrox', 'key': '266', 'name': 'Aatrox'},
        'Ahri': {'id': 'Ahri', 'key': '103', 'name': 'Ahri'}
    }
}

champions = Champions()
for champion in j1["data"].values():
    setattr(champions, champion["id"], Champion(**champion))

print(champions.Ahri.key)

again giving you 103

Note: The @dataclass decorator can likely be omitted from Champion().

Answer 2

The closest you can probably get - at least in a safe enough manner - is as @JonSG suggests, using champs.data['Ahri'].key.

Here's a straightforward example using the dataclass-wizard. It doesn't do a strict type checking as I know dacite does.

Instead, it opts to do implicit type coercision where possible, which is useful in some cases; you can see an example of this below - str to annotated int in this case.

Note: This example should work for Python 3.7+ with the included __future__ import.

from __future__ import annotations

from dataclasses import dataclass
from dataclass_wizard import fromdict


data = {
    'type': 'champion',
    'data': {
          'Aatrox': {'id': 'Aatrox', 'key': '266', 'name': 'Aatrox'},
          'Ahri': {'id': 'Ahri', 'key': '103', 'name': 'Ahri'},
    }
}


@dataclass
class P:
    type: str
    data: dict[str, Character]


@dataclass
class Character:
    id: str
    key: int
    name: str


champs = fromdict(P, data)

print(champs)
print(champs.data['Ahri'].key)

Output:

P(type='champion', data={'Aatrox': Character(id='Aatrox', key=266, name='Aatrox'), 'Ahri': Character(id='Ahri', key=103, name='Ahri')})
103

Answer 3

How to do this

d = {
    "type": "champion",
    "data": {
        "Aatrox": {"id": "Aatrox", "key": "266", "name": "Aatrox"},
        "Ahri": {"id": "Ahri", "key": "103", "name": "Ahri"},
    },
}


def dict_to_class(d) -> object:
    if isinstance(d, dict):

        class C:
            pass

        for k, v in d.items():
            setattr(C, k, dict_to_class(v))
        return C
    else:
        return d


champ = dict_to_class(d)

print(champ.data.Ahri.key)
# 103

The key here is the setatter builtin method, which takes an object, a string, and some value, and creates an attribute (field) on that object, named according to the string and containing the value.

Don't do this!

I must stress that there is almost never a good reason to do this. When dealing with JSON data of an unknown shape, the correct way to represent it is a dict.

If you do know the shape of the data, you should create a specialized dataclass, like so:

from dataclasses import dataclass

d = {
    "type": "champion",
    "data": {
        "Aatrox": {"id": "Aatrox", "key": "266", "name": "Aatrox"},
        "Ahri": {"id": "Ahri", "key": "103", "name": "Ahri"},
    },
}

@dataclass
class Champion:
    id: str
    key: str
    name: str

champions = {name: Champion(**attributes) for name, attributes in d["data"].items()}

print(champions)
# {'Aatrox': Champion(id='Aatrox', key='266', name='Aatrox'), 'Ahri': Champion(id='Ahri', key='103', name='Ahri')}

print(champions["Aatrox"].key)
# 266

Answer 4

The dacite docs have a section about nested structures that is very close to what you want. The example they use, verbatim, is as follows:

@dataclass
class A:
    x: str
    y: int


@dataclass
class B:
    a: A


data = {
    'a': {
        'x': 'test',
        'y': 1,
    }
}

result = from_dict(data_class=B, data=data)

assert result == B(a=A(x='test', y=1))

We can access fields at arbitrary depth as e.g. result.a.x == 'test'.

The critical difference between this and your data is that the dictionary under the data key has keys with arbitrary values (Aatrox, Ahri, etc.). dacite isn't set up to create new field names on the fly, so the best you're going to get is something like the latter part of @JonSG's answer, which uses setattr to dynamically build new fields.

Let's imagine how you would use this data for a moment, though. Probably you'd want a some point to be able to iterate over your champions in order to perform a filter/transform/etc. operation. It's possible to iterate over fields in python, but you have to really dig into python internals, which means your code will be less readable/generally comprehensible.

Much better would be one of the following:

1. Preprocess j1 into a shape that fits the structure you want to use, and then use dacite with a dataclass that fits the new structure. For example, maybe it makes sense to pull the values of the data dict out into a list.
2. Process in steps using dacite. For example, something like the following:

from dataclasses import dataclass
from dacite import from_dict


@dataclass
class TopLevel:
    type: str
    data: dict


j1 = {
    "type": "champion",
    "data": {
        "Aatrox": {"id": "Aatrox", "key": "266", "name": "Aatrox"},
        "Ahri": {"id": "Ahri", "key": "103", "name": "Ahri"},
    },
}

champions = from_dict(data_class=TopLevel, data=j1)
# champions.data is a dict of dicts

@dataclass
class Champion:
    id: str
    key: str
    name: str


# transform champions.data into a dict of Champions
for k, v in champions.data.items():
    champions.data[k] = from_dict(data_class=Champion, data=v)

# now, you can do interesting things like the following filter operation
start_with_a = [
    champ for champ in champions.data.values() if champ.name.lower().startswith("a")
]
print(start_with_a)
# [Champion(id='Aatrox', key='266', name='Aatrox'), Champion(id='Ahri', key='103', name='Ahri')]