2

In python 3, how can I deserialize an object structure from json?

Example json:

{ 'name': 'foo',
  'some_object': { 'field1': 'bar', 'field2' : '0' },
  'some_list_of_objects': [
      { 'field1': 'bar1', 'field2' : '1' },
      { 'field1': 'bar2', 'field2' : '2' }, 
      { 'field1': 'bar3', 'field2' : '3' },
  ]
}

Here's my python code:

import json

class A:
  name: str
  some_object: B
  some_list_of_objects: list(C)

  def __init__(self, file_name):
    with open(file_name, "r") as json_file:
        self.__dict__ = json.load(json_file)

class B:
  field1: int
  field2: str

class C:
  field1: int
  field2: str

How to force some_object to be of type B and some_list_of_objects to be of type list of C?

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
Z .
  • 12,657
  • 1
  • 31
  • 56
  • 2
    Are these classes [dataclasses](https://docs.python.org/3/library/dataclasses.html#module-dataclasses)? – martineau Nov 02 '21 at 17:07
  • 1
    yes, they could be dataclasses – Z . Nov 02 '21 at 17:25
  • 1
    Not sure if this quite meets the criteria of a duplicate, but some interesting answers can be found [here](https://stackoverflow.com/questions/53376099/python-dataclass-from-a-nested-dict). – Axe319 Nov 02 '21 at 17:39
  • Yes agreed, that is also why I retracted my close vote. – rv.kvetch Nov 02 '21 at 17:50
  • @Z.D. check out my updated answer below for one possible approach. There are also other equally valid solutions in the post as included above. – rv.kvetch Nov 02 '21 at 18:42
  • @OneCricketeer Curious, why was the [tag:python3] tag removed? It appears like this would ideally only be a python 3 solution - assuming dataclasses are used of course, as mentioned above. – rv.kvetch Nov 02 '21 at 18:48
  • 1
    An argument could be made that at this point, `python3` is redundant. Python 3 is Python since Python 2.7 reached end-of-life almost a year ago. – chepner Nov 02 '21 at 19:00
  • 1
    @rv.kvetch There is a specific tag for `python-dataclasses`. Read the tag description for `python-3.x` to see that it generally should not be used except for specific cases (specifically, not to be added to indicate what version the code is running as) – OneCricketeer Nov 02 '21 at 19:14

1 Answers1

2

As you're using Python 3, I would suggest using dataclasses to model your classes. This should improve your overall code quality and also eliminate the need to explicltly declare an __init__ constructor method for your class, for example.

If you're on board with using a third-party library, I'd suggest looking into an efficient JSON serialization library like the dataclass-wizard that performs implicit type conversion - for example, string to annotated int as below. Note that I'm using StringIO here, which is a file-like object containing a JSON string to de-serialize into a nested class model.

Note: the following approach should work in Python 3.7+.

from __future__ import annotations

from dataclasses import dataclass
from io import StringIO

from dataclass_wizard import JSONWizard


json_data = StringIO("""
{ "name": "foo",
  "some_object": { "field1": "bar", "field2" : "0" },
  "some_list_of_objects": [
      { "field1": "bar1", "field2" : "1" },
      { "field1": "bar2", "field2" : "2" },
      { "field1": "bar3", "field2" : "3" }
  ]
}
""")


@dataclass
class A(JSONWizard):
    name: str
    some_object: B
    some_list_of_objects: list[C]


@dataclass
class B:
    field1: str
    field2: int


@dataclass
class C:
    field1: str
    field2: int


a = A.from_json(json_data.read())

print(f'{a!r}')  # alternatively: print(repr(a))

Output

A(name='foo', some_object=B(field1='bar', field2=0), some_list_of_objects=[C(field1='bar1', field2=1), C(field1='bar2', field2=2), C(field1='bar3', field2=3)])

Loading from a JSON file

As per the suggestions in this post, I would discourage overriding the constructor method to pass the name of a JSON file to load the data from. Instead, I would suggest creating a helper class method as below, that can be invoked like A.from_json_file('file.json') if desired.

    @classmethod
    def from_json_file(cls, file_name: str):
        """Deserialize json file contents into an A object."""
        with open(file_name, 'r') as json_file:
            return cls.from_dict(json.load(json_file))

Suggestions

Note that variable annotations (or annotations in general) are subscripted using square brackets [] rather than parentheses as appears in the original version above.

some_list_of_objects: list(C)

In the above solution, I've instead changed that to:

some_list_of_objects: list[C]

This works because using subscripted values in standard collections was introduced in PEP 585. However, using the from __future__ import annotations statement introduced to Python 3.7+ effectively converts all annotations to forward-declared string values, so that new-style annotations that normally only would work in Python 3.10, can also be ported over to Python 3.7+ as well.

One other change I made, was in regards to swapping out the order of declared class annotations. For example, note the below:

class B:
  field1: int
  field2: str

However, note the corresponding field in the JSON data, that would be deserialized to a B object:

'some_object': { 'field1': 'bar', 'field2' : '0' },

In the above implementation, I've swapped out the field annotations in such cases, so class B for instance is declared as:

class B:
    field1: str
    field2: int
rv.kvetch
  • 9,940
  • 3
  • 24
  • 53