Pydantic how to create a model with required fields and dynamic fields?

Question

I have a dynamic json like:

{
  "ts": 1111111,  // this field is required
  ...             // others are dynamic
}

The fields are all dynamic except ts. E.g:

{"ts":111,"f1":"aa","f2":"bb"}

{"ts":222,"f3":"cc","f4":"dd"}

How to declare this model with Pydantic?

class JsonData(BaseModel):
    ts: int
    ...  # and then?

Equivalent to typescript:

interface JsonData {
  ts: number;
  [key: string]: any;
}

Thanks.

Does this answer your question? [Make every fields as optional with Pydantic](https://stackoverflow.com/questions/67699451/make-every-fields-as-optional-with-pydantic) — rzlvmp, Jan 28 '22 at 04:18
No. The fields are all dynamic except `ts`. E.g: `{"ts":111,"f1":"aa","f2":"bb"}`, `{"ts":"f3":"cc","f4":"dd"}` — Feng Yu, Jan 28 '22 at 04:50
Are you sure that you need `Pydantic`? Field names are dynamic, field value types also not determined. Here is no profit of `Pydantic` usage. Just use simple dictionary with simple `if 'ts' in my_dict and type(my_dict['ts']) == int` check — rzlvmp, Jan 28 '22 at 07:51
Yes. I use FastAPI with this model to generate OpenAPI schema. — Feng Yu, Jan 28 '22 at 08:09

Paul P · Accepted Answer · 2022-01-28T16:20:22.630

Without Type Validation

If type validation doesn't matter then you could use the extra option allow:

'allow' will assign the attributes to the model.

class JsonData(BaseModel):
    ts: int

    class Config:
        extra = "allow"

This will give:

data = JsonData.parse_raw("""
{
   "ts": 222,
   "f3": "cc",
   "f4": "dd"
}
""")
repr(data)
# "JsonData(ts=222, f4='dd', f3='cc')"

And the fields can be accessed via:

print(data.f3)
# cc

With Type Validation

However, if you could change the request body to contain an object holding the "dynamic fields", like the following:

{
  "ts": 111,
  "fields": {
    "f1": "aa",
    "f2": "bb"
  }
}

or

{
  "ts": 222,
  "fields": {
    "f3": "cc",
    "f4": "dd"
  }
}

you could use a Pydantic model like this one:

from pydantic import BaseModel

class JsonData(BaseModel):
    ts: int
    fields: dict[str, str] = {}

That way, any number of fields could be processed, while the field type would be validated, e.g.:

# No "extra" fields; yields an empty dict:
data = JsonData.parse_raw("""
{
  "ts": "222"
}
""")
repr(data)
# "JsonData(ts=222, fields={})"

# One extra field:
data = JsonData.parse_raw("""
{
  "ts": 111,
  "fields": {
    "f1": "aa"
  }
}
""")
repr(data)
# "JsonData(ts=111, fields={'f1': 'aa'})"

# Several extra fields:
data = JsonData.parse_raw("""
{
  "ts": 222,
  "fields": {
    "f2": "bb",
    "f3": "cc",
    "f4": "dd"
  }
}
""")
repr(data)
# "JsonData(ts=222, fields={'f2': 'bb', 'f3': 'cc', 'f4': 'dd'})"

The fields can be accessed like this:

print(data.fields["f2"])
# bb

In this context, you might also want to consider the field types to be StrictStr as opposed to str. When using str, other types will get coerced into strings, for example when an integer or float is passed in. This doesn't happen with StrictStr.

Pydantic how to create a model with required fields and dynamic fields?

1 Answers1

Without Type Validation

With Type Validation

Linked