Pydantic: Detect if a field value is missing or given as null

Question

I want to allow users to selectively update fields using PUT calls. On the pydantic model, I have made the fields Optional. In the FastAPI handler if the model attribute is None, then the field was not given and I do not update it.

The problem with this approach is that there is no way for the client to "blank out" a field that isn't required for certain types.

In particular, I have date fields that I want clients to be able to clear by sending in a null in the JSON. How can I detect the difference between the client sending null or the client not sending the field/value at all? The model attribute is just None in either case.

Answer 1

Pydantic V1

The pydantic documentation desccribes two options that can be used with the .dict() method of models.

• exclude_unset: whether fields which were not explicitly set when creating the model should be excluded from the returned dictionary; default False. Prior to v1.0, exclude_unset was known as skip_defaults; use of skip_defaults is now deprecated

• exclude_defaults: whether fields which are equal to their default values (whether set or otherwise) should be excluded from the returned dictionary; default False

So you can create a model class with optional fields:

from typing import Optional
from pydantic import BaseModel


class MyModel(BaseModel):
    foo: Optional[int] = None
    bar: Optional[int] = None

And still generate a dict with fields explicitely set to None, but without default values:

baz = MyModel(foo=None)
assert baz.dict(exclude_unset=True) == {"foo": None}

baz = MyModel(bar=None)
assert baz.dict(exclude_unset=True) == {"bar": None}

Pydantic V2

Pydantic V2 is available since June 30, 2023

The .dict() method has been removed in V2. In order to get a dictionary out of a BaseModel instance, one must use the model_dump() method instead:

from __future__ import annotations
from pydantic import BaseModel


class MyModel(BaseModel):
    foo: int | None = None
    bar: int | None = None

baz = MyModel(foo=None)
assert baz.model_dump(exclude_unset=True) == {"foo": None}

baz = MyModel(bar=None)
assert baz.model_dump(exclude_unset=True) == {"bar": None}

Answer 2

You can check obj.__fields_set__ to see whether the value was missing or not.

from typing import Optional
from pydantic import BaseModel

class Foo(BaseModel):
    first: Optional[int] = None
    second: Optional[int] = None

foo = Foo.parse_raw('{"first": null}')

assert foo.first is None and foo.second is None
assert foo.__fields_set__ == {"first"}

Answer 3

Another way to differentiate between a provided None and no value is to use a @root_validator(pre=True); e.g., to allow nullable non-optional fields.

To be honest I'm not sure if it's suited for your use case, but I was pointed here by various google searches as it answers your question title, so I'll post my answer anyway.

The advantage is that it's checked "inside the model" and can be validated, e.g., via MyModel.validate(my_dict) without external extra assertions.

from typing import Optional, root_validator
from pydantic import BaseModel


class MyModel(BaseModel):
    foo: Optional[int]
    bar: Optional[int]

    @root_validator(pre=True)
    def _check_whether_foo_present(cls, values):
        if not "foo" in values:
            # do something, e.g.:
            raise ValueError('"foo" was not provided to the model')
        return values