Can I override fields from a Pydantic parent model to make them optional?

Question

I have two pydantic classes like this.

class Parent(BaseModel):
    id: int
    name: str
    email: str

class ParentUpdate(BaseModel):
    id: Optional[int]
    name: Optional[str]
    email: Optional[str]

Both of these are practically the same but the Parent class makes all fields required. I want to use the Parent class for POST request body in FastAPI, hence all fields should be required. But I want to use the latter for PUT request body since the user can set selective fields and the remaining stays the same. I have taken a look at Required Optional Fields but they do not correspond to what I want to do.

If there was a way I could inherit the Parent class in ParentUpdate and modified all the fields in Parent to make them Optional that would reduce the clutter. Additionally, there are some validators present in the Parent class which I have to rewrite in the ParentUpdate class which I also want to avoid.

Is there any way of doing this?

Answer 1

You can make optional fields required in subclasses, but you cannot make required fields optional in subclasses. In fastapi author tiangolo's boilerplate projects, he utilizes a pattern like this for your example:

class ParentBase(BaseModel):
    """Shared properties."""
    name: str
    email: str

class ParentCreate(ParentBase):
    """Properties to receive on item creation."""
    # dont need id here if your db autocreates it
    pass

class ParentUpdate(ParentBase):
    """Properties to receive on item update."""
    # dont need id as you are likely PUTing to /parents/{id}
    # other fields should not be optional in a PUT
    # maybe what you are wanting is a PATCH schema?
    pass

class ParentInDBBase(ParentBase):
    """Properties shared by models stored in DB - !exposed in create/update."""
    # primary key exists in db, but not in base/create/update
    id: int                             

class Parent(ParentInDBBase):
    """Properties to return to client."""
    # optionally include things like relationships returned to consumer
    # related_things: List[Thing]
    pass

class ParentInDB(ParentInDBBase):
    """Additional properties stored in DB."""
    # could be secure things like passwords?
    pass

Yes, I agree this is incredibly verbose and I wish it wasn't. You still likely end up with other schemas more specific to particular forms in your UI. Obviously, you can remove some of these as they aren't necessary in this example, but depending on other fields in your DB, they may be needed, or you may need to set defaults, validation, etc.

In my experience for validators, you have to re-declare them but you can use a shared function, ie:

def clean_article_url(cls, v):
    return clean_context_url(v.strip())

class MyModel(BaseModel):
    article_url: str

    _clean_url = pydantic.validator("article_url", allow_reuse=True)(clean_article_url)

Answer 2

Overriding fields is possible and easy. (Somebody mentioned it is not possible to override required fields to optional, but I do not agree).

This example works without any problems:

class Parent(BaseModel):
    id: int
    name: str
    email: str

class ParentUpdate(Parent): ## Note that this inherits 'Parent' class (not BaseModel)
    id: Optional[int]  # this will convert id from required to optional

Answer 3

As already outlined in an answer to a similar question, I am using the following approach (credit goes to Aron Podrigal):

import inspect   
from pydantic import BaseModel   


def optional(*fields):
    """Decorator function used to modify a pydantic model's fields to all be optional.
    Alternatively, you can  also pass the field names that should be made optional as arguments
    to the decorator.
    Taken from https://github.com/samuelcolvin/pydantic/issues/1223#issuecomment-775363074
    """   
    def dec(_cls):
        for field in fields:
            _cls.__fields__[field].required = False
        return _cls

    if fields and inspect.isclass(fields[0]) and issubclass(fields[0], BaseModel):
        cls = fields[0]
        fields = cls.__fields__
        return dec(cls)

    return dec

   

In your example you'd use it like this:

@optional
class ParentUpdate(Parent):
    pass

Answer 4

I apologize in advance, and I'm definitely sure that is a horrible workaround, but it worked for me:

def make_child_fields_optional(parent_class: Type[BaseModel], child_class: Type[BaseModel]):
    for key in parent_class.__fields__:
        child_class.__fields__.get(key).required = False

class BasePerson(BaseModel):
    name: str
    email: str
    login: str

class UpdatePerson(BasePerson):
    pass  # or whatever

make_child_fields_optional(BasePerson, UpdatePerson)

Answer 5

My advice is to not invent difficult schemas, I was also interested in pydantic capabilities, but all of them look very ugly and hard to understand (or even not intended for some tasks and have constraints). See Python pydantic, make every field of ancestor are Optional Answer from pydantic maintainer

Answer 6

For my case creating a new class was the only solution that worked, but packed into a function it is quite convenient:

from pydantic import BaseModel, create_model
from typing import Optional

def make_optional(baseclass):
    # Extracts the fields and validators from the baseclass and make fields optional
    fields = baseclass.__fields__
    validators = {'__validators__': baseclass.__validators__}
    optional_fields = {key: (Optional[item.type_], None) for key, item in fields.items()}
    return create_model(f'{baseclass.__name__}Optional', **optional_fields, __validators__=validators)

class Parent(BaseModel):
    id: int
    name: str
    email: str

ParentUpdate = make_optional(Parent)

Comparing after and before:

Parent.__fields__

{'id': ModelField(name='id', type=int, required=True),
 'name': ModelField(name='name', type=str, required=True),
 'email': ModelField(name='email', type=str, required=True)}

ParentUpdate.__fields__

{'id': ModelField(name='id', type=Optional[int], required=False, default=None),
 'name': ModelField(name='name', type=Optional[str], required=False, default=None),
 'email': ModelField(name='email', type=Optional[str], required=False, default=None)}

It does work, and also it allows you to filter out some fields of the class if it is required.

Moreover for FastApi you can directly use make_optional(Parent) as the type-hint in the API call and that will generate the documentation correctly. Another advantage of this approach is that it can reduce the boilerplate a lot.

Answer 7

If you want to override only some given fields to be optional without the repetition of the type hints, you can do that using a decorator like this:

from typing import Optional

from pydantic import BaseModel


def set_fields_optional(*field_names):
    def decorator(cls: BaseModel):
        for field_name in field_names:
            field = cls.__fields__[field_name]
            field.required = False
            field.annotation = Optional[field.annotation]
            field.allow_none = True

        return cls

    return decorator

According to the documentation, the __fields__ is a model property and it can be used to access the fields of the model by name and modify their attributes.

Apply the decorator with the names of the fields that you want to make optional:

class BaseWithOnlyRequiredFields(BaseModel):
    x: int
    y: str
    z: float


@set_fields_optional('x', 'z')
class DerivedWithSomeOptionalFields(BaseWithOnlyRequiredFields):
    pass


DerivedWithSomeOptionalFields(y='y-value', x=None)
# DerivedWithSomeOptionalFields(x=None, y='y-value', z=None)