Initial Checks

• I have searched Google & GitHub for similar requests and couldn't find anything
• I have read and followed the docs and still think this feature is missing

Description

I am using pydantic directly to write json to kafka, or via FastAPI to expose API responses. In both scenario, a good practice we where doing so far in my company with pydantic v1, was, when a property was optional, to omit it totally if it was not "present" (saving network/storage spaces)

With pydantic v2, it is not possible anymore easily. From the migration guide table here it is missing a "Not required, cannot be None, no default"

Also, the generated json schema are declaring they might return "null", when we actually don't want to do so

With the current set of feature, we have various options :

• set exclude_none to True every time we dump an object (it could be quite a lot)
• define a BaseModel overriding few methods (below what we currently have, might contain bugs)
• having default, but then excluding them with exclude_defaults (but it has a lot of downsides)

In my opinion, for a non required field, the approach of:

• omitting the property in json dump if None
• having a simpler json schema

should be the default, or at least, Pydantic should expose a config to have this behavior

It is some kind of common practice to do such schema https://json-schema.org/learn/json-schema-examples instead of mutating property with anyOf and a nullable type everywhere

Our BaseModel override

from typing import Any

from pydantic import BaseModel as OriginBaseModel, GetJsonSchemaHandler
from pydantic.json_schema import JsonSchemaValue
from pydantic_core import CoreSchema

_DEFAULT_DUMP_ARGS = {"exclude_none": True}


class BaseModel(OriginBaseModel):

    @classmethod
    def __get_pydantic_json_schema__(cls, __core_schema: CoreSchema,
                                     __handler: GetJsonSchemaHandler) -> JsonSchemaValue:
        schema = __handler(__core_schema)
        schema["properties"] = {k: cls._remove_nullable(v) for k, v in schema.get("properties", {}).items()}
        return schema

    @classmethod
    def _remove_nullable(cls, property_def):
        if "default" in property_def and property_def["default"] is None:
            del (property_def["default"])
            not_null_def = property_def["anyOf"][0]
            del (property_def["anyOf"])
            return {
                **property_def,
                **not_null_def
            }

        return property_def

    def model_dump(self, **kwargs) -> dict[str, Any]:
        return super().model_dump(**{**_DEFAULT_DUMP_ARGS, **kwargs})

    def model_dump_json(self, **kwargs) -> str:
        return super().model_dump_json(**{**_DEFAULT_DUMP_ARGS, **kwargs})

    def dict(self, **kwargs) -> dict[str, Any]:
        return super().dict(**{**_DEFAULT_DUMP_ARGS, **kwargs})

Affected Components

• Compatibility between releases
• Data validation/parsing
• Data serialization - .model_dump() and .model_dump_json()
• JSON Schema
• Dataclasses
• Model Config
• Field Types - adding or changing a particular data type
• Function validation decorator
• Generic Models
• Other Model behaviour - model_construct(), pickling, private attributes, ORM mode
• Plugins and integration with other tools - mypy, FastAPI, python-devtools, Hypothesis, VS Code, PyCharm, etc.