Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.rst
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ Added
- Signature methods now when given ``sub_configs=True``, list of paths types can
now receive a file containing a list of paths (`#816
<https://github.com/omni-us/jsonargparse/pull/816>`__).
- Public interface for enabling/disabling support of type subclasses (`#817
<https://github.com/omni-us/jsonargparse/pull/817>`__).

Fixed
^^^^^
Expand Down
188 changes: 128 additions & 60 deletions DOCUMENTATION.rst
Original file line number Diff line number Diff line change
Expand Up @@ -501,15 +501,15 @@ Some notes about this support are:
:py:meth:`.ArgumentParser.instantiate_classes` can be used to instantiate all
classes in a config object. For more details see :ref:`sub-classes`.

- ``Protocol`` types are also supported the same as sub-classes. The protocols
- ``Protocol`` types are also supported the same as subclasses. The protocols
are not required to be ``runtime_checkable``. But the accepted classes must
match exactly the signature of the protocol's public methods.

- ``dataclasses`` are supported even when nested. Final classes, attrs'
``define`` decorator, and pydantic's ``dataclass`` decorator and ``BaseModel``
classes are supported and behave like standard dataclasses. For more details
see :ref:`dataclass-like`. If a dataclass is mixed inheriting from a normal
class, it is considered a subclass type instead of a dataclass.
- ``dataclasses`` are supported even when nested and by default don't accept
subclasses. Final classes, attrs' ``define``, pydantic's ``dataclass`` and
pydantic's ``BaseModel`` classes are supported and behave like standard
dataclasses. For more details see :ref:`subclasses-disabled`. If a dataclass
is mixed inheriting from a normal class, by default it will accept subclasses.

- User-defined ``Generic`` types are supported. For more details see
:ref:`generic-types`.
Expand Down Expand Up @@ -953,55 +953,6 @@ be achieved as follows:
Namespace(dict={'key1': 'val1', 'key2': 'val2'})


.. _dataclass-like:

Dataclass-like classes
----------------------

In contrast to subclasses, which requires the user to provide a ``class_path``,
in some cases it is not expected to have subclasses. In this case the init args
are given directly in a dictionary without specifying a ``class_path``. This is
the behavior for standard ``dataclasses``, ``final`` classes, attrs' ``define``
decorator, and pydantic's ``dataclass`` decorator and ``BaseModel`` classes.

As an example, take a class that is decorated with :func:`.final`, meaning that
it shouldn't be subclassed. The code below would accept the corresponding YAML
structure.

.. testsetup:: final_classes

cwd = os.getcwd()
tmpdir = tempfile.mkdtemp(prefix="_jsonargparse_doctest_")
os.chdir(tmpdir)
with open("config.yaml", "w") as f:
f.write("data:\n number: 8\n accepted: true\n")

.. testcleanup:: final_classes

os.chdir(cwd)
shutil.rmtree(tmpdir)

.. testcode:: final_classes

from jsonargparse.typing import final


@final
class FinalClass:
def __init__(self, number: int = 0, accepted: bool = False):
...


parser = ArgumentParser()
parser.add_argument("--data", type=FinalClass)
cfg = parser.parse_path("config.yaml")

.. code-block:: yaml

data:
number: 8
accepted: true

.. _generic-types:

Generic types
Expand Down Expand Up @@ -1132,9 +1083,9 @@ requires to give both a serializer and a deserializer as seen below.
.. note::

The registering of types is only intended for simple types. By default any
class used as a type hint is considered a sub-class (see :ref:`sub-classes`)
class used as a type hint is considered a subclass (see :ref:`sub-classes`)
which might be good for many use cases. If a class is registered with
:func:`.register_type` then the sub-class option is no longer available.
:func:`.register_type` then the subclass option is no longer available.


.. _custom-types:
Expand Down Expand Up @@ -1953,8 +1904,8 @@ In Python, dependency injection is achieved by:

.. _sub-classes:

Class type and sub-classes
--------------------------
Class type and subclasses
-------------------------

When a class is used as a type hint, jsonargparse expects in config files a
dictionary with a ``class_path`` entry indicating the dot notation expression to
Expand All @@ -1963,7 +1914,7 @@ instantiate it. When parsing, it will be checked that the class can be imported,
that it is a subclass of the given type and that ``init_args`` values correspond
to valid arguments to instantiate it. After parsing, the config object will
include the ``class_path`` and ``init_args`` entries. To get a config object
with all nested sub-classes instantiated, the
with all nested subclasses instantiated, the
:py:meth:`.ArgumentParser.instantiate_classes` method is used.

Additional to using a class as type hint in signatures, for low level
Expand Down Expand Up @@ -2314,6 +2265,123 @@ to :ref:`instance-factories`.
module from where the respective object can be imported.


.. _subclasses-disabled:

Class types with subclasses disabled
------------------------------------

In certain situations, it is preferable to use a class as a type hint with no
intention to receive subclasses. From a parser perspective, this means that
providing a subclass is not permitted, and when serializing, the instantiation
arguments are stored directly, without including ``class_path`` and
``init_args``. The standard Python approach for this scenario is to decorate
classes with :func:`.final`, which explicitly indicates that subclassing is not
intended. A parsing example would be:

.. testcode:: final_classes

from jsonargparse.typing import final


@final
class FinalClass:
def __init__(self, number: int = 0, accepted: bool = False):
...


parser = ArgumentParser()
parser.add_argument("--data", type=FinalClass)
cfg = parser.parse_args(["--data.number=8", "--data.accepted=true"])

for which a dump would give as output:

.. doctest:: final_classes

>>> print(parser.dump(cfg)) # doctest: +NORMALIZE_WHITESPACE
data:
number: 8
accepted: true

In some cases, subclasses are not intended, but the :func:`.final` decorator is
not applied. For example, having ``class_path`` for a simple ``x, y``
coordinates dataclass would be unnecessarily cumbersome. For this reason,
``jsonargparse`` early on, implemented the same behavior for pure (not mixed
with normal classes) ``dataclasses``, attrs' ``define``, pydantic's
``dataclass``, and pydantic's ``BaseModel`` classes. However, since these
classes technically support subclassing, subclass support can be enabled as
described below. Subclass support has been kept disabled for these types by
default to avoid introducing breaking changes.


.. _enable-disable-subclasses:

Enable/disable subclasses
-------------------------

The :func:`.set_parsing_settings` function provides the ``subclasses_disabled``
and ``subclasses_enabled`` parameters, which, as their names suggest, control
which class types support subclasses. The ``subclasses_disabled`` parameter
accepts a list of class types and functions. When a type is provided, that type
and its descendants will have subclass support disabled. Functions in the list
should accept a type and return ``True`` if subclasses should be disabled for
that type.

The ``subclasses_enabled`` parameter accepts a list of class types and function
names. When a type is provided, both the type and its descendants will have
subclass support enabled. Types specified in ``subclasses_enabled`` take
precedence over those in ``subclasses_disabled``. If a function name is given to
``subclasses_enabled``, it must correspond to a function previously registered
in ``subclasses_disabled``; in this case, the effect is to unregister it. By
default, the following disable functions are registered: ``is_pure_dataclass``,
``is_pydantic_model``, ``is_attrs_class``, and ``is_final_class``.

Some examples. Since ``subclasses_enabled`` takes precedence, it is possible to
keep subclass support disabled for dataclasses, but enable enable it for a
specific dataclass as follows:

.. testsetup:: enable_disable_subclasses

selectors = _common.subclasses_disabled_selectors
_common.subclasses_disabled_selectors = selectors.copy()

@dataclass
class DataClassBaseType:
pass

.. testcleanup:: enable_disable_subclasses

_common.subclasses_disabled_selectors = selectors

.. testcode:: enable_disable_subclasses

from jsonargparse import set_parsing_settings

set_parsing_settings(subclasses_enabled=[DataClassBaseType])

To enable subclass support for all pydantic models, the following can be done:

.. testcode:: enable_disable_subclasses

set_parsing_settings(subclasses_enabled=["is_pydantic_model"])

To enable subclass support for all dataclasses, but have it disabled for a
specific dataclass, the following can be done:

.. testcode:: enable_disable_subclasses

set_parsing_settings(
subclasses_enabled=["is_pure_dataclass"],
subclasses_disabled=[DataClassBaseType],
)

.. note::

Enabling subclass support for types is currently experimental. While the
interface and behavior is expected to be stable, fundamental issues may
arise that require changes to the design, which could result in breaking
changes in future releases.


.. _argument-linking:

Argument linking
Expand Down
8 changes: 4 additions & 4 deletions jsonargparse/_actions.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
from contextvars import ContextVar
from typing import Any, Optional, Union

from ._common import Action, NonParsingAction, is_not_subclass_type, is_subclass, parser_context
from ._common import Action, NonParsingAction, is_subclass, is_subclasses_disabled, parser_context
from ._loaders_dumpers import get_loader_exceptions, load_value
from ._namespace import Namespace, NSKeyError, split_key, split_key_root
from ._optionals import _get_config_read_mode, ruamel_support
Expand Down Expand Up @@ -365,13 +365,13 @@ def update_init_kwargs(self, kwargs):
self._typehint = kwargs.pop("_typehint")
self._help_types = self.get_help_types(self._typehint)
assert self._help_types and all(isinstance(b, type) for b in self._help_types)
self._not_subclass = len(self._help_types) == 1 and is_not_subclass_type(self._help_types[0])
self._single_class = len(self._help_types) == 1 and is_subclasses_disabled(self._help_types[0])
self._basename = iter_to_set_str(t.__name__ for t in self._help_types)

if len(self._help_types) == 1:
kwargs["nargs"] = 0 if self._not_subclass else "?"
kwargs["nargs"] = 0 if self._single_class else "?"

if self._not_subclass:
if self._single_class:
msg = ""
else:
kwargs["metavar"] = "CLASS_PATH_OR_NAME"
Expand Down
69 changes: 61 additions & 8 deletions jsonargparse/_common.py
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,8 @@ def set_parsing_settings(
parse_optionals_as_positionals: Optional[bool] = None,
stubs_resolver_allow_py_files: Optional[bool] = None,
omegaconf_absolute_to_relative_paths: Optional[bool] = None,
subclasses_disabled: Optional[list[Union[type, Callable[[type], bool]]]] = None,
subclasses_enabled: Optional[list[Union[type, str]]] = None,
) -> None:
"""
Modify settings that affect the parsing behavior.
Expand Down Expand Up @@ -138,6 +140,16 @@ def set_parsing_settings(
with ``omegaconf+`` parser mode, absolute interpolation paths are
converted to relative. This is only intended for backward
compatibility with ``omegaconf`` parser mode.
subclasses_disabled: List of types or functions, so that when parsing
only the exact type hints (not their subclasses) are accepted.
Descendants of the configured types are also disabled. Functions
should return ``True`` for types to disable.
subclasses_enabled: List of types or disable function names, so that
subclasses are accepted. Types given here have precedence over those
in ``subclasses_disabled``. Giving a function name removes the
corresponding function from ``subclasses_disabled``. By default, the
following disable functions are registered: ``is_pure_dataclass``,
``is_pydantic_model``, ``is_attrs_class`` and ``is_final_class``.
"""
# validate_defaults
if isinstance(validate_defaults, bool):
Expand Down Expand Up @@ -171,6 +183,12 @@ def set_parsing_settings(
raise ValueError(
f"omegaconf_absolute_to_relative_paths must be a boolean, but got {omegaconf_absolute_to_relative_paths}."
)
# subclass behavior
if subclasses_disabled or subclasses_enabled:
subclass_type_behavior(
subclasses_disabled=subclasses_disabled,
subclasses_enabled=subclasses_enabled,
)


def get_parsing_setting(name: str):
Expand Down Expand Up @@ -283,20 +301,55 @@ def is_pure_dataclass(cls) -> bool:
return all(dataclasses.is_dataclass(c) for c in classes)


not_subclass_type_selectors: dict[str, Callable[[type], Union[bool, int]]] = {
"final": is_final_class,
"dataclass": is_pure_dataclass,
"pydantic": is_pydantic_model,
"attrs": is_attrs_class,
subclasses_enabled_types: set[type] = set()
subclasses_disabled_types: set[type] = set()
subclasses_disabled_selectors: dict[str, Callable[[type], Union[bool, int]]] = {
"is_pure_dataclass": is_pure_dataclass,
"is_pydantic_model": is_pydantic_model,
"is_attrs_class": is_attrs_class,
"is_final_class": is_final_class,
}


def is_not_subclass_type(cls) -> bool:
def is_subclasses_disabled(cls) -> bool:
if is_generic_class(cls):
return is_not_subclass_type(cls.__origin__)
return is_subclasses_disabled(cls.__origin__)
if not inspect.isclass(cls):
return False
return any(validator(cls) for validator in not_subclass_type_selectors.values())
subclass_disabled = any(selector(cls) for selector in subclasses_disabled_selectors.values())
if not subclass_disabled:
subclass_disabled = any(issubclass(cls, disable_type) for disable_type in subclasses_disabled_types)
if subclass_disabled:
subclass_disabled = not any(issubclass(cls, enable_type) for enable_type in subclasses_enabled_types)
return subclass_disabled


def subclass_type_behavior(
subclasses_disabled: Optional[list[Union[type, Callable[[type], bool]]]] = None,
subclasses_enabled: Optional[list[Union[type, str]]] = None,
) -> None:
"""Configures whether class types accept or not subclasses."""
for enable_item in subclasses_enabled or []:
if isinstance(enable_item, str):
if enable_item not in subclasses_disabled_selectors:
raise ValueError(f"There is no function '{enable_item}' registered in subclasses_disabled")
subclasses_disabled_selectors.pop(enable_item)
elif inspect.isclass(enable_item):
subclasses_enabled_types.add(enable_item)
else:
raise ValueError(
f"Expected 'subclasses_enabled' list items to be types or strings, but got {enable_item!r}"
)

for disable_item in subclasses_disabled or []:
if inspect.isclass(disable_item):
subclasses_disabled_types.add(disable_item)
elif inspect.isfunction(disable_item):
subclasses_disabled_selectors[disable_item.__name__] = disable_item
else:
raise ValueError(
f"Expected 'subclasses_disabled' list items to be types or functions, but got {disable_item!r}"
)


def default_class_instantiator(class_type: type[ClassType], *args, **kwargs) -> ClassType:
Expand Down
4 changes: 2 additions & 2 deletions jsonargparse/_core.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@
class_instantiators,
debug_mode_active,
get_optionals_as_positionals_actions,
is_not_subclass_type,
is_subclasses_disabled,
lenient_check,
parser_context,
supports_optionals_as_positionals,
Expand Down Expand Up @@ -126,7 +126,7 @@ def add_argument(self, *args, enable_path: bool = False, **kwargs):
return ActionParser._move_parser_actions(parser, args, kwargs)
ActionConfigFile._ensure_single_config_argument(self, kwargs["action"])
if "type" in kwargs:
if is_not_subclass_type(kwargs["type"]):
if is_subclasses_disabled(kwargs["type"]):
nested_key = args[0].lstrip("-")
self.add_class_arguments(kwargs.pop("type"), nested_key, **kwargs)
return _find_action(parser, nested_key)
Expand Down
Loading
Loading