From 14a4f672d2275df691e3e1e71a66e80b6ade2cec Mon Sep 17 00:00:00 2001 From: Leandro Lucarella Date: Mon, 21 Sep 2026 12:37:43 +0000 Subject: [PATCH 1/7] Add a guideline on how to deprecate Python symbols `semver-0.x.x.md` says when a deprecated symbol may be removed, but not how to deprecate one, and deprecation applies just as much to libraries past 1.0.0, so it can't live there. The document collects what was learned by actually doing this across our repositories: that a deprecation must never break downstream builds, that `typing_extensions.deprecated` covers the type checker, the rendered docs and the runtime warning from a single message, what PEP 702 explicitly left out and therefore what only the rendered admonition can reach, how to keep an old import path working through a module `__getattr__`, and the test and release note every deprecation needs. Signed-off-by: Leandro Lucarella --- python/deprecations.md | 212 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 212 insertions(+) create mode 100644 python/deprecations.md diff --git a/python/deprecations.md b/python/deprecations.md new file mode 100644 index 0000000..2111f94 --- /dev/null +++ b/python/deprecations.md @@ -0,0 +1,212 @@ +# Deprecations + +This document describes how Python projects at Frequenz deprecate things: what +a deprecation promises to users, how to mark a symbol so that the promise is +visible everywhere it can be, what to do when no marker reaches the thing being +deprecated, and what has to accompany every deprecation. + +It applies to every project regardless of its version. The [0.x.x versioning +rules](semver-0.x.x.md) say *when* a deprecated symbol may be removed in +projects that are still below 1.0.0; this document says how to deprecate it in +the first place, which is the same work after 1.0.0. + +In this document we use the words like MUST, MAY, etc. with the meaning defined +by [RFC2119](https://datatracker.ietf.org/doc/html/rfc2119). + +## A deprecation is never a breaking change + +A deprecated symbol keeps working exactly as before. Code that uses it MUST +keep running, and the test suite of a downstream project MUST keep passing, +until the release that actually removes the symbol. The deprecation is a +message; the removal is the change. + +Two things that are sometimes proposed break this promise and MUST NOT be done: + +* Telling consumers to run with `-W error::DeprecationWarning`. That turns + every deprecation into a crash at the point of use. +* Withholding a symbol from type checkers, for example by defining it only at + runtime, so that `mypy` reports `attr-defined`. That turns every deprecation + into a failing type check. + +Frequenz projects configure `pytest` accordingly, in `pyproject.toml`: + +```toml +[tool.pytest.ini_options] +filterwarnings = [ + "error", + "once::DeprecationWarning", + "once::PendingDeprecationWarning", +] +``` + +This is the same policy written as test configuration: every other warning is +an error, deprecations are reported once and do not fail the suite. + +## Removing a deprecated symbol + +When possible the removal point follows downstream adoption. + +Before removing a deprecated symbol, check whether downstream projects still +import it or still expose it in a public signature. Remove it once they have +migrated, or keep the compatibility symbol until they have. For clients built +on `frequenz-client-common`, this is the rule already stated in its +[deprecation and compatibility +guide](https://github.com/frequenz-floss/frequenz-client-common-python/blob/v0.x.x/docs/wrapping-guide/deprecation-and-compatibility.md). + +When this is not possible, removal after a fixed number of releases or a +specific target release is fine. + +In any case, when removing a deprecated symbol, **always** follow [semver +2.0.0](https://semver.org/spec/v2.0.0.html) (remove only in major releases) and +our own [semver-0.x.x.md](semver-0.x.x.md) (remove only in minor +releases) rules. + +## Use `warnings.deprecated` / `typing_extensions.deprecated` where it reaches + +On a class or a function, the +[`warnings.deprecated`](https://docs.python.org/3/library/warnings.html#warnings.deprecated) +decorator (or +[`typing_extensions.deprecated`](https://typing-extensions.readthedocs.io/en/stable/#typing_extensions.deprecated) +for Python <3.12) gives three things from one message: + +* `mypy` reports every use of the symbol. +* The `griffe-warnings-deprecated` extension turns the message into the + rendered `Deprecated:` admonition in the API documentation. +* The symbol warns at runtime. + +```python +@deprecated( + "fqn.mypkg.OldThing is deprecated since v1.2.0. " + "Use [fqn.mypkg.NewThing][] instead." +) +class OldThing: ... +``` + +Five things that are easy to get wrong: + +* The message is also the runtime warning text, so keep it to a sentence or + two, and write it as implicit concatenation of single-line strings. A + triple-quoted multi-line message keeps its indentation, which stops + cross-references from resolving and prints an indented warning in the + terminal. +* Always use the pattern `fqn.mypkg.OldThing is deprecated`, this allows to add + warning filters that make using deprecations originated for an own library an + error to make sure the library doesn't ship using deprecated symbols. +* Cross-references work in the message. Use the bare `[some.qualified.Name][]` + form rather than wrapping the name in backticks: the backticks buy code font + in the documentation at the cost of more noise in the console. +* Say which version deprecated the symbol, in the sentence itself, as in `"X is + deprecated since v1.2.0. Use [Y][] instead."`. A separate "since" line cannot + be expressed through the decorator, so the two would drift apart. +* On a class, the decorator warns on instantiation only, because it wraps + `__new__`. A type that users receive rather than construct therefore emits no + runtime warning at all, which is worth knowing when judging whether the + runtime signal is doing any work for that symbol. + +## Write the admonition by hand where the decorator cannot reach + +The decorator cannot be applied to a module-level alias, an individual function +argument, an enum member, or a whole module. For those, write a `Deprecated:` +admonition in the docstring: + +```python +def connect(*, payload: bytes, raw: bytes | None = None) -> None: + """Connect to the service. + + Deprecated: + The `raw` argument is deprecated since v1.2.0. Pass `payload` instead. + """ +``` + +* Use no custom title. A title replaces the word "Deprecated" in the rendered + output, so `Deprecated: v1.2.0` renders as just "v1.2.0". The version goes in + the text. +* Put the admonition immediately after the summary line, which is where the + `griffe-warnings-deprecated` extension puts the generated ones. +* Never hand-write an admonition for a symbol the decorator already marks, or + the page shows two. + +Anything longer than "use X instead", such as renamed fields or changed +behavior, belongs in the docstring body as ordinary prose, not in a second +admonition. + +Enum members are the one case with a dedicated helper: +[`frequenz.core.enum.deprecated_member`](https://frequenz-floss.github.io/frequenz-core-python/v1/reference/frequenz/core/enum/#frequenz.core.enum.deprecated_member) +keeps the old member usable and warns when code reaches it. + +## What type checkers cannot do + +The type checker can't report an exact alias, a re-exported name, a module and +a constant. + +[PEP 702](https://peps.python.org/pep-0702/) covers classes, functions and +overloads only. It explicitly rejected deprecating whole modules, object +attributes and constants, and rejected a `Deprecated[T, message]` type +modifier. Accordingly, `mypy` accepts the decorator on `FuncDef`, +`OverloadedFuncDef` and `TypeInfo`, and nothing else. + +A type alias also launders an existing deprecation. Given `Alias: TypeAlias = +DeprecatedClass`, only the alias definition itself is reported; uses of `Alias` +are not reported at all. + +So for everything in that category the rendered admonition is the only channel +that reaches the user, which is why writing it is not optional. + +## Moving a symbol to another module + +When a symbol keeps its identity and only changes location, a module +`__getattr__` can serve the old import path while returning the very same +object, so `isinstance` keeps working through both paths. +[`frequenz.core.warnings.deprecated_aliases()`](https://frequenz-floss.github.io/frequenz-core-python/v1/reference/frequenz/core/warnings/#frequenz.core.warnings.deprecated_aliases) +(available since `frequenz-core` v1.5.0) builds one: + +```python +from typing import TYPE_CHECKING, TypeAlias + +from frequenz.core.warnings import deprecated_aliases + +if TYPE_CHECKING: + from newpkg.newmod import NewThing as _NewThing + + OldThing: TypeAlias = _NewThing +else: + __getattr__ = deprecated_aliases(__name__, {"OldThing": "newpkg.newmod:NewThing"}) +``` + +Two things MUST be observed when doing this: + +* The `__getattr__` assignment belongs in the `else:` branch of an `if + TYPE_CHECKING:` block. An unconditional module-level `__getattr__` makes + `mypy` resolve every unknown name in that module to `Any`, silently, even + under `--strict`, so a typo in a downstream import stops being an error. +* It aliases names, not modules. A package that moved wholesale needs a real + `__init__.py` at the old path whose body holds the alias table, because the + import system never consults a parent package's `__getattr__`. + +## When an exact alias is not the right answer + +An alias keeps the old and new names interchangeable, but it also removes the +type checker's ability to report anything, as described above. Which side wins +depends on the symbol: + +* **Enums:** an enum with members cannot be subclassed, so an alias is the only + option that keeps values comparable. +* **A type whose shape changed:** a separate deprecated class is correct, since + the two are not interchangeable anyway. +* **A type users construct:** a real deprecated class is usually worth more + than an alias, because identity, representation and the static signal matter + more there than cross-compatibility. + +## Every deprecation needs a test and a release note + +Every deprecation MUST come with: + +* An explicit assertion that the symbol warns, for example with + [`pytest.deprecated_call()`](https://docs.pytest.org/en/stable/reference/reference.html#pytest.deprecated_call), + checking the message. +* A migration bullet in the release notes saying what to use instead and what + differs. + +The test matters more than it looks. Because `once::DeprecationWarning` is +configured rather than `error`, a deprecation that silently stops firing does +not fail the suite. Only an explicit assertion catches it. From ef36ae707abc92d6d18fdd48f64d5dfd126053b4 Mon Sep 17 00:00:00 2001 From: Leandro Lucarella Date: Mon, 21 Sep 2026 12:37:47 +0000 Subject: [PATCH 2/7] Point the semver 0.x.x rules at the deprecations guideline Deprecation is what the 0.x.x rules lean on to keep patch releases backwards-compatible, so readers landing there need a way to find how it is actually done. Signed-off-by: Leandro Lucarella --- python/semver-0.x.x.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/python/semver-0.x.x.md b/python/semver-0.x.x.md index 34133c0..4185493 100644 --- a/python/semver-0.x.x.md +++ b/python/semver-0.x.x.md @@ -16,6 +16,11 @@ development fast, and yet make the life of users of those libraries easier, without ending in [dependency hell](https://en.wikipedia.org/wiki/Dependency_hell). +This document covers *when* symbols may be deprecated and removed in 0.x.x +projects. How to actually mark a symbol as deprecated is covered in +[Deprecations](deprecations.md), which applies to every project, including +those past 1.0.0. + In this document we use the words like MUST, MAY, etc. with the meaning defined by [RFC2119](https://datatracker.ietf.org/doc/html/rfc2119). @@ -103,7 +108,8 @@ When tightening a validation rule or invariant on existing code: #### Deprecating a single member or attribute When deprecating individual parts of an existing structure rather than an -entire type or function: +entire type or function (see [Deprecations](deprecations.md) for how to mark +each of these): * **Methods and properties:** Mark them as `@deprecated`. * **Enum members:** Use `frequenz.core.enum.Enum` to mark members as deprecated. From 19f519525e1e1694f8108c6625a762fe93596120 Mon Sep 17 00:00:00 2001 From: Leandro Lucarella Date: Mon, 28 Sep 2026 16:14:03 +0000 Subject: [PATCH 3/7] Document the frequenz-core warnings tooling in the deprecations guide The guide predates three things it now has to describe: the final `frequenz.core.warnings` API (frequenz-core#199), the `griffe-frequenz-core` extension, and the `Deprecated:` admonition convention from frequenz-repo-config#641. Enum members and moved symbols now have helpers that warn at runtime and, through `griffe_frequenz_core.deprecations`, get the same rendered admonition as a decorated symbol, so they move out of the "write it by hand" list, which keeps only function arguments, modules, constants and attributes. The alias example passes a `message` with the version, since the default template has none and the guide requires one. Library code touching its own deprecated symbols is told to use `ignoring_deprecations()` instead of `warnings.catch_warnings()`, which resets the deduplication history for the whole program, and tests get `asserting_no_deprecations()` to check that a replacement doesn't go through the deprecated symbol, a mistake the `once::DeprecationWarning` filter hides. The claim that hand-written admonitions go where the extension puts the generated ones was wrong: both extensions insert them above the summary, which a docstring can't do. The new last section shows the `mkdocs.yml` configuration for both extensions. Signed-off-by: Leandro Lucarella --- python/deprecations.md | 188 +++++++++++++++++++++++++++++++++++++---- 1 file changed, 171 insertions(+), 17 deletions(-) diff --git a/python/deprecations.md b/python/deprecations.md index 2111f94..1a8cbcc 100644 --- a/python/deprecations.md +++ b/python/deprecations.md @@ -71,7 +71,8 @@ for Python <3.12) gives three things from one message: * `mypy` reports every use of the symbol. * The `griffe-warnings-deprecated` extension turns the message into the - rendered `Deprecated:` admonition in the API documentation. + rendered `Deprecated:` admonition in the API documentation (see [Rendering + deprecations in the API documentation](#rendering-deprecations-in-the-api-documentation)). * The symbol warns at runtime. ```python @@ -103,11 +104,48 @@ Five things that are easy to get wrong: runtime warning at all, which is worth knowing when judging whether the runtime signal is doing any work for that symbol. -## Write the admonition by hand where the decorator cannot reach +## Use `frequenz.core` for enum members and moved symbols -The decorator cannot be applied to a module-level alias, an individual function -argument, an enum member, or a whole module. For those, write a `Deprecated:` -admonition in the docstring: +The decorator cannot be applied to an enum member or to a module-level alias. +`frequenz-core` has a helper for each, and both warn at runtime: + +* [`frequenz.core.enum.deprecated_member()`](https://frequenz-floss.github.io/frequenz-core-python/v1/reference/frequenz/core/enum/#frequenz.core.enum.deprecated_member) + keeps an enum member usable, on an enum built from + [`frequenz.core.enum.Enum`](https://frequenz-floss.github.io/frequenz-core-python/v1/reference/frequenz/core/enum/#frequenz.core.enum.Enum), + and warns when code reaches it by name. +* [`frequenz.core.warnings.deprecated_aliases()`](https://frequenz-floss.github.io/frequenz-core-python/v1/reference/frequenz/core/warnings/#frequenz.core.warnings.deprecated_aliases) + keeps an old import path working, see [Moving a symbol to another + module](#moving-a-symbol-to-another-module). + +```python +from frequenz.core.enum import Enum, deprecated_member + + +class TaskStatus(Enum): + OPEN = 1 + PENDING = deprecated_member( + 1, + "fqn.mypkg.TaskStatus.PENDING is deprecated since v1.2.0. " + "Use [fqn.mypkg.TaskStatus.OPEN][] instead.", + ) +``` + +Their messages follow the same rules as the decorator's, and are written as +string literals right in the call. The `griffe_frequenz_core.deprecations` +extension (see [Rendering deprecations in the API +documentation](#rendering-deprecations-in-the-api-documentation)) reads them +without running the code and renders them as the same `Deprecated:` +admonition, so don't write one by hand for these either, unless you need to say +more than the message does: the extension leaves an existing `Deprecated:` +admonition alone instead of adding a second one. A message it can't read, such +as one held in a constant or built by a helper, gets a generic admonition and a +warning instead. + +## Write the admonition by hand where nothing else reaches + +Neither the decorator nor the `frequenz-core` helpers reach an individual +function argument, a whole module, a constant, or an attribute. For those, +write a `Deprecated:` admonition in the docstring: ```python def connect(*, payload: bytes, raw: bytes | None = None) -> None: @@ -118,22 +156,27 @@ def connect(*, payload: bytes, raw: bytes | None = None) -> None: """ ``` +* Write `Deprecated:`, not `Warning: Deprecated`. Only the former renders with + the deprecation style, and it is what the extensions generate. * Use no custom title. A title replaces the word "Deprecated" in the rendered output, so `Deprecated: v1.2.0` renders as just "v1.2.0". The version goes in the text. -* Put the admonition immediately after the summary line, which is where the - `griffe-warnings-deprecated` extension puts the generated ones. -* Never hand-write an admonition for a symbol the decorator already marks, or - the page shows two. +* Put the admonition immediately after the summary line. The extensions put + the generated ones at the very top, above the summary, but a docstring has to + start with its summary, so right after it is as close as a hand-written one + gets. +* Never hand-write an admonition for a symbol the decorator already marks. The + `griffe-warnings-deprecated` extension adds its own regardless, so the page + shows two. +* The admonition is only read in the documentation, right where the deprecated + symbol is, so the message rules above, written for text that is also printed + in a console, don't apply: there is no need to repeat the symbol's name, and + code font and cross-references can be used freely. Anything longer than "use X instead", such as renamed fields or changed behavior, belongs in the docstring body as ordinary prose, not in a second admonition. -Enum members are the one case with a dedicated helper: -[`frequenz.core.enum.deprecated_member`](https://frequenz-floss.github.io/frequenz-core-python/v1/reference/frequenz/core/enum/#frequenz.core.enum.deprecated_member) -keeps the old member usable and warns when code reaches it. - ## What type checkers cannot do The type checker can't report an exact alias, a re-exported name, a module and @@ -149,8 +192,9 @@ A type alias also launders an existing deprecation. Given `Alias: TypeAlias = DeprecatedClass`, only the alias definition itself is reported; uses of `Alias` are not reported at all. -So for everything in that category the rendered admonition is the only channel -that reaches the user, which is why writing it is not optional. +So for everything in that category the rendered admonition, and the runtime +warning where there is one, are the only channels that reach the user, which is +why the admonition is not optional. ## Moving a symbol to another module @@ -163,16 +207,39 @@ object, so `isinstance` keeps working through both paths. ```python from typing import TYPE_CHECKING, TypeAlias -from frequenz.core.warnings import deprecated_aliases +from frequenz.core.warnings import DeprecatedAlias, deprecated_aliases if TYPE_CHECKING: from newpkg.newmod import NewThing as _NewThing OldThing: TypeAlias = _NewThing + """A thing, now called `NewThing`.""" else: - __getattr__ = deprecated_aliases(__name__, {"OldThing": "newpkg.newmod:NewThing"}) + __getattr__ = deprecated_aliases( + __name__, + DeprecatedAlias( + "OldThing", + new_module="newpkg.newmod", + new_name="NewThing", + since="v1.2.0", + ), + ) ``` +Each alias is a `DeprecatedAlias` naming the old symbol and where it is now: +`new_module`, the module it lives in now, and `new_name`, its new name when it +was renamed too. Without `new_module`, the alias points at a symbol renamed in +its own module. Give `since`, the version the alias is deprecated in, and it +warns with this guide's standard wording, `{old} is deprecated since {since}. +Use {new} instead.`; since every alias gives its own, each can say a different +version. The documentation of the alias says "Deprecated since {since}. Use +`{new}` instead.", linking the target: unlike the warning, it is read right +where the alias is, so it doesn't repeat its name. Give `message` instead for a +custom template taking only `{old}` and `{new}`, the fully qualified old and new +names, when the standard wording is not enough. It is a runtime message like a +decorator's, so it follows the same rules, with `[{new}][]` to link the target, +and it is also what the documentation of the alias shows. + Two things MUST be observed when doing this: * The `__getattr__` assignment belongs in the `else:` branch of an `if @@ -197,6 +264,34 @@ depends on the symbol: than an alias, because identity, representation and the static signal matter more there than cross-compatibility. +## Using a deprecated symbol from the library's own code + +A library often has to keep using a symbol it deprecated itself, for example +to convert to or from the old type. The user was already warned when they used +the deprecated symbol, so warning them again from the library's internals is +noise. Wrap the call that reaches the deprecated symbol, and only that call, in +[`frequenz.core.warnings.ignoring_deprecations()`](https://frequenz-floss.github.io/frequenz-core-python/v1/reference/frequenz/core/warnings/#frequenz.core.warnings.ignoring_deprecations): + +```python +from frequenz.core.warnings import ignoring_deprecations + + +def from_wire(raw: str) -> OldThing: + with ignoring_deprecations(): + return OldThing(raw) +``` + +Don't use `warnings.catch_warnings()` for this. Entering and leaving it resets +the warnings deduplication history of the whole program, so every warning +already shown is shown again, on every call +([python/cpython#73858](https://github.com/python/cpython/issues/73858)). +[`ignoring_warnings()`](https://frequenz-floss.github.io/frequenz-core-python/v1/reference/frequenz/core/warnings/#frequenz.core.warnings.ignoring_warnings) +does the same for other warning categories. + +Both only add a filter while the block runs, and that filter is normally shared +by every thread and asyncio task, so keep the block short and don't hold it +across an `await`. + ## Every deprecation needs a test and a release note Every deprecation MUST come with: @@ -210,3 +305,62 @@ Every deprecation MUST come with: The test matters more than it looks. Because `once::DeprecationWarning` is configured rather than `error`, a deprecation that silently stops firing does not fail the suite. Only an explicit assertion catches it. + +The same configuration hides the opposite mistake: a replacement that still +goes through the deprecated symbol internally. To test that the replacement +doesn't warn, wrap it in +[`frequenz.core.warnings.asserting_no_deprecations()`](https://frequenz-floss.github.io/frequenz-core-python/v1/reference/frequenz/core/warnings/#frequenz.core.warnings.asserting_no_deprecations), +or +[`asserting_no_warnings()`](https://frequenz-floss.github.io/frequenz-core-python/v1/reference/frequenz/core/warnings/#frequenz.core.warnings.asserting_no_warnings) +for any category: + +```python +from frequenz.core.warnings import asserting_no_deprecations + + +def test_new_thing_does_not_warn() -> None: + with asserting_no_deprecations(): + NewThing("raw") +``` + +It fails with an `AssertionError` listing each deprecation raised inside the +block. It resets the deduplication history too, so keep it to tests. + +## Rendering deprecations in the API documentation + +The `Deprecated:` admonitions are rendered by two griffe extensions, enabled in +the mkdocstrings Python handler in `mkdocs.yml`: + +* [`griffe-warnings-deprecated`](https://mkdocstrings.github.io/griffe-warnings-deprecated/) + for the `deprecated` decorator. +* [`griffe-frequenz-core`](https://github.com/frequenz-floss/griffe-frequenz-core) + (`griffe_frequenz_core.deprecations`, v1.0.1 or later) for + `deprecated_aliases()` and `deprecated_member()`. + +```yaml +plugins: + - mkdocstrings: + handlers: + python: + options: + extensions: + - griffe_warnings_deprecated: + kind: deprecated + title: Deprecated + - griffe_frequenz_core.deprecations: + kind: deprecated + title: Deprecated +``` + +Both packages go in the `dev-mkdocs` dependencies. The `deprecated` kind is +styled in `docs/_css/mkdocstrings.css`, which, like the first extension, comes +with the [repository configuration +template](https://github.com/frequenz-floss/frequenz-repo-config-python); a +hand-written `Deprecated:` admonition uses the same kind, so all of them look +alike. + +Frequenz projects build their documentation in strict mode, so warnings fail +the build. `griffe-frequenz-core` warns whenever it can't document a +deprecation in full, and a link to a replacement that can't be resolved warns +too. For a symbol that moved to another project, add that project's +`objects.inv` to the `inventories` of the Python handler, so the link resolves. From 38fa1eaaed4bfbc343ffe4ff1bf9d3f10ef2f48a Mon Sep 17 00:00:00 2001 From: Leandro Lucarella Date: Mon, 28 Sep 2026 16:31:59 +0000 Subject: [PATCH 4/7] Point the semver enum member rule at deprecated_member() The "Enum members" bullet says to use `frequenz.core.enum.Enum` but not how a member is actually marked, which the deprecations guide now covers with `deprecated_member()`, so link that section. Signed-off-by: Leandro Lucarella --- python/semver-0.x.x.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/python/semver-0.x.x.md b/python/semver-0.x.x.md index 4185493..31070ef 100644 --- a/python/semver-0.x.x.md +++ b/python/semver-0.x.x.md @@ -112,7 +112,8 @@ entire type or function (see [Deprecations](deprecations.md) for how to mark each of these): * **Methods and properties:** Mark them as `@deprecated`. -* **Enum members:** Use `frequenz.core.enum.Enum` to mark members as deprecated. +* **Enum members:** Use `frequenz.core.enum.Enum` to mark members as deprecated, + see [Use `frequenz.core` for enum members and moved symbols](deprecations.md#use-frequenzcore-for-enum-members-and-moved-symbols). * **Plain data attributes:** Plain data attributes and public fields typically lack runtime interception points. Mark them as deprecated in documentation and static type annotations, and defer structural removal or renaming to a From e6ea1b246184f7c19dc4ae1e9f511533acd6e19a Mon Sep 17 00:00:00 2001 From: Leandro Lucarella Date: Tue, 29 Sep 2026 07:52:15 +0000 Subject: [PATCH 5/7] Add the own deprecations error filter to the pytest configuration The repository configuration template now adds a `filterwarnings` entry that turns deprecation warnings mentioning the project's own fully qualified names into errors, so a project can't release code still using symbols it deprecated itself. Dependencies' deprecations stay warnings, so this doesn't contradict the rule that a deprecation must never break downstream builds. The guide showed the configuration without it, and the message rule that makes it work was only loosely motivated, so both now point at each other, and the rule says the name must be plain text, since backticks or a cross-reference stop the filter from matching. The testing section assumed every deprecation was only reported once. It now says that tests using the project's own deprecated symbols on purpose need `pytest.deprecated_call()` or a `filterwarnings` mark, and that the filter catches a replacement still going through the deprecated symbol only heuristically, which is why `asserting_no_deprecations()` is still recommended. Signed-off-by: Leandro Lucarella --- python/deprecations.md | 45 ++++++++++++++++++++++++++++++++---------- 1 file changed, 35 insertions(+), 10 deletions(-) diff --git a/python/deprecations.md b/python/deprecations.md index 1a8cbcc..616d1ef 100644 --- a/python/deprecations.md +++ b/python/deprecations.md @@ -36,11 +36,26 @@ filterwarnings = [ "error", "once::DeprecationWarning", "once::PendingDeprecationWarning", + 'error:.*fqn\.mypkg\.[\w\.]+ (is|was) deprecated:DeprecationWarning', ] ``` This is the same policy written as test configuration: every other warning is -an error, deprecations are reported once and do not fail the suite. +an error, deprecations coming from dependencies are reported once and do not +fail the suite. + +The last entry is the exception: using a symbol the project deprecated itself +is an error, so the project never releases code that still uses it, which +users would get as warnings they can do nothing about. It doesn't break +downstream projects, since it only matches the project's own package (here +`fqn.mypkg`), and it goes after the `once::` entries because later filters take +precedence. A filter can only tell the project's own deprecations apart by +their message, so this is a heuristic that relies on every message starting +with the fully qualified name of the deprecated symbol (see [the message +rules](#use-warningsdeprecated--typing_extensionsdeprecated-where-it-reaches)). +The [repository configuration +template](https://github.com/frequenz-floss/frequenz-repo-config-python) +generates this entry and its migration script adds it to existing projects. ## Removing a deprecated symbol @@ -90,9 +105,12 @@ Five things that are easy to get wrong: triple-quoted multi-line message keeps its indentation, which stops cross-references from resolving and prints an indented warning in the terminal. -* Always use the pattern `fqn.mypkg.OldThing is deprecated`, this allows to add - warning filters that make using deprecations originated for an own library an - error to make sure the library doesn't ship using deprecated symbols. +* Always start with the pattern `fqn.mypkg.OldThing is deprecated`, with the + fully qualified name as plain text, neither in backticks nor as a + cross-reference. The `pytest` filter in [A deprecation is never a breaking + change](#a-deprecation-is-never-a-breaking-change) relies on it to make the + project's own uses of the symbol an error, and misses any message that + doesn't follow it. * Cross-references work in the message. Use the bare `[some.qualified.Name][]` form rather than wrapping the name in backticks: the backticks buy code font in the documentation at the cost of more noise in the console. @@ -302,13 +320,20 @@ Every deprecation MUST come with: * A migration bullet in the release notes saying what to use instead and what differs. -The test matters more than it looks. Because `once::DeprecationWarning` is -configured rather than `error`, a deprecation that silently stops firing does -not fail the suite. Only an explicit assertion catches it. +The test matters more than it looks. A deprecation that silently stops firing +fails nothing, whatever the warning filters say, since there is no warning left +to turn into an error. Only an explicit assertion catches it. -The same configuration hides the opposite mistake: a replacement that still -goes through the deprecated symbol internally. To test that the replacement -doesn't warn, wrap it in +`pytest.deprecated_call()` also records the warning instead of letting the +filter for the project's own deprecations turn it into an error, so any test +that uses one of the project's deprecated symbols on purpose needs it. Where +checking the warning doesn't make sense, mark the test with +`@pytest.mark.filterwarnings("once::DeprecationWarning")` instead. + +That filter also catches the opposite mistake, a replacement that still goes +through the deprecated symbol internally, but only for messages it matches, and +a deprecation coming from a dependency is still only reported once. To test +that the replacement doesn't warn at all, wrap it in [`frequenz.core.warnings.asserting_no_deprecations()`](https://frequenz-floss.github.io/frequenz-core-python/v1/reference/frequenz/core/warnings/#frequenz.core.warnings.asserting_no_deprecations), or [`asserting_no_warnings()`](https://frequenz-floss.github.io/frequenz-core-python/v1/reference/frequenz/core/warnings/#frequenz.core.warnings.asserting_no_warnings) From 3d6122d71a92fc77c1977c5d87eab048dc725247 Mon Sep 17 00:00:00 2001 From: Leandro Lucarella Date: Tue, 6 Oct 2026 10:16:29 +0000 Subject: [PATCH 6/7] Write the semver example's deprecation message as the guide says The example deprecating `foo` used a message the deprecations guide asks not to write: no fully qualified name to start it, which the own deprecations pytest filter relies on, no version, and the replacement in backticks instead of a cross-reference. Signed-off-by: Leandro Lucarella --- python/semver-0.x.x.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/python/semver-0.x.x.md b/python/semver-0.x.x.md index 31070ef..455cd4b 100644 --- a/python/semver-0.x.x.md +++ b/python/semver-0.x.x.md @@ -166,7 +166,7 @@ We can create a new function `foo2` with the new name and deprecate the old one: ```python -@deprecated("foo is deprecated, please use `foo2` instead") +@deprecated("mypkg.foo is deprecated since v0.1.1. Use [mypkg.foo2][] instead.") def foo(wrong_arg: int) -> None: ... def foo2(right_arg: str) -> None: ... From 545ca05761a465a23d9827a94f2a7ebb82607119 Mon Sep 17 00:00:00 2001 From: Leandro Lucarella Date: Tue, 6 Oct 2026 13:28:24 +0000 Subject: [PATCH 7/7] Separate the deprecation warning from the documentation notice A single message served both the runtime warning and the documentation notice, so it was a compromise for both: cross-references and backticks are noise in a terminal and in mypy's output, the version churns the warning without helping whoever runs the code, and the fully qualified name the warning needs is redundant right where the symbol is documented. Following the guide also didn't make anyone link the replacement, since nothing said it had to. The guide now describes them as two texts with their own rules: the warning is plain text starting with the fully qualified name (which the own deprecations pytest filter needs) and saying what to use instead, without the version; the notice says since which version and links the replacement in code font, with a short link text when the context makes it obvious. Symbols sharing a long explanation, such as the members of a deprecated enum, link the notice that explains it instead of repeating it, while their warnings still explain it in full. It describes the tooling as it will be when this work is done: deprecated_member() takes new_name and since like DeprecatedAlias, DeprecatedAlias's since only feeds the notice and is independent of its message, and griffe-frequenz-core also handles the deprecated decorator, so a hand-written notice always replaces a generated one, which is the one way to customize the documentation. A new section sums up what is generated, what has to be written by hand, and what happens otherwise. Signed-off-by: Leandro Lucarella --- python/deprecations.md | 272 ++++++++++++++++++++++++++--------------- python/semver-0.x.x.md | 9 +- 2 files changed, 182 insertions(+), 99 deletions(-) diff --git a/python/deprecations.md b/python/deprecations.md index 616d1ef..b30e665 100644 --- a/python/deprecations.md +++ b/python/deprecations.md @@ -51,8 +51,8 @@ downstream projects, since it only matches the project's own package (here `fqn.mypkg`), and it goes after the `once::` entries because later filters take precedence. A filter can only tell the project's own deprecations apart by their message, so this is a heuristic that relies on every message starting -with the fully qualified name of the deprecated symbol (see [the message -rules](#use-warningsdeprecated--typing_extensionsdeprecated-where-it-reaches)). +with the fully qualified name of the deprecated symbol (see [the rules for +warnings](#the-warning-and-the-notice)). The [repository configuration template](https://github.com/frequenz-floss/frequenz-repo-config-python) generates this entry and its migration script adds it to existing projects. @@ -76,56 +76,100 @@ In any case, when removing a deprecated symbol, **always** follow [semver our own [semver-0.x.x.md](semver-0.x.x.md) (remove only in minor releases) rules. +## The warning and the notice + +Every deprecation reaches users through two texts, written for different +readers, so they follow different rules: + +* The **warning**, emitted at runtime and, for decorated symbols, also reported + by `mypy`. It is read in a terminal or a log, far from the deprecated symbol: + it points at the line that used it, not at the symbol itself. +* The **notice**, the `Deprecated:` admonition in the API documentation. It is + read right where the deprecated symbol is documented. + +The warning MUST: + +* Start with the pattern `fqn.mypkg.OldThing is deprecated`, with the fully + qualified name as plain text. The `pytest` filter in [A deprecation is never + a breaking change](#a-deprecation-is-never-a-breaking-change) relies on it to + make the project's own uses of the symbol an error, and misses any message + that doesn't follow it. +* Say what to use instead, by its fully qualified name too, as in + `fqn.mypkg.OldThing is deprecated. Use fqn.mypkg.NewThing instead.` +* Be plain text, without backticks or cross-references, which are only noise in + a terminal. +* Be a sentence or two, written as implicit concatenation of single-line + strings. A triple-quoted multi-line message keeps its indentation and prints + an indented warning. + +It SHOULD NOT say since which version the symbol is deprecated. What matters +to whoever runs the code is that it is deprecated now; the version belongs in +the notice, and leaving it out keeps the warning short. + +The notice MUST: + +* Say since which version the symbol is deprecated and what to use instead, + linking the replacement in code font, as in ``Deprecated since v1.2.0. Use + [`NewThing`][fqn.mypkg.NewThing] instead.`` +* Not repeat the name of the deprecated symbol, which is right above it. + +For the link text, use the fully qualified name when the replacement lives in +another module, so readers see where it went, as in +``[`fqn.newpkg.NewThing`][]``, and just its name when the context makes it +obvious, such as another member of the same enum, as in +``[`OPEN`][fqn.mypkg.TaskStatus.OPEN]``. + +Anything longer than "use X instead", such as renamed fields or changed +behavior, belongs in the docstring body as ordinary prose, not in the notice. + +When many symbols share the same long explanation, such as every member of a +deprecated enum, explain it once, in the notice of the symbol they belong to, +and have the others link it: ``Deprecated since v1.2.0. See +[`OldEnum`][fqn.mypkg.OldEnum] for what to use instead.`` A link is easy to +follow in the documentation, but not in a terminal, so each warning still +explains the replacement in full. + +The tools below generate both texts wherever they have what they need, which +[What the documentation tooling does](#what-the-documentation-tooling-does) +sums up; everywhere else the notice is [written by +hand](#writing-the-notice-by-hand). + ## Use `warnings.deprecated` / `typing_extensions.deprecated` where it reaches On a class or a function, the [`warnings.deprecated`](https://docs.python.org/3/library/warnings.html#warnings.deprecated) decorator (or [`typing_extensions.deprecated`](https://typing-extensions.readthedocs.io/en/stable/#typing_extensions.deprecated) -for Python <3.12) gives three things from one message: +for Python <3.12) gives three things: * `mypy` reports every use of the symbol. -* The `griffe-warnings-deprecated` extension turns the message into the - rendered `Deprecated:` admonition in the API documentation (see [Rendering - deprecations in the API documentation](#rendering-deprecations-in-the-api-documentation)). * The symbol warns at runtime. +* The symbol is marked as deprecated in the API documentation. + +The decorator only takes the warning, so the notice is written by hand in the +docstring: ```python -@deprecated( - "fqn.mypkg.OldThing is deprecated since v1.2.0. " - "Use [fqn.mypkg.NewThing][] instead." -) -class OldThing: ... +@deprecated("fqn.mypkg.OldThing is deprecated. Use fqn.mypkg.NewThing instead.") +class OldThing: + """A thing, now called `NewThing`. + + Deprecated: + Deprecated since v1.2.0. Use [`NewThing`][fqn.mypkg.NewThing] instead. + """ ``` -Five things that are easy to get wrong: - -* The message is also the runtime warning text, so keep it to a sentence or - two, and write it as implicit concatenation of single-line strings. A - triple-quoted multi-line message keeps its indentation, which stops - cross-references from resolving and prints an indented warning in the - terminal. -* Always start with the pattern `fqn.mypkg.OldThing is deprecated`, with the - fully qualified name as plain text, neither in backticks nor as a - cross-reference. The `pytest` filter in [A deprecation is never a breaking - change](#a-deprecation-is-never-a-breaking-change) relies on it to make the - project's own uses of the symbol an error, and misses any message that - doesn't follow it. -* Cross-references work in the message. Use the bare `[some.qualified.Name][]` - form rather than wrapping the name in backticks: the backticks buy code font - in the documentation at the cost of more noise in the console. -* Say which version deprecated the symbol, in the sentence itself, as in `"X is - deprecated since v1.2.0. Use [Y][] instead."`. A separate "since" line cannot - be expressed through the decorator, so the two would drift apart. -* On a class, the decorator warns on instantiation only, because it wraps - `__new__`. A type that users receive rather than construct therefore emits no - runtime warning at all, which is worth knowing when judging whether the - runtime signal is doing any work for that symbol. +On a class, the decorator warns on instantiation only, because it wraps +`__new__`. A type that users receive rather than construct therefore emits no +runtime warning at all, which is worth knowing when judging whether the runtime +signal is doing any work for that symbol. ## Use `frequenz.core` for enum members and moved symbols The decorator cannot be applied to an enum member or to a module-level alias. -`frequenz-core` has a helper for each, and both warn at runtime: +`frequenz-core` has a helper for each, both warn at runtime, and both take the +deprecation as structured arguments, so the warning and the notice are both +generated from them: * [`frequenz.core.enum.deprecated_member()`](https://frequenz-floss.github.io/frequenz-core-python/v1/reference/frequenz/core/enum/#frequenz.core.enum.deprecated_member) keeps an enum member usable, on an enum built from @@ -141,59 +185,94 @@ from frequenz.core.enum import Enum, deprecated_member class TaskStatus(Enum): OPEN = 1 - PENDING = deprecated_member( - 1, - "fqn.mypkg.TaskStatus.PENDING is deprecated since v1.2.0. " - "Use [fqn.mypkg.TaskStatus.OPEN][] instead.", + PENDING = deprecated_member(1, new_name="OPEN", since="v1.2.0") +``` + +`new_name` is the member to use instead, in the same enum, and `since` the +version it is deprecated in. Reaching `TaskStatus.PENDING` warns +`fqn.mypkg.TaskStatus.PENDING is deprecated. Use fqn.mypkg.TaskStatus.OPEN +instead.`, and its notice says ``Deprecated since v1.2.0. Use +[`OPEN`][fqn.mypkg.TaskStatus.OPEN] instead.`` + +When there is no member to point at, or the standard wording is not enough, +pass a `message`, written by the rules for warnings above. It replaces the +warning only, so write the notice by hand too: + +```python +class TaskStatus(Enum): + UNSPECIFIED = deprecated_member( + 0, + "fqn.mypkg.TaskStatus.UNSPECIFIED is deprecated. " + "Use the integer 0 instead if you need that low-level value.", + since="v1.2.0", ) + """The status is unspecified. + + Deprecated: + Deprecated since v1.2.0. Use the integer `0` instead if you need that + low-level value. + """ ``` -Their messages follow the same rules as the decorator's, and are written as -string literals right in the call. The `griffe_frequenz_core.deprecations` -extension (see [Rendering deprecations in the API -documentation](#rendering-deprecations-in-the-api-documentation)) reads them -without running the code and renders them as the same `Deprecated:` -admonition, so don't write one by hand for these either, unless you need to say -more than the message does: the extension leaves an existing `Deprecated:` -admonition alone instead of adding a second one. A message it can't read, such -as one held in a constant or built by a helper, gets a generic admonition and a -warning instead. +The documentation is built without running the code, so write these arguments +as string literals right in the call. What can't be read that way is documented +as well as possible, with a warning, see [What the documentation tooling +does](#what-the-documentation-tooling-does). + +## Writing the notice by hand -## Write the admonition by hand where nothing else reaches +Write a `Deprecated:` admonition in the docstring: -Neither the decorator nor the `frequenz-core` helpers reach an individual -function argument, a whole module, a constant, or an attribute. For those, -write a `Deprecated:` admonition in the docstring: +* For what nothing marks: an individual function argument, a whole module, a + constant, or an attribute. +* For a symbol deprecated with the decorator, which only carries the warning. +* To say more than a generated notice does, for example for an alias or enum + member with its own `message`. A hand-written notice always replaces the + generated one, and it is the only way to change what the documentation says: + `message` only changes the warning. ```python def connect(*, payload: bytes, raw: bytes | None = None) -> None: """Connect to the service. Deprecated: - The `raw` argument is deprecated since v1.2.0. Pass `payload` instead. + The `raw` argument is deprecated since v1.2.0. Use `payload` instead. """ ``` +* Follow the rules for notices in [The warning and the + notice](#the-warning-and-the-notice). A notice about part of a symbol, such + as one argument, has to say which part. * Write `Deprecated:`, not `Warning: Deprecated`. Only the former renders with - the deprecation style, and it is what the extensions generate. + the deprecation style, and it is what the tooling generates. * Use no custom title. A title replaces the word "Deprecated" in the rendered output, so `Deprecated: v1.2.0` renders as just "v1.2.0". The version goes in the text. -* Put the admonition immediately after the summary line. The extensions put - the generated ones at the very top, above the summary, but a docstring has to - start with its summary, so right after it is as close as a hand-written one - gets. -* Never hand-write an admonition for a symbol the decorator already marks. The - `griffe-warnings-deprecated` extension adds its own regardless, so the page - shows two. -* The admonition is only read in the documentation, right where the deprecated - symbol is, so the message rules above, written for text that is also printed - in a console, don't apply: there is no need to repeat the symbol's name, and - code font and cross-references can be used freely. - -Anything longer than "use X instead", such as renamed fields or changed -behavior, belongs in the docstring body as ordinary prose, not in a second -admonition. +* Put it immediately after the summary line, since a docstring has to start + with its summary. On a symbol the tooling marks as deprecated, a decorated + class or function, an enum member or an alias, the notice is moved to the + top, where generated ones go, above the summary. Anywhere else, such as on a + module or for an argument, it stays where it is written. +* Write one notice per symbol. Anything longer than "use X instead" goes in the + docstring body as ordinary prose. + +## What the documentation tooling does + +| Deprecated with | Warning | Notice generated from | Write the notice by hand | +| -------------------------------------- | ----------------------------- | -------------------------- | --------------------------------------- | +| `@deprecated(...)` | the decorator's message | nothing | always | +| `deprecated_member(..., new_name=...)` | generated, or `message` | `since` and `new_name` | to say more, or with a custom `message` | +| `deprecated_member(..., message)` | `message` | nothing | always | +| `DeprecatedAlias(...)` | generated, or `message` | `since` and the new path | to say more, or with a custom `message` | +| nothing (module, constant, argument) | whatever the code emits | nothing | always | + +A deprecated symbol the tooling marks always gets a notice, even when it can't +be written as this guide asks: if there is no hand-written notice and nothing to +generate one from, the documentation shows the warning, or as much as could be +read, such as ``Deprecated. Use [`NewThing`][fqn.mypkg.NewThing] instead.`` +when `since` is missing. Each such case logs a warning, which fails the strict +build, so it doesn't go unnoticed; the fix is to give the missing arguments as +string literals, or to write the notice by hand. ## What type checkers cannot do @@ -247,16 +326,19 @@ else: Each alias is a `DeprecatedAlias` naming the old symbol and where it is now: `new_module`, the module it lives in now, and `new_name`, its new name when it was renamed too. Without `new_module`, the alias points at a symbol renamed in -its own module. Give `since`, the version the alias is deprecated in, and it -warns with this guide's standard wording, `{old} is deprecated since {since}. -Use {new} instead.`; since every alias gives its own, each can say a different -version. The documentation of the alias says "Deprecated since {since}. Use -`{new}` instead.", linking the target: unlike the warning, it is read right -where the alias is, so it doesn't repeat its name. Give `message` instead for a -custom template taking only `{old}` and `{new}`, the fully qualified old and new -names, when the standard wording is not enough. It is a runtime message like a -decorator's, so it follows the same rules, with `[{new}][]` to link the target, -and it is also what the documentation of the alias shows. +its own module. `since` is the version the alias is deprecated in; since every +alias gives its own, each can say a different version. + +From those, the alias warns with the standard wording, `{old} is deprecated. +Use {new} instead.`, with fully qualified names, and its notice says +``Deprecated since {since}. Use [`{new}`][] instead.``, showing just the new +name for a rename within the same module. + +When the standard warning is not enough, give `message` too, a template taking +only `{old}` and `{new}`, the fully qualified old and new names, and written by +the rules for warnings. It replaces the warning only: to say more in the +documentation, write the notice by hand in the docstring of the alias declared +for type checkers, which replaces the generated one. Two things MUST be observed when doing this: @@ -353,14 +435,12 @@ block. It resets the deduplication history too, so keep it to tests. ## Rendering deprecations in the API documentation -The `Deprecated:` admonitions are rendered by two griffe extensions, enabled in -the mkdocstrings Python handler in `mkdocs.yml`: - -* [`griffe-warnings-deprecated`](https://mkdocstrings.github.io/griffe-warnings-deprecated/) - for the `deprecated` decorator. -* [`griffe-frequenz-core`](https://github.com/frequenz-floss/griffe-frequenz-core) - (`griffe_frequenz_core.deprecations`, v1.0.1 or later) for - `deprecated_aliases()` and `deprecated_member()`. +The notices are rendered by the +[`griffe-frequenz-core`](https://github.com/frequenz-floss/griffe-frequenz-core) +extension (`griffe_frequenz_core.deprecations`), enabled in the mkdocstrings +Python handler in `mkdocs.yml`. It handles the `deprecated` decorator, +`deprecated_member()` and `deprecated_aliases()` alike, as [What the +documentation tooling does](#what-the-documentation-tooling-does) describes: ```yaml plugins: @@ -369,23 +449,21 @@ plugins: python: options: extensions: - - griffe_warnings_deprecated: - kind: deprecated - title: Deprecated - griffe_frequenz_core.deprecations: kind: deprecated title: Deprecated ``` -Both packages go in the `dev-mkdocs` dependencies. The `deprecated` kind is -styled in `docs/_css/mkdocstrings.css`, which, like the first extension, comes -with the [repository configuration +The package goes in the `dev-mkdocs` dependencies. The `deprecated` kind is +styled in `docs/_css/mkdocstrings.css`, and both come with the [repository +configuration template](https://github.com/frequenz-floss/frequenz-repo-config-python); a hand-written `Deprecated:` admonition uses the same kind, so all of them look alike. Frequenz projects build their documentation in strict mode, so warnings fail the build. `griffe-frequenz-core` warns whenever it can't document a -deprecation in full, and a link to a replacement that can't be resolved warns -too. For a symbol that moved to another project, add that project's -`objects.inv` to the `inventories` of the Python handler, so the link resolves. +deprecation as this guide asks, and a link to a replacement that can't be +resolved warns too. For a symbol that moved to another project, add that +project's `objects.inv` to the `inventories` of the Python handler, so the link +resolves. diff --git a/python/semver-0.x.x.md b/python/semver-0.x.x.md index 455cd4b..f4a518c 100644 --- a/python/semver-0.x.x.md +++ b/python/semver-0.x.x.md @@ -166,8 +166,13 @@ We can create a new function `foo2` with the new name and deprecate the old one: ```python -@deprecated("mypkg.foo is deprecated since v0.1.1. Use [mypkg.foo2][] instead.") -def foo(wrong_arg: int) -> None: ... +@deprecated("mypkg.foo is deprecated. Use mypkg.foo2 instead.") +def foo(wrong_arg: int) -> None: + """Do foo. + + Deprecated: + Deprecated since v0.1.1. Use [`foo2`][mypkg.foo2] instead. + """ def foo2(right_arg: str) -> None: ... ```