Fix static doc viewing in IDEs

.pre-commit-config.yaml

@@ -95,6 +95,17 @@ repos:
         language: python
         entry: ./tools/hooks/sort_contributor_names.py
         files: '^(doc/changes/names.inc|.mailmap)$'
+      - id: check-static-docs
+        name: Check statically filled docstrings against docdict
+        language: python
+        entry: ./tools/hooks/check_static_docs.py
+        args: [--fix]
+        # the hook edits files beyond those passed (docs.py, other users of an
+        # edited entry), so parallel batched invocations would race
+        require_serial: true
+        additional_dependencies: [numpy, scipy, matplotlib, decorator, lazy_loader, packaging, jinja2, pooch, tqdm]
+        files: '^mne/.*\.py$'
+        exclude: '^mne/.*/tests/'
 
   # zizmor
   - repo: https://github.com/woodruffw/zizmor-pre-commit

AGENTS.md

@@ -99,11 +99,17 @@ cropping, projections, export) lives in mixins under `mne/channels/`, `mne/filte
 `mne/utils/mixin.py`, etc. and is composed via multiple inheritance rather than duplicated per
 class.
 
-### Shared/templated docstrings
-Common parameter descriptions live in a central dict in `mne/utils/docs.py` and are spliced into
-function/method docstrings via the `@fill_doc` decorator + `%(param_name)s` placeholders — grep
-for `docdict[` / `@fill_doc` before writing out a parameter docstring by hand, it's likely already
-defined.
+### Shared docstrings (`docdict`)
+Common parameter descriptions live in a central dict in `mne/utils/docs.py`; grep for `docdict[`
+before writing out a parameter docstring by hand, it's likely already defined. New code should use
+`@fill_doc_static("key", ...)` / `@verbose_static("key", ...)` with the full text written out in
+the docstring (and `@copy_function_doc_to_method_doc_static("func:...")` for methods that copy a
+function's docstring); `tools/hooks/check_static_docs.py --fix FILE` expands `%(key)s`
+placeholders and keeps the expanded text in sync with `docdict`. Shared text can be edited either
+in `docdict` or in any one expanded copy; the pre-commit hook (which runs with `--fix`)
+propagates the edit to the other side and fails the commit once so the changes can be reviewed
+and staged. The legacy `@fill_doc`/`@verbose` decorators substitute `%(key)s` at import time
+(IDEs can't see the result); don't add new uses.
 
 ### Changelog is per-PR fragment files (towncrier), not a single hand-edited file
 User-facing changes need a file `doc/changes/dev/<PR-number>.<type>.rst` (types: `notable`,

doc/changes/dev/14195.bugfix.rst

@@ -0,0 +1 @@
+Make docstrings resolve properly in IDEs by fully writing them out in the Python code, by `Eric Larson`_.

doc/development/contributing.rst

@@ -846,6 +846,82 @@ relatively complex. To run some basic tests on documentation, you can use::
     $ make ruff
 
 
+Shared parameter descriptions (the ``docdict``)
+-----------------------------------------------
+
+Many parameters (``picks``, ``n_jobs``, ``verbose``, ``baseline``, ...) and some
+longer passages (e.g., the notes on plotting backends) recur across the API.
+Their text is written once, in the ``docdict`` dictionary in
+:file:`mne/utils/docs.py`, and reused everywhere. Before writing a parameter
+description by hand, ``git grep`` the parameter name in :file:`mne/utils/docs.py`
+--- it is probably already there.
+
+There are two ways a docstring can use ``docdict`` entries:
+
+**Static (preferred for new code):** the docstring contains the *complete*
+text, and the function is decorated with ``@fill_doc_static(...)`` (or
+``@verbose_static(...)`` if it takes a ``verbose`` argument) listing the
+``docdict`` keys it contains::
+
+    @verbose_static("picks_all", "n_jobs")
+    def my_function(inst, picks=None, n_jobs=None, verbose=None):
+        """Do something.
+
+        Parameters
+        ----------
+        inst : instance of Raw
+            The data.
+        picks : str | array-like | slice | None
+            Channels to include. ...          <-- full text of docdict["picks_all"]
+        n_jobs : int | None
+            The number of jobs to run in parallel. ...
+        verbose : bool | str | int | None
+            Control verbosity of the logging output. ...
+        """
+
+Nothing is substituted at import time, so the text you see in the source is
+exactly what ``help()``, Sphinx, and your IDE's hover tooltips show. Methods
+whose docstring is copied from a function (e.g., :meth:`mne.io.Raw.plot` from
+:func:`mne.viz.plot_raw`) work the same way with
+``@copy_function_doc_to_method_doc_static("func:mne.viz.plot_raw")`` (or
+``@copy_doc_static("meth:...")``) above the fully written-out docstring.
+
+A pre-commit hook (``tools/hooks/check_static_docs.py``) keeps every copy in
+sync with its source. It runs with ``--fix``, so it *rewrites* files and fails
+the commit when it had to; review the changes, ``git add`` them, and commit
+again. You can also run it by hand::
+
+    $ python tools/hooks/check_static_docs.py --fix mne/some_module.py
+
+In practice this means:
+
+- **Adding a shared parameter to a function:** write ``%(key)s`` on its own
+  line in the parameter list; the hook expands it and adds the key to the
+  decorator.
+- **Changing shared text:** edit it *either* in :file:`mne/utils/docs.py` *or*
+  in any one docstring that uses it --- the hook detects which side changed
+  (by comparing ``docdict`` with ``git HEAD``) and propagates the edit to
+  ``docdict`` and every other docstring. Entries that are built from templates
+  in :file:`mne/utils/docs.py` rather than written as plain strings can only be
+  edited there; the hook tells you when that is the case.
+- **Site-specific additions** (a ``.. versionadded::`` note, an extra
+  sentence) may follow the shared text within the same parameter block or
+  paragraph; the hook only manages the shared part. When adding such text,
+  start it with a blank line --- lines added directly after the shared text are
+  taken to be shared, and propagated everywhere.
+- **Text that is nearly but not quite shared:** add a new ``docdict`` entry
+  (e.g., ``picks_good_data`` next to ``picks_all``) rather than editing one
+  copy.
+
+**Dynamic (legacy):** the docstring contains ``%(key)s`` placeholders and the
+function is decorated with ``@fill_doc`` (or ``@verbose``), which substitutes
+them at import time. This keeps the source short, but static analysis tools
+only ever see the placeholders (see :gh:`8218`). Existing uses are being
+migrated; please do not add new ones. ``@fill_doc``, ``@verbose``,
+``@copy_doc`` and ``@copy_function_doc_to_method_doc`` remain available (and
+unchanged) for downstream packages.
+
+
 Cross-reference everywhere
 --------------------------
 

doc/development/roadmap.rst

@@ -39,7 +39,11 @@ dictionary called the ``docdict``). There are two major downsides:
    hover-tooltips in IDEs are less useful than they would be if the docstrings were
    complete in-place.
 
-A possible route forward:
+The route now being taken (see :ref:`the contributing guide <contributing>`):
+keep the ``docdict`` as the single place shared text is *authored*, but expand it
+into the source code (``@fill_doc_static`` / ``@verbose_static``) and enforce
+consistency with a pre-commit hook that can also apply ``docdict`` changes to every
+docstring. An earlier alternative that was considered:
 
 - Convert all docstrings to be fully spelled out in the source code.
 - Instead of maintaining the ``docdict``, maintain a registry of sets of

mne/baseline.py

@@ -6,7 +6,7 @@
 
 import numpy as np
 
-from .utils import _check_option, _validate_type, logger, verbose
+from .utils import _check_option, _validate_type, logger, verbose_static
 
 
 def _log_rescale(baseline, mode="mean"):
@@ -23,7 +23,7 @@ def _log_rescale(baseline, mode="mean"):
     return msg
 
 
-@verbose
+@verbose_static("baseline_rescale")
 def rescale(data, times, baseline, mode="mean", copy=True, picks=None, verbose=None):
     """Rescale (baseline correct) data.
 
@@ -34,7 +34,18 @@ def rescale(data, times, baseline, mode="mean", copy=True, picks=None, verbose=N
         dimension should be time.
     times : 1D array
         Time instants is seconds.
-    %(baseline_rescale)s
+    baseline : None | tuple of length 2
+        The time interval to consider as "baseline" when applying baseline
+        correction. If ``None``, do not apply baseline correction.
+        If a tuple ``(a, b)``, the interval is between ``a`` and ``b``
+        (in seconds), including the endpoints.
+        If ``a`` is ``None``, the **beginning** of the data is used; and if ``b``
+        is ``None``, it is set to the **end** of the data.
+        If ``(None, None)``, the entire time interval is used.
+
+        .. note::
+            The baseline ``(a, b)`` includes both endpoints, i.e. all timepoints ``t``
+            such that ``a <= t <= b``.
     mode : 'mean' | 'ratio' | 'logratio' | 'percent' | 'zscore' | 'zlogratio'
         Perform baseline correction by
 
@@ -54,7 +65,11 @@ def rescale(data, times, baseline, mode="mean", copy=True, picks=None, verbose=N
         Whether to return a new instance or modify in place.
     picks : list of int | None
         Data to process along the axis=-2 (None, default, processes all).
-    %(verbose)s
+    verbose : bool | str | int | None
+        Control verbosity of the logging output. If ``None``, use the default
+        verbosity level. See the :ref:`logging documentation <tut-logging>` and
+        :func:`mne.verbose` for details. Should only be passed as a keyword
+        argument.
 
     Returns
     -------

mne/cov.py

@@ -71,6 +71,7 @@
     fill_doc,
     logger,
     verbose,
+    verbose_static,
     warn,
 )
 
@@ -2266,7 +2267,7 @@ def _regularized_covariance(
     return cov
 
 
-@verbose
+@verbose_static("info", "picks_good_data_noref", "rank_none", "on_rank_mismatch")
 def compute_whitener(
     noise_cov,
     info=None,
@@ -2285,10 +2286,63 @@ def compute_whitener(
     ----------
     noise_cov : Covariance
         The noise covariance.
-    %(info)s Can be None if ``noise_cov`` has already been
+    info : mne.Info | None
+        The :class:`mne.Info` object with information about the sensors and methods of measurement.
+        Can be None if ``noise_cov`` has already been
         prepared with :func:`prepare_noise_cov`.
-    %(picks_good_data_noref)s
-    %(rank_none)s
+    picks : str | array-like | slice | None
+        Channels to include. Slices and lists of integers will be interpreted as
+        channel indices. In lists, channel *type* strings (e.g., ``['meg',
+        'eeg']``) will pick channels of those types, channel *name* strings (e.g.,
+        ``['MEG0111', 'MEG2623']`` will pick the given channels. Can also be the
+        string values ``'all'`` to pick all channels, or ``'data'`` to pick
+        :term:`data channels`. None (default) will pick good data channels
+        (excluding reference MEG channels). Note that channels in ``info['bads']``
+        *will be included* if their names or indices are explicitly provided.
+    rank : None | 'info' | 'full' | dict
+        This controls the rank computation that can be read from the
+        measurement info or estimated from the data. When a noise covariance
+        is used for whitening, this should reflect the rank of that covariance,
+        otherwise amplification of noise components can occur in whitening (e.g.,
+        often during source localization).
+
+        :data:`python:None`
+            The rank will be estimated from the data after proper scaling of
+            different channel types.
+        ``'info'``
+            The rank is inferred from ``info``. If data have been processed
+            with Maxwell filtering, the Maxwell filtering header is used.
+            Otherwise, the channel counts themselves are used.
+            In both cases, the number of projectors is subtracted from
+            the (effective) number of channels in the data.
+            For example, if Maxwell filtering reduces the rank to 68, with
+            two projectors the returned value will be 66.
+        ``'full'``
+            The rank is assumed to be full, i.e. equal to the
+            number of good channels. If a `~mne.Covariance` is passed, this can
+            make sense if it has been (possibly improperly) regularized without
+            taking into account the true data rank.
+        :class:`dict`
+            Calculate the rank only for a subset of channel types, and explicitly
+            specify the rank for the remaining channel types. This can be
+            extremely useful if you already **know** the rank of (part of) your
+            data, for instance in case you have calculated it earlier.
+
+            This parameter must be a dictionary whose **keys** correspond to
+            channel types in the data (e.g. ``'meg'``, ``'mag'``, ``'grad'``,
+            ``'eeg'``), and whose **values** are integers representing the
+            respective ranks. For example, ``{'mag': 90, 'eeg': 45}`` will assume
+            a rank of ``90`` and ``45`` for magnetometer data and EEG data,
+            respectively.
+
+            The ranks for all channel types present in the data, but
+            **not** specified in the dictionary will be estimated empirically.
+            That is, if you passed a dataset containing magnetometer, gradiometer,
+            and EEG data together with the dictionary from the previous example,
+            only the gradiometer rank would be determined, while the specified
+            magnetometer and EEG ranks would be taken for granted.
+
+        The default is ``None``.
 
         .. versionadded:: 0.18
            Support for 'info' mode.
@@ -2315,8 +2369,18 @@ def compute_whitener(
         .. versionadded:: 0.18
     return_colorer : bool
         If True, return the colorer as well.
-    %(on_rank_mismatch)s
-    %(verbose)s
+    on_rank_mismatch : str
+        If an explicit MEG value is passed, what to do when it does not match
+        an empirically computed rank (only used for covariances).
+        Can be 'raise' to raise an error, 'warn' (default) to emit a warning, or
+        'ignore' to ignore.
+
+        .. versionadded:: 0.23
+    verbose : bool | str | int | None
+        Control verbosity of the logging output. If ``None``, use the default
+        verbosity level. See the :ref:`logging documentation <tut-logging>` and
+        :func:`mne.verbose` for details. Should only be passed as a keyword
+        argument.
 
     Returns
     -------

mne/evoked.py

@@ -65,6 +65,7 @@
     repr_html,
     sizeof_fmt,
     verbose,
+    verbose_static,
     warn,
 )
 from .utils._typing import Color, Self
@@ -446,7 +447,14 @@ def save(
         """
         write_evokeds(fname, self, overwrite=overwrite)
 
-    @verbose
+    @verbose_static(
+        "export_fmt_support_evoked",
+        "export_warning",
+        "fname_export_params",
+        "export_fmt_params_evoked",
+        "overwrite",
+        "export_warning_note_evoked",
+    )
     def export(
         self,
         fname: str,
@@ -457,22 +465,41 @@ def export(
     ) -> None:
         """Export Evoked to external formats.
 
-        %(export_fmt_support_evoked)s
+        Supported formats:
+
+        - MFF (``.mff``, uses :func:`mne.export.export_evokeds_mff`)
 
-        %(export_warning)s
+        .. warning::
+            Since we are exporting to external formats, there's no guarantee that all
+            the info will be preserved in the external format. See Notes for details.
 
         Parameters
         ----------
-        %(fname_export_params)s
-        %(export_fmt_params_evoked)s
-        %(overwrite)s
-        %(verbose)s
+        fname : str
+            Name of the output file.
+        fmt : 'auto' | 'mff'
+            Format of the export. Defaults to ``'auto'``, which will infer the format
+            from the filename extension. See supported formats above for more
+            information.
+        overwrite : bool
+            If True (default False), overwrite the destination file if it
+            exists.
+        verbose : bool | str | int | None
+            Control verbosity of the logging output. If ``None``, use the default
+            verbosity level. See the :ref:`logging documentation <tut-logging>` and
+            :func:`mne.verbose` for details. Should only be passed as a keyword
+            argument.
 
         Notes
         -----
         .. versionadded:: 1.1
 
-        %(export_warning_note_evoked)s
+        Export to external format may not preserve all the information from the
+        instance. To save in native MNE format (``.fif``) without information loss,
+        use :meth:`mne.Evoked.save` instead.
+        Export does not apply projector(s). Unapplied projector(s) will be lost.
+        Consider applying projector(s) before exporting with
+        :meth:`mne.Evoked.apply_proj`.
         """
         from .export import export_evokeds