Settings form logic and saving settings into a plugin block (M5 part 3a-1)

CLAUDE.md

@@ -108,6 +108,7 @@ Every PR costs txoof review time. Too many small PRs and too few huge ones both
 ## Tools and commands
 - Python 3.13 (the version in Raspberry Pi OS trixie). `uv` installs Python and all packages into `.venv`.
 - `uv sync`: install. `uv run pytest`: tests. `uv run ruff check .` and `uv run ruff format .`: code style.
+- `uv run pytest -n auto` runs the tests on every processor core at once (pytest-xdist; about half the time on the 4-core Pi). While working, run only the tests of the files you changed; run all of them once before the first push of a PR. GitHub runs all of them for every pull request and every merge to main.
 - Plain `.py` files only. No Jupyter notebooks in the repo.
 - Tests that need a real display are marked `@pytest.mark.hardware`. They are skipped by default; run them on the Pi with `uv run pytest -m hardware`.
 

docs/decisions/web-interface.md

@@ -35,6 +35,8 @@ In v2 the web interface is the main way to set up and change PaperPi: plugins, s
 - On Save, the values are checked with the same description. Errors are shown next to the field. A saved change applies at once (see `live-config-reload.md`).
 - A setting that needs more than a plain field can name an input type from a small list in the web interface, e.g. `location` (lat/lon lookup), `server_search` (find music servers), `layout_picker`. New input types are added when a plugin needs one.
 - Plugins added later show up without changes to the web code.
+- *Agreed with txoof on 2026-10-08 (M5 part 3a):* the page shows the plugin's `name` first, then its own settings, then display time, refresh, layout and level; the other shared settings (time limit, alert and storage settings) are folded under "More settings", which opens by itself when one of them has an error or a value that is not the default. Each field shows its value, or the default when the file doesn't set it.
+- How a save works (M5 part 3a-1): only settings that changed are written. An empty field means "the default" (an empty check box: off; an empty list: nothing picked); a value changed to the default is taken out of the file and becomes the comment `# key = default` again (a value left as it is stays, also one written in the file that equals the default), so the help text above it still fits. A new setting takes the place of its `# key = ...` comment line when there is one. If any value has an error, nothing is saved; each error is shown next to its field, with what was typed (never a secret). A secret (such as an API key) is never shown on the page; leaving its field empty keeps the saved one. A setting the form can't show (a group of settings, a list of numbers) is shown as it is in the file, to be changed there. A setting written over several lines in the file is never changed by the page.
 
 ### Logging in
 

pyproject.toml

@@ -27,6 +27,7 @@ dev = [
     "httpx2>=2.13.1",
     "pytest>=8",
     "pytest-html>=4",
+    "pytest-xdist>=3.8.0",
     "ruff>=0.6",
 ]
 

src/paperpi/example.py

@@ -113,13 +113,13 @@ def plugin_block(
     ]
     if shared:
         lines += _comment(entry["id"].description or "")
-    lines.append(f"id = {_toml(plugin_id)}")
+    lines.append(f"id = {toml_value(plugin_id)}")
     named = "name" in values
     if shared:
         lines += _setting("name", entry["name"], values.get("name", ""), comment=not named)
     elif named:
-        lines.append(f"name = {_toml(values['name'])}")
-    lines.append(f"type = {_toml(plugin.type)}")
+        lines.append(f"name = {toml_value(values['name'])}")
+    lines.append(f"type = {toml_value(plugin.type)}")
     for key, value in values.items():
         if key == "name":
             continue
@@ -199,11 +199,11 @@ def _setting(key: str, info: FieldInfo, value: Any, *, comment: bool = False) ->
     help_text = info.description or key
     if is_required(info):
         help_text += " (required)"
-    choices = _choices(info.annotation)
-    named = all(re.search(rf"\b{re.escape(str(c))}\b", help_text) for c in choices)
-    if choices and not named:
-        help_text += f". One of: {', '.join(_toml(c) for c in choices)}"
-    line = f"{key} = {_toml(value)}" if value is not None else f"{key} ="
+    allowed = choices(info.annotation)
+    named = all(re.search(rf"\b{re.escape(str(c))}\b", help_text) for c in allowed)
+    if allowed and not named:
+        help_text += f". One of: {', '.join(toml_value(c) for c in allowed)}"
+    line = f"{key} = {toml_value(value)}" if value is not None else f"{key} ="
     return [*_comment(help_text), f"# {line}" if comment else line]
 
 
@@ -219,7 +219,7 @@ def _check_reads_back(text: str, values: Mapping[str, Any]) -> None:
     TOML than Python's reader understands; such a block would make the whole file
     unreadable.
     """
-    expected = {"plugin": [{k: _plain(v) for k, v in values.items()}]}
+    expected = {"plugin": [{k: plain(v) for k, v in values.items()}]}
     try:
         read = tomllib.loads(text)
     except tomllib.TOMLDecodeError as error:
@@ -228,7 +228,7 @@ def _check_reads_back(text: str, values: Mapping[str, Any]) -> None:
         raise ValueError("can't be written so that PaperPi reads it back the same")
 
 
-def _choices(annotation: Any) -> tuple:
+def choices(annotation: Any) -> tuple:
     """The allowed values of a ``Literal`` setting (also inside ``... | None``)."""
     if typing.get_origin(annotation) is Literal:
         return typing.get_args(annotation)
@@ -239,7 +239,7 @@ def _choices(annotation: Any) -> tuple:
     return ()
 
 
-def _plain(value: Any) -> Any:
+def plain(value: Any) -> Any:
     """``value`` as plain data: text, numbers, true/false, lists and dictionaries."""
     if hasattr(value, "get_secret_value"):
         value = value.get_secret_value()
@@ -248,15 +248,15 @@ def _plain(value: Any) -> Any:
     return to_jsonable_python(value)
 
 
-def _toml(value: Any) -> str:
+def toml_value(value: Any) -> str:
     """``value`` written as in a TOML file: ``24``, ``true``, ``"Berlin"``, ``[1, 2]``,
     ``{a = 1}`` (a group of settings on one line)."""
-    value = _plain(value)
+    value = plain(value)
     if isinstance(value, dict):
-        items = (f"{tomlkit.key(k).as_string()} = {_toml(v)}" for k, v in value.items())
+        items = (f"{tomlkit.key(k).as_string()} = {toml_value(v)}" for k, v in value.items())
         return "{" + ", ".join(items) + "}"
     if isinstance(value, list):
-        return "[" + ", ".join(_toml(v) for v in value) + "]"
+        return "[" + ", ".join(toml_value(v) for v in value) + "]"
     if isinstance(value, float) and value.is_integer():
         value = int(value)  # 120, not 120.0
     return tomlkit.item(value).as_string()

src/paperpi/web/config_file.py

@@ -4,11 +4,11 @@
   file's permissions (and owner, when run as root). Hold :data:`LOCK` from reading to
   saving, so two changes at the same moment (a plugin and the password) can't undo each
   other. :func:`saved_here` tells whether a text is the one the web interface saved last.
-- The plugin changes (:func:`move`, :func:`remove`, :func:`set_enabled`, :func:`add`) work on
-  the file's text, one ``[[plugin]]`` block at a time. The comment lines just above a
-  ``[[plugin]]`` line (with a blank line above them) belong to that block: they move with it
-  and are removed with it. A comment on the same line as ``enabled =`` is lost when the
-  plugin is switched on or off.
+- The plugin changes (:func:`move`, :func:`remove`, :func:`set_enabled`, :func:`set_settings`,
+  :func:`add`) work on the file's text, one ``[[plugin]]`` block at a time. The comment lines
+  just above a ``[[plugin]]`` line (with a blank line above them) belong to that block: they
+  move with it and are removed with it. A comment at the end of a setting's line (such as
+  ``enabled =``) is lost when that setting is changed.
 - Every change is checked before it is saved: PaperPi must read the new text back as exactly
   the old settings plus the change. A file written in a way these functions don't
   understand (for example a ``[[plugin]]`` line written differently, or a value line
@@ -25,11 +25,14 @@
 import stat
 import threading
 import tomllib
+from collections.abc import Mapping
 from dataclasses import dataclass
 from pathlib import Path
 from typing import Any
 
-from .. import config, limits
+import tomlkit
+
+from .. import config, example, limits
 from ..files import write_atomic
 
 #: Held from reading the config file to saving the changed copy.
@@ -254,6 +257,119 @@ def set_enabled(text: str, index: int, plugin_id: str, enabled: bool) -> str:
     return _checked("".join(new), data | {"plugin": expected})
 
 
+def set_settings(
+    text: str,
+    index: int,
+    plugin_id: str,
+    values: Mapping[str, Any],
+    defaults: Mapping[str, Any] | None = None,
+) -> str:
+    """``text`` with settings of plugin block ``index`` changed: ``values`` maps a setting
+    to its new value, or to ``None`` to remove it (back to its default).
+
+    - A setting that is in the block gets its new value on the same line (a comment at the
+      end of that line is lost).
+    - A new one replaces a ``# key = ...`` comment line, so the help text above it stays.
+      Without one, ``name`` goes below ``id`` and the others below the block's last setting.
+    - A removed setting with a value in ``defaults`` becomes the comment ``# key = default``
+      (``# key =`` for ``None``), as in the example config; others are taken out.
+    - A setting written over several lines can't be changed: :class:`UnknownForm`.
+    """
+    defaults = defaults or {}
+    data, scan = _check(text, index, plugin_id)
+    block = scan.blocks[index]
+    lines = _ended(scan.lines)
+    newline = _newline(lines)
+    own = [n for n in range(block.header + 1, block.own_end) if n not in scan.in_string]
+    expected = plugin_blocks(data)
+    changed = dict(expected[index])
+    new: dict[int, str | None] = {}  # line -> its new text (None: taken out)
+    added: list[tuple[str, str]] = []  # (setting, line) with no place in the block yet
+    for key, value in values.items():
+        at = _setting_line(lines, own, re.escape(key))
+        if at is not None and not _one_line(lines[at]):
+            raise UnknownForm()
+        commented = _commented_setting(lines, own, key)
+        if value is None:
+            changed.pop(key, None)
+            if at is not None:
+                # Back to "# key = default", unless the block has that comment already.
+                new[at] = None if commented is not None else _default_line(key, defaults, newline)
+            continue
+        changed[key] = example.plain(value)
+        line = f"{_key(key)} = {example.toml_value(value)}{newline}"
+        if at is None:
+            at = commented  # a commented-out setting, so its help text above stays
+        if at is None:
+            added.append((key, line))
+        else:
+            new[at] = line
+
+    def setting_after(number: int) -> bool:
+        line = new.get(number, lines[number])
+        return line is not None and bool(line.strip()) and not line.lstrip().startswith("#")
+
+    last = max((n for n in own if setting_after(n)), default=block.header)
+    below: dict[int, list[str]] = {}  # line -> new lines to put below it
+    for key, line in added:
+        at = _setting_line(lines, own, "id") if key == "name" else None
+        at = _value_end(lines, last if at is None else at, block.own_end)
+        below.setdefault(at, []).append(line)
+    result = []
+    for number, line in enumerate(lines):
+        replaced = new.get(number, line)
+        if replaced is not None:
+            result.append(replaced)
+        result += below.get(number, [])
+    expected[index] = changed
+    return _checked("".join(result), data | {"plugin": expected})
+
+
+def _value_end(lines: list[str], line: int, end: int) -> int:
+    """The last line of the setting that starts on ``line``: a list or a text written over
+    several lines ends further down. ``line`` itself when it is not a setting."""
+    if not _HEADER.fullmatch(lines[line].rstrip("\r\n")) and not _one_line(lines[line]):
+        for last in range(line + 1, end):
+            if _one_line("".join(lines[line : last + 1])):
+                return last
+    return line
+
+
+def _commented_setting(lines: list[str], within: list[int], key: str) -> int | None:
+    """The line of a commented-out ``key``: ``# key = value`` (a setting without the ``#``)
+    or ``# key =`` (not set), not a help text that starts the same way."""
+    for number in within:
+        line = lines[number].strip()
+        if not re.match(rf"#\s*{re.escape(key)}\s*=", line):
+            continue
+        if _one_line(line[1:]) or re.fullmatch(rf"#\s*{re.escape(key)}\s*=", line):
+            return number
+    return None
+
+
+def _key(key: str) -> str:
+    """``key`` as written in a TOML file (in quotes when it needs them)."""
+    return tomlkit.key(key).as_string()
+
+
+def _default_line(key: str, defaults: Mapping[str, Any], newline: str) -> str | None:
+    """The comment that takes the place of a removed setting, or ``None`` for none."""
+    if key not in defaults:
+        return None
+    default = defaults[key]
+    shown = "" if default is None else f" {example.toml_value(default)}"
+    return f"# {_key(key)} ={shown}{newline}"
+
+
+def _one_line(line: str) -> bool:
+    """True when the setting on ``line`` is complete on that line."""
+    try:
+        tomllib.loads(line)
+    except tomllib.TOMLDecodeError:
+        return False
+    return True
+
+
 def add(text: str, block_text: str) -> str:
     """``text`` with ``block_text`` (one ``[[plugin]]`` block, as written by
     :func:`paperpi.example.plugin_block`) added after the last plugin block."""