Wave 20: a handback that only claims to wait on background work gets a bounded block

.claude/skills/agentic-delivery/scripts/handback_cap.py

@@ -65,7 +65,9 @@
 skip it.
 
 BACKGROUND WORK (policy `background_guard`: `off|warn|block`, default `warn`; env PERUN_BACKGROUND_GUARD overrides). A lane
-waits for background work from its worktree until it exits. When the lane's cwd is inside a LINKED worktree and a test or build runner
+waits for background work from its worktree until it exits. The switch sets what a live test or build and a dirty worktree get
+(`warn` a note, `block` exit 2); a handback that only claims to be waiting gets a bounded block in every mode but `off`
+(below). When the lane's cwd is inside a LINKED worktree and a test or build runner
 (BG_RE: pytest, jest, vitest, playwright, make, `npm test`, `cargo test`, `go build`, a `*test*.py|sh` script, ...) has its
 cwd inside that worktree, `warn` sends the owner a `systemMessage` naming the pid and command, and `block` exits 2 with the
 same text on stderr (released after PUSH_MAX_S, like a live `git push` or `git commit`: PUSH_RE matches both for a process whose argv[0] is `git`, `-C`/`-c`
@@ -87,8 +89,15 @@
 has one), it sits under the lane root, is clean including ignored files, equals its upstream, and no test/build runs from
 it (a blind scan only reports). Other trees and the main checkout are never touched; any doubt keeps the tree and says so
 on stderr. Full rule list: `_reap_own_worktree`.
-When the scan finds nothing but the handback text itself says the lane is "waiting on" its background tests (WAIT_RE),
-a `systemMessage` flags it in every mode but `off`; never a block (the text is a claim, not evidence).
+When the scan finds nothing but the handback text itself says the lane is "waiting on" its background tests (WAIT_RE), the
+stop is blocked (exit 2) in every mode but `off`, because a lane that stops now ends with its work unverified. stderr says to
+wait in this turn (a local process: `lane_guard.py wait --pid <pid> --timeout <s>`) and hand back the result, or to hand back
+status=UNVERIFIED naming what is still pending (for example a remote check). The text is a claim, not evidence, so the block
+is bounded (`_wait_block`): none when the input has no agent_id (the counter has nothing to key on), none while it has
+`stop_hook_active` true, none when the first status line of the handback (its first STATUS_RE match) reads UNVERIFIED (it
+already says the work is unverified, so the way out above cannot loop; a later UNVERIFIED line does not exempt it), and at most
+MAX_BLOCKS per agent_id (its own `.wait` counter file beside the block counter). Where no block applies a `systemMessage` flags
+the claim and the stop goes through. A hold above (push or commit, background, dirty) runs first and wins.
 
 NARRATION NUDGE: when a non-exempt lane wrote at least HANDBACK_NARR_NUDGE (default 100, 0 = off) estimated tokens of
 prose between tool calls, a `systemMessage` names the amount and the per-call `narr_rate`. Detect-and-measure after the
@@ -301,7 +310,7 @@ def _skip_pids(table, me):
     return skip | kids
 
 
-# The lane's own words: it hands back "waiting on" its background tests. Advisory only (see check()).
+# The lane's own words: it hands back "waiting on" its background tests. A claim, not evidence: a bounded block (see _wait_block).
 _WAIT_KW = (r"(?:background|bg|tests?|test\s+(?:suite|run)|suites?|builds?|pytest|playwright|e2e|gates?|pre-push|ci|checks?"
             r"|lint|workflows?|actions|jobs?|runs?)")
 WAIT_RE = re.compile(
@@ -322,8 +331,8 @@ def _waiting_claim(msg):
     """True when a counted line of `msg` (fences and '! ' lines excluded, as for the cap) claims the lane is waiting
     on background work. A match whose own clause (the text back to the previous punctuation, and the next five words
     after it) carries a negation ("not waiting on tests", "wait for CI not needed") or names a person ("waiting on
-    owner to run tests") does not count. A heuristic: it misses wordings it has no keyword for, which is why it only
-    advises."""
+    owner to run tests") does not count. A heuristic: it misses wordings it has no keyword for, and a wrong hit costs
+    one bounded block (`_wait_block`)."""
     for ln in _counted(msg):
         for m in WAIT_RE.finditer(ln):
             head = _CLAUSE_END.split(ln[:m.start()])[-1]
@@ -334,6 +343,30 @@ def _waiting_claim(msg):
     return False
 
 
+def _wait_block(data):
+    """True when this stop is blocked for a handback that claims to wait on background work. Not when `data` has no
+    agent_id (the counter has nothing to key on), not while it has `stop_hook_active` (a stop hook already sent this stop
+    back), not when the first status line of the handback (its first STATUS_RE match) reads UNVERIFIED (it already says the
+    work is unverified, so the way out in the block text cannot loop; a later UNVERIFIED line does not exempt it), and at
+    most MAX_BLOCKS times per agent_id, counted in its own `.wait` file beside `_state_path` (the shape-block counter is
+    separate). Side effect: the counter file. A state file that cannot be written means the bound cannot be kept: no block."""
+    aid = data.get("agent_id")
+    first = STATUS_RE.search(data.get("last_assistant_message") or "")
+    if not aid or data.get("stop_hook_active") or (first and "unverified" in first.group(0).lower()):
+        return False
+    path = _state_path(str(aid)) + ".wait"
+    n = _read_blocks(path)
+    if n >= MAX_BLOCKS:
+        return False
+    try:
+        os.makedirs(STATE_DIR, exist_ok=True)
+        with open(path, "w") as fh:
+            fh.write(str(n + 1))
+    except OSError:
+        return False
+    return True
+
+
 def _bg_found(table, pids, me):
     """[(pid, command)] of test/build processes among `pids`. `table` maps pid -> (ppid, command). Never counts
     `_skip_pids(table, me)`."""
@@ -492,7 +525,8 @@ def check(data):
     """Return the hook exit code (0 pass, 2 block) for one hook-input dict.
     The push or commit, background and dirty-worktree checks are all evaluated before it returns 2, so each one that finds
     no condition resets its hold's window even while another hold blocks (`_release`; a check that could not run, None,
-    keeps it). The first hold that blocks wins, in that order, and a lower hold is never started while a higher one blocks."""
+    keeps it). The first hold that blocks wins, in that order, and a lower hold is never started while a higher one blocks.
+    After the holds, a handback that claims to wait on background work is blocked, within the bounds of `_wait_block`."""
     if str(data.get("agent_type") or "") in _exempt_types():
         return 0
     ro = READONLY_RE.search(str(data.get("agent_type") or ""))  # a read-only type never owns a commit or push: not held, not scanned
@@ -541,6 +575,14 @@ def check(data):
         return 2
     msg = data.get("last_assistant_message") or ""
     if not bg and mode != "off" and _waiting_claim(msg):  # scan saw nothing (or was blind) but the lane says it is waiting
+        if _wait_block(data):
+            sys.stderr.write(
+                "Handback blocked: it says this lane is waiting on background work, and a lane that stops now ends with its work unverified. "
+                "Wait in this turn (a local process: lane_guard.py wait --pid <pid> --timeout <s>) and hand back the result, or hand back "
+                "status=UNVERIFIED naming what is still pending (for example a remote check). "
+                f"This block is bounded: it does not repeat once a stop hook has sent this stop back, and stops after {MAX_BLOCKS} per agent id.\n"
+            )
+            return 2
         notes.append("background_guard: this lane's handback says it is waiting on background work; it must wait in its own turn (lane_guard.py wait) and hand back the result.")
     if notes:  # one JSON line however many findings
         _SENT.append(1)
@@ -789,7 +831,7 @@ def main():
 def _selftest():
     import io
     import shutil
-    from contextlib import redirect_stderr
+    from contextlib import redirect_stderr, redirect_stdout
 
     global STATE_DIR
     tmp = tempfile.mkdtemp(prefix="handback-cap-selftest-")
@@ -969,6 +1011,50 @@ def cd(c):
     case("git-push-x-does-not-match", bool(PUSH_RE.search("git push-x origin")), False)
     case("git-log-grep-commit-does-not-match", bool(PUSH_RE.search("git log --grep commit")), False)
 
+    # a handback that claims to wait on background work is blocked, bounded (holds above win and never reach this)
+    wait_msg = "Gate running in background; waiting on completion."
+    saved_guard = os.environ.get("PERUN_BACKGROUND_GUARD")
+
+    def wait_check(mode, aid, msg=wait_msg, **extra):  # (rc, stdout, stderr) of one check() in background_guard mode `mode`
+        os.environ["PERUN_BACKGROUND_GUARD"] = mode
+        out, err = io.StringIO(), io.StringIO()
+        with redirect_stdout(out), redirect_stderr(err):
+            rc = check({"last_assistant_message": msg, **({"agent_id": aid} if aid else {}), **extra})
+        return rc, out.getvalue(), err.getvalue()
+    try:
+        for mode in ("warn", "block"):
+            case(f"wait-claim-blocks-in-{mode}-mode", wait_check(mode, f"wc-{mode}")[0], 2)
+        case("wait-claim-block-names-the-way-out",
+             [s in wait_check("warn", "wc-text")[2] for s in ("lane_guard.py wait --pid <pid> --timeout <s>", "status=UNVERIFIED")], [True, True])
+        case("wait-claim-with-stop-hook-active-passes-with-the-note",
+             [(r := wait_check("warn", "wc-active", stop_hook_active=True))[0], "waiting on background work" in r[1]], [0, True])
+        case("wait-claim-block-releases-after-max-blocks",
+             [wait_check("warn", "wc-max")[0] for _ in range(MAX_BLOCKS + 1)], [2] * MAX_BLOCKS + [0])
+        remote = "Waiting on CI for PR 1234; local gates green."
+        case("wait-claim-on-a-remote-check-is-blocked-and-names-unverified",
+             [(r := wait_check("warn", "wc-remote", remote))[0], "status=UNVERIFIED" in r[2]], [2, True])
+        case("wait-claim-without-an-agent-id-keeps-the-note",
+             [(r := wait_check("warn", None, remote))[0], "waiting on background work" in r[1], r[2]], [0, True, ""])
+        case("unverified-handback-naming-a-wait-is-never-blocked",
+             wait_check("warn", "wc-unv", "status=UNVERIFIED sha=abc1234 result=pushed tests=3/3 blockers=waiting on CI for PR 1234")[0], 0)
+        verified = "status=VERIFIED sha=abc1234 result=done tests=3/3"  # the first status line decides, not a later UNVERIFIED line
+        case("wait-claim-after-a-verified-status-is-blocked-despite-a-later-unverified-line",
+             wait_check("warn", "wc-late-line", verified + "\nUNVERIFIED: waiting on CI for PR 1234")[0], 2)
+        case("wait-claim-after-a-verified-status-is-blocked-despite-a-later-unverified-field",
+             wait_check("warn", "wc-late-field", verified + "\n- unverified: lint\nWaiting on CI for PR 1234.")[0], 2)
+        case("wait-claim-after-an-unverified-first-status-passes-with-the-note",
+             [(r := wait_check("warn", "wc-first-unv", "status=UNVERIFIED sha=abc1234 result=pushed tests=3/3\nWaiting on CI for PR 1234."))[0],
+              "waiting on background work" in r[1]], [0, True])
+        case("wait-claim-mode-off-is-silent", wait_check("off", "wc-off")[:2], (0, ""))
+        case("verified-handback-is-silent",
+             wait_check("warn", "wc-ok", "status=VERIFIED sha=abc1234 result=done tests=3/3 blockers=none")[:2], (0, ""))
+    finally:
+        if saved_guard is None:
+            os.environ.pop("PERUN_BACKGROUND_GUARD", None)
+        else:
+            os.environ["PERUN_BACKGROUND_GUARD"] = saved_guard
+        _SENT.clear()  # the notes above printed a systemMessage; the narration nudge cases below need a clean slate
+
     # short message passes
     case("short-passes", check({"agent_id": "a1", "last_assistant_message": "VERIFIED | verdict=ok"}), 0)
 

.claude/skills/agentic-delivery/scripts/perun_policy.py

@@ -28,7 +28,8 @@
 `wip_cap` takes an integer >= 0 (default 0 = off, opt-in): the same hook adds a "WIP: N open PRs > cap M" warn line when more non-draft PRs are open.
 `ready_age_p50_h` and `ready_age_p90_h` (hours; the watchdog warns when green PRs wait longer), `merges_per_hour_target` and `train_yield_floor` take an integer >= 0 (default 0 = off, opt-in): the same hook warns when merges per hour on the default branch fall below the target while green PRs wait, and when PRs per landed train (merges landed back-to-back) fall below the floor. `lkg_gate` (`shadow|enforce|off`, default `shadow`): land-release.sh warns (shadow) or refuses (enforce) to cut a release from a commit that is not the last-known-good recorded by lkg.py (the newest commit where the full ci-tests suite passed; otherwise the untested gap is reported); `lkg_stale_h` (hours >= 0, default 0 = off) makes the watchdog warn when that pointer is older. `red_uncovered_max` and `red_oldest_h` (integers >= 0, 0 = off) are the red_main.py thresholds: reds with no owner or fix, and the age in hours of the oldest red. `lane_idle_h` (hours >= 0, 0 = off) with `lane_globs` ({lane: [fnmatch globs]}) makes the watchdog warn when a lane has ready PRs and its last merge is older than `lane_idle_h`.
 `background_guard` takes `off|warn|block` (default `warn`): whether handback_cap.py (SubagentStop) warns or blocks when a lane hands back
-while a test or build started from its worktree is still running; env `PERUN_BACKGROUND_GUARD` overrides it.
+while a test or build started from its worktree is still running; env `PERUN_BACKGROUND_GUARD` overrides it. A handback that only
+says it is waiting on background work gets a bounded block in every mode but `off` (it must wait in the same turn, or hand back status=UNVERIFIED).
 `auto_compact` takes `on|off` (default `on`): whether `install.sh --apply-operating-layer` may set `autoCompactWindow: 200000`
 (`off` skips it; a value the user already set is never overridden either way). `statusline` takes `on|off` (default `on`): the same
 for the cost `statusLine` (cost_statusline.py). `spend_nudge` takes `on|off` (default `on`): whether session_brief.py (SessionStart) prints the one-line spend nudge from `token_report.py --nudge`. `release_every` takes an integer >= 1 (default 1): `land-release.sh publish` releases only when the minor is a multiple of it.

.claude/skills/agentic-delivery/templates/lane-preamble.md

@@ -36,7 +36,7 @@ default for this lane, not a suggestion.
 - Before running tests from a hook, scrub inherited env: `scripts/scrub_env.sh <cmd>` unsets GIT_DIR, GIT_WORK_TREE, GIT_INDEX_FILE and every other GIT_* var plus DB vars (DATABASE_URL, PG*, MYSQL_*, REDIS_URL); a test's `git init` that inherited GIT_DIR once flipped the host repo to core.bare=true. It also unsets the host's systemd/session env (DBUS_*, XDG_RUNTIME_DIR); `scripts/hermetic_env_lint.py` fails a test of a host-reading script that pins neither.
 - NO POLLING. Never hand-roll a sleep/loop waiting on background work; use the bounded wait below.
 - Never run delete/kill experiments on the host (container/VM or skip) and never `rm -rf` a variable-built path; deny rules are text-only, the sandbox is the boundary (`operating-discipline.md` item 8).
-- A lane owns its background work until it completes: never hand back while its own `git push` or `git commit` (a pre-push or pre-commit hook can run 15+ minutes), test run or build is still running; `handback_cap.py` blocks a handback while a push or commit from this worktree is live and warns (policy `background_guard=block` to block) on a live test or build. Never end your turn with background jobs running (browser tests, builds; kill your own dev servers): `python3 .claude/skills/agentic-delivery/scripts/lane_guard.py wait --pid <pid> --port <port> --file <output> --timeout <s> --progress <worktree>/.lane-progress && lane_guard.py handback …`; `--progress` rewrites one liveness line every 15 min (`--progress-every`), so the parent reads that file instead of waiting on interim notices. `COULD_NOT_CHECK` → hand back what is still running, not "waiting"; a handback that says it is waiting on its tests gets an advisory flag even when the scan finds nothing.
+- A lane owns its background work until it completes: never hand back while its own `git push` or `git commit` (a pre-push or pre-commit hook can run 15+ minutes), test run or build is still running; `handback_cap.py` blocks a handback while a push or commit from this worktree is live and warns (policy `background_guard=block` to block) on a live test or build. Never end your turn with background jobs running (browser tests, builds; kill your own dev servers): `python3 .claude/skills/agentic-delivery/scripts/lane_guard.py wait --pid <pid> --port <port> --file <output> --timeout <s> --progress <worktree>/.lane-progress && lane_guard.py handback …`; `--progress` rewrites one liveness line every 15 min (`--progress-every`), so the parent reads that file instead of waiting on interim notices. `COULD_NOT_CHECK` → hand back what is still running, not "waiting"; a handback that says it is waiting on its tests is blocked (bounded) even when the scan finds nothing; wait in the same turn with `lane_guard.py wait`, or hand back status=UNVERIFIED naming what remains.
 - Final message is `key=value` fields only, one per line, no sentences, under 400 characters and 8 lines: `status=VERIFIED|UNVERIFIED`, `sha=`, `result=`, `tests=n/n`, `blockers=`, `report=<path>` for any detail. `handback_cap.py` only logs a miss, it never sends you back. Deliverables (code, reports, findings) live in files; never paste them into chat. If findings do not fit in the fields, write them to a file and hand back its path. A lane that ran long or touched several things always adds `report=<details-file path>`; the parent reads the file instead of re-querying state.
 - Push, `gh` or network fails inside the sandbox (TLS x509, connection refused, no route): do not retry or work around it. Commit, hand back branch plus sha, and the main thread pushes.
 - Board posts only through `python3 .claude/skills/agentic-delivery/scripts/board_post.py --repo <owner/name> --issue <issue> --type <CLAIM|RELEASE|HANDOFF|BLOCKER> ...` — typed posts, never free-form chat to a shared board.

changelog.d/handback-wait-block.md

@@ -0,0 +1,5 @@
+### Fixed
+- A handback that says the lane is waiting on its background tests now blocks the stop (exit 2) in every mode but `off`, including the default `warn`, instead of only adding a `systemMessage` and letting the lane end with its work unverified. The reason on stderr tells the lane to wait in this turn (a local process: `lane_guard.py wait --pid <pid> --timeout <s>`) and hand back the result, or to hand back `status=UNVERIFIED` naming what is still pending (for example a remote check). The text is a claim, not evidence, so the block is bounded: it does not fire when the hook input has no `agent_id` (the counter has nothing to key on, so the note stays), when it has `stop_hook_active` true, or when the first status line of the handback reads UNVERIFIED (so the way out above cannot loop; a later `UNVERIFIED:` line or `- unverified:` field does not exempt a handback whose first status line is VERIFIED); otherwise it fires at most `MAX_BLOCKS` (3) times per agent id, counted in a `.wait` file beside the lane's other state files, and a state file that cannot be written means no block. Where no block applies the `systemMessage` stays and the stop goes through. The push, commit, background and dirty-worktree holds still run first and win. The `background_guard` policy text, the `handback_cap.py` docstring and the lane preamble now say so.
+
+### Changed
+- Tests: `python3 .claude/skills/agentic-delivery/scripts/handback_cap.py --selftest` (already run by CI) goes from 117 to 130 cases. Two of the new cases pin the first-status-line rule: a wait claim after a VERIFIED first status line is still blocked when a later line reads `UNVERIFIED: ...` or `- unverified: ...`. The rest pin the block text, the bound, the no-`agent_id`, `stop_hook_active`, UNVERIFIED-first, mode `off` and verified-handback paths. `scripts/test_bg_wait_claim.py` now runs its flag cases with `stop_hook_active` true, the bound past which a waiting claim is only flagged.