fix(tools): require committed live evidence for upstream syncs

PORTING.md

@@ -211,12 +211,14 @@ This section is the only sync procedure. A runner gives it three inputs: the rec
 
 The manual-dispatch GitHub Action in `.github/workflows/upstream-sync.yml` is one runner. It needs the `CLAUDE_CODE_OAUTH_TOKEN` repository secret. Its checkout has `.no-mistakes.yaml` but no initialized gate, so it prepares the sync and reports publication as blocked. It does not push a branch or open a PR with its workflow token. A maintainer resumes the sync in an initialized checkout and publishes through the configured gate. The Action checkout is temporary; its local commits are not a durable handoff. Host-specific schedules, machine paths, and maintainer-specific tooling are intentionally not documented in this repository.
 
-Run `tools/upstream-sync.sh` for the local scheduled runner. It requires Git, `gh`, Python 3, and authenticated Claude Code. It checks upstream and open sync PRs before invoking Claude. Each attempt starts with a fresh clone; attempts that invoke Claude get a unique branch and retain their clone. `PSTACK_SYNC_STATE_DIR` overrides the default `~/.local/state/pstack-sync` directory. The directory contains `sync.log` and an `attempt-*` directory for each run. An attempt that invokes Claude retains `repo/`, `prompt.txt`, `upstream-sha`, and `claude.log`.
+Run `tools/upstream-sync.sh` for the local scheduled runner. It requires Git, `gh`, Python 3, and authenticated Claude Code. It checks upstream and open sync PRs before invoking Claude. Each attempt starts with a fresh clone; attempts that invoke Claude get a unique branch and retain their clone. The runner pins the clone's initial commit before Claude runs and saves it in `base-sha`. `PSTACK_SYNC_STATE_DIR` overrides the default `~/.local/state/pstack-sync` directory. The directory contains `sync.log` and an `attempt-*` directory for each run. An attempt that invokes Claude retains `repo/`, `base-sha`, `prompt.txt`, `upstream-sha`, and `claude.log`.
 
 The local runner records the user's explicit authorization to bypass no-mistakes only to publish that attempt's sync branch and create its draft PR. This exception applies only to that runner's publication in step 10. All local checks remain required. It does not authorize marking the PR ready, merging, pushing to `main`, or force-pushing. The Action and other publication routes keep their existing gate requirements.
 
 Claude's exit code alone does not establish success. The runner requires clean committed work, the intended upstream SHA in the commit, and a live open draft PR to `main`. The PR must belong to this repository and match the attempt's branch and commit SHA. A failure after Claude starts retains the clone and exits nonzero. Unchanged-upstream skips, open-PR skips, and blocks caused by a prior unfinished attempt remove only the new untouched clone and retain the attempt directory. An unfinished attempt blocks another Claude run after the unchanged-upstream and open-PR checks. To recover, inspect `sync.log`, resume the retained `repo/` using `prompt.txt`, and resolve its work. Then move the whole resolved attempt outside the state directory before retrying. Keep that copy until its work is safe. Successful attempts remain retained with a `complete` file containing the verified PR URL. Harmless skips also have a `complete` file that records their reason.
 
+Before checking the PR, the runner independently reads the committed promise ledgers at the pinned initial commit and candidate head. Missing or malformed ledger data blocks completion. It takes live owner paths from both ledgers and compares changed paths without rename detection. A changed or deleted owner still triggers the check after its promise is removed. A changed live owner requires an added or updated `.md` file under `evals/promises/results/` with nonempty committed contents. An unchanged report, a mode-only change, or a deleted report does not qualify. When no live owner changes, the runner requires no new report. A missing-report refusal names the changed promises and owners, retains the work, and leaves the attempt incomplete. This check proves that the sync committed a report. Reviewers still assess the report's truth, case coverage, and upstream prose fidelity. The check does not establish that every case passed.
+
 1. Add the `upstream` remote `https://github.com/cursor/plugins.git` if the checkout lacks it, and run `git fetch upstream main`. Without a runner, the recorded commit is the SHA in `.github/upstream-sha` (originally the import commit), and the latest commit is `git rev-list -1 upstream/main -- pstack`. When the two match, there is nothing to sync.
 2. Study `git log <recorded>..upstream/main -- pstack/` and `git diff <recorded>..upstream/main -- pstack/`. Map upstream `pstack/skills/<x>` to `skills/<x>`, except `tdd` to `poteto-tdd` and `teach` to `poteto-teach`. Map upstream docs to this repo's root files where they exist.
 3. Apply each incoming hunk with the substitution map. Merge it into the port's file and never overwrite the file, so the port additions survive: GitNexus nudges, pstack-harness pointers, `agents/openai.yaml` files, and renamed skill references. Apply the map's `disable-model-invocation` row to incoming frontmatter and its sibling-skill-load row to every incoming load of another skill. A passage that quotes what the user types goes in `USER_FACING` in `tools/skill-references.py`. Every SKILL.md frontmatter `name:` must equal its directory name. Every port addition and every "on sync" rule elsewhere in this file binds this step.
@@ -234,7 +236,7 @@ Claude's exit code alone does not establish success. The runner requires clean c
    - `tools/gate-check.sh test`. It needs Bun. `tools/gate-check.sh` picks its own Python, and `PSTACK_GATE_PYTHON` overrides it, as `README.md` describes. On a host with mise shims on `PATH`, put the real `bun` and `rg` ahead of the shims, and say in the PR body which tools ran. #166 tracks one pinned entry point per test tier.
    - `python3 evals/promises/ledger.py audit --upstream-ref upstream/main`. An "unclassified upstream unit" is a guide sentence that upstream added or reworded. An unclassified or stale port unit is guide text that this sync changed, such as a count or a link from step 5. Classify it in `evals/promises/ledger.json` per `evals/promises/README.md`, and bind each new promise to a quote, a test, or a case. Port the sentence into `docs/guide` with the substitution map, unless you classify it `cursor-only` with its reason or `substituted` with the port units that replace it. A "quote no longer in" line names a guide promise whose skill text the sync rewrote. Keep the promise true, change the guide and the ledger together, or record the change for human judgment in the PR body. A quote recorded for human judgment can leave other checks failing that `main` passes. These are the lint tier's `promise ledger` check and the `AuditTest` cases in `evals/promises/test_ledger.py`, such as `test_the_committed_ledger_audits_clean`. These local failures may remain only when the complete listed set of held quotes explains every remaining cause. List each quote and each affected check in the PR body. Any unrelated failure blocks this local exception. Recording the hold waives neither authoritative CI nor no-mistakes Lint. Keep gated publication pending when the configured gate cannot accept the gap through its supported authority. Do not bypass or weaken checks. When you reword a guide sentence that ports an upstream unit, point that unit's `ported_as` at the new unit. List every ledger change in the PR body under "Guide promises".
    - `skill-ci check`
-8. After a sync that changes a live promise's owner files, rerun that promise's cases on all four harnesses with `evals/promises/live.py run`. When the runner lacks a harness, list the cases that did not run in the PR body as needing judgment. Render the matrix with `live.py report`, save it as `evals/promises/results/<date>.md`, and compare it with the newest earlier file there, if one exists. Read that file's "Comparison baseline" section first. `2026-10-03.md` measures the tree before its own fixes, so compare against its "Fixed in this stack" rows, and run a full screen on the current tree before treating any other row as a baseline.
+8. After a sync that changes a live promise's owner files, rerun that promise's cases on all four harnesses with `evals/promises/live.py run`. When the runner lacks a harness, list the cases that did not run in the report and PR body as needing judgment. Render the matrix with `live.py report` and save it as `evals/promises/results/<date>.md`. Record the final skills revision evaluated, the matrix, the baseline comparison, and explicit judgments for missing cases or harnesses. These are report contents, not a fixed format; the local runner checks report presence only, not format or meaning. Save and commit the report even when some cases cannot run. Compare it with the newest earlier file there, if one exists, or state that no earlier baseline exists. Read that file's "Comparison baseline" section first. `2026-10-03.md` measures the tree before its own fixes, so compare against its "Fixed in this stack" rows, and run a full screen on the current tree before treating any other row as a baseline.
 9. Write the latest upstream commit to `.github/upstream-sha`.
 10. Create the runner-provided branch, or `sync/upstream-<YYYYMMDD>` without one, run `git add -A`, and commit everything with a message that summarizes the upstream commits ported. A ledger classification from step 7 goes in the same commit. For `tools/upstream-sync.sh`, use the runner-provided branch and the scheduled draft-only publication exception described above. For every other route, resolve publication through **Gated publish** in `skills/poteto-mode/playbooks/opening-a-pr.md`. Except for that local scheduled runner, a runner route never bypasses a configured gate. For those other routes, that section owns publication and PR readiness in a gated checkout. When their configured gate is missing, uninitialized, or broken, stop before publication and report the upstream commits, local branch and head, checks, and required gated handoff. The Action follows this stop path; a workflow token is not a gate exemption. On the scheduled runner's authorized route or a supported ungated path, publish a draft PR to `main` through the runner's route. Without a route, push the branch with `git push -u origin <branch>`, then run `gh pr create --draft --repo github.com/mdsmithaustin/pstack`. The explicit host and repository keep `gh` from selecting cursor/plugins through the `upstream` remote. Every published outcome requires a PR, because the `main` ruleset grants no bypass. Never push to `main` and never force-push.
 11. Prepare the PR body, or include the same information in the report when publication is blocked. List each upstream commit, the transformation applied to it, the checks that ran, and anything that needs human judgment. When you are unsure whether a change is portable, include it with a "needs judgment" note instead of dropping it. When the only change is the `.github/upstream-sha` bump, say that nothing else was portable. Report only CI checks that actually ran; local checks do not establish that the Docker suites or other CI-only checks passed.

evals/promises/results/2026-10-20-sync-88d47514.md

@@ -0,0 +1,46 @@
+# Sharpen sync verification, 2026-10-20
+
+## Examined revision
+
+[Port PR #278](https://github.com/mdsmithaustin/pstack/pull/278) ports upstream [#729](https://github.com/cursor/plugins/pull/729), commit `88d47514b82e7c44f00f2b2a72ff6dd12158038c`. The tested portable skills revision is `bf97cceece38422b281db97a403723d050bddd1c`.
+
+This receipt records the checks that ran for that revision. It does not claim that the port has merged or that another revision has been tested. The results were initially retained outside Git and are committed here to close that reporting gap.
+
+The eight canonical help grades and three Codex scenarios at `bf97cceece38422b281db97a403723d050bddd1c` are historical evidence from before the later local sharpen boundary fixes. The [final sharpen validation receipt](2026-10-20T1935-sharpen-729-final.md) is the current sharpen validation record, tested at `815e15cbfbaba27eade02a1b1f91270b0588a63b` after both fixes. The outcomes and PR status below describe the original run.
+
+## Changed live owner
+
+The port changes `skills/poteto-help/SKILL.md`, an owner of `poteto-help-finds-need-and-routes`. Both bound cases ran on every supported harness against the committed skills revision above. No case or harness was unavailable.
+
+| Case | Claude Code | Codex | Hermes | Grok |
+|---|---|---|---|---|
+| `poteto-help-run` | PASS | PASS | PASS | PASS |
+| `poteto-help-unclear-run` | PASS | PASS | PASS | PASS |
+
+The first case checks branch-review routing and a copyable prompt. The second checks a clarifying question for an unclear request. These are routing results. They do not establish `/sharpen` compliance on all four harnesses.
+
+The matrix came from `evals/promises/live.py run` with both cases, each harness, and `--skills-at bf97cceece38422b281db97a403723d050bddd1c`. `live.py report` returned eight PASS verdicts for the promise. Nonzero command exit or a non-PASS applicable verdict is a failure signal.
+
+## Direct sharpen scenarios
+
+The no-mistakes Test phase also ran three native Codex scenarios against the same skill revision. Each passed:
+
+- A prompt-rewrite request returned the four framing answers and a rewritten prompt.
+- A request containing an instruction to write `onboarding.md` left that file unwritten. Its rewritten prompt left the database, language, and hosting choices open.
+- A rewrite-only help request received a `/sharpen` recommendation and a copyable prompt.
+
+The gate retained actual replies, grades, project-state checks, an HTML matrix, and a repeatable scenario driver. Inspect them through `no-mistakes axi logs --run 01M5CNXT0E1AT5BF19ADNK351A --step test --full`. A nonzero exit or unavailable evidence is a verification gap. The recorded Test verdict is `go`, with no findings. These direct scenarios ran on Codex only.
+
+## Comparison baseline
+
+The previous sync receipt is [2026-10-20-sync-7a9a3ddb.md](2026-10-20-sync-7a9a3ddb.md). Its `poteto-help-finds-need-and-routes` row records PASS (2) on all four harnesses. This rerun matches that row with both cases passing on every harness. No regression appeared in those two cases.
+
+The previous receipt's comparison section establishes that wave as the baseline for the next sync and lists its known failures and unobserved behavior. This focused rerun does not close unrelated evaluation debt listed in earlier receipts.
+
+## Other verification
+
+The upstream-aware ledger audit returned zero problems. `mise run gate-lint`, `mise run gate-test`, type checks, and `skill-ci check` exited zero. The Python suites ran 248 tools tests, 822 promise tests, and 185 setup tests. Bun reported 541 passed, one skipped, and zero failed. Each check fails on nonzero exit; the suite summaries must also show executed tests.
+
+A different-agent documentation review passed on the six upstream prose changes and portable metadata. The reviewed diff fingerprint is SHA-256 `185a20e1bcb9ca30ba1f93788499a88403c7b6ca538eef5f4a14edf247af2228`.
+
+All seven GitHub checks passed on the PR head above. The no-mistakes run `01M5CNXT0E1AT5BF19ADNK351A` returned `checks-passed`. The PR remains a draft and unmerged at the time of this receipt.