Tester-feedback triage: converged statuses, owner decisions, and the batch slate (revisions + next pass + measurement)

[Carr1005]

Triage of the pre-F12 tester feedback (4 new testers + the P1/P2/P3 list), verified item-by-item against the post-F12 state (the PR #23 branch), with the owner's design decisions and the routing we converged on. This issue is the candidate slate for the next batch; nothing here is applied yet.

Verified statuses

#	Item	Status	Evidence / disposition
1	Time/cost/dependency-size expectations	Not in spec — owner decided it stays out	No wall-clock/cost/GB figure anywhere (embedding model named twice with no size; Oracle cost qualitative only). Owner: cost figures are perishable, machine-dependent data — the spec-philosophy class we deliberately exclude. Instead: the recommended baseline IS the minimum-resource path, and the scaled test measures it (see Measurement below).
2	Footprint/runtime stop rule for the build agent	Dropped by owner decision	No bound exists in §6/§7; none will be added. The baseline path is the structural resource bound. Note for fairness to the 5.6 GB report: most of that footprint is legitimate dependencies (torch + model), not runaway output. Replaced by heavy-option flags (next-pass item B2).
3	Keyless path not framed as a legitimate finish	Mechanism YES, framing NO	The 19-offline/6-live split, RUN_LIVE=1 gating, and skip-with-reason reporting all exist and work (run-09 and gate-probe-2 both completed keyless). But no sentence blesses it — "5 skipped" reads as unfinished debt. Fix: the baseline-completion sentence — "the recommended baseline build is complete when the offline suite is green; live ACs are an optional keyed extension; skipping them without a key is the expected result." Also resolves item 7. → batch item A1.
4	Stale model names / OpenAI pinning / parameter constraints	Defaults unchanged; scaffolding exists; F12 built the right slot	D4's invariant is already provider-agnostic; CTX-D declares gpt-5 names perishable; R4 falls back to 128k for unknown models. Missing: current-model Options examples, a CTX-D "perishable parameter constraints" category (the temperature rejection), §2 key wording. Provenance constraint: "(course default)" stays on the gpt-5 family — provider-neutrality must not overwrite provenance. Current-model advice belongs in the gate-time "(Recommended)" slot F12 defined, not baked into defaults. → batch items A3 (spec) + B3 (guide rule).
5	Seed/fixture ingestion idempotency (P3: verify-only)	Split	Toolbox re-registration dedup is specified + AC-tested (R9/AC14); store init is create-if-missing/no-wipe (R1/AC1/AC2, restart-tested). Re-running seed ingestion into a populated store is undefined — can't crash in the current realization (generated ids), but silently duplicates. Proposed (not applied): idempotent-seed AC or an explicit "re-seeding duplicates; reset() first" note. → watchlist C3.
6	arXiv 429 (P3: verify-only)	Not in text; evidence now n=2	run-09's AC24 hit 429 twice, passed later — transient; the arxiv client retries internally; inside the agent loop a 429 is already non-fatal via R11 (failed tool-log row, turn continues). Proposed (not applied): document transient-429 as expected for live (net) ACs rather than mandate backoff. → watchlist C4.
7	Skipped ACs read as failure (P3: verify-only)	Mechanism YES, healthy-shape sentence NO	§7's skip rule is right; nothing states what a healthy run looks like per mode. Folded into the baseline-completion sentence (item 3). → batch item A1.
8a	File density / no on-ramp (R16, R18 verbatim: "overwhelming at first due to its density"; "needed a better explanations and instructions in the beginning"; R16's ask: "a brief, high-level overview at the top summarizing what the system actually does")	NOT addressed	First ~100 lines are meta-blockquote → gate instructions → 12-row Ledger; §1 Objective (the first "what does this build" sentence) is buried after all of it. Fix: an orientation block at the top — what you're building + the baseline-completion sentence (no cost numbers, per item 1). Sits with the opening blockquote, before §0, so the gate's "first section" rule is untouched. → batch items A1 (spec) + B1 (guide template).
8b	Process ceremony (R17 verbatim: "the process wrapper (how to ask) was heavier than it needed to be"; "I'd lean on the default → checklist → build flow")	Addressed by PR #23	R17's requested flow is literally the F12 express lane: one baseline question skips all per-row questions → deviation-marked checklist → build. Gate-probe-2 demonstrated the exact flow. Note: R16 praised the stop-and-ask constraint and R17 called the decision content "excellent" — the objection was front-loading, not the mechanism; no further weakening. → watchlist C1 (confirm with next testers).

Owner decisions locked in this triage

1. No cost/time/size figures in specs — perishable data; the scaled test measures the baseline instead.
2. No build-agent stop rule — the recommended baseline is the structural minimum-resource path.
3. Heavy options get their setup weight inline in the Options-cell text (e.g. "(course default — heavy setup: container + admin credentials, multi-GB image)"), NOT a new (warning:) label — F12's two-sanctioned-labels rule stays intact.
4. The keyless offline run is the baseline's completion criterion (the blessing sentence).

The batch (executed together after the full issue sweep)

A. Spec revisions batch (Loop A, agent-memory spec.md):

• A1. Orientation block at top: plain-language overview + baseline-completion sentence (covers 3, 7, 8a).
• A2. Summary-thread-scoping Ledger row (owner decision recorded in Guide residuals from guide-pins-v2: route closure, constants provenance, fixture churn, gate-semantics reform #18 / the F12 matrix; contradicted route, thread-scoped default, global pool alternative with the isolation-vs-recall trade-off).
• A3. D4 Options broadened with current-model examples + CTX-D "perishable parameter constraints" category (temperature rejection) + §2 key wording; provenance labels untouched.
• A4. Heavy-option setup-weight notes inline in D11/D12 (and any other heavy row) Options cells.

B. Next promotion pass (guide; one Opus validation round covers all):

• B1. Orientation-block template (course-agnostic) in the spec anatomy.
• B2. Rule: a heavy option states its setup weight inline in its Options entry.
• B3. Rule: model-currency advice is gate-time "(Recommended)" business, never baked into defaults; CTX-D gains the perishable-parameter-constraints category.
• B4. From Guide residuals from guide-pins-v2: route closure, constants provenance, fixture churn, gate-semantics reform #18's existing queue: identifier-format mining incl. alphabet (cross-model pinning gap); importable fixture module filenames; summary-scoping example sentence + P1→13 canon update.

C. Scaled-test measurement watchlist (measure, don't write):

• C1. Post-F12 ceremony weight (R17's complaint — is one baseline question light enough?).
• C2. Recommended-baseline resource benchmark (wall-clock, tokens, disk) — the numbers item 1 wanted, measured not promised.
• C3. Ingestion idempotency recurrence (item 5).
• C4. arXiv 429 recurrence (item 6).

Remaining from earlier, for the full sweep

• Guide residuals from guide-pins-v2: route closure, constants provenance, fixture churn, gate-semantics reform #18: F8 fixture churn (deferred, proposal recorded unapplied); the three queued findings (fold into B4); the row-set-variance residual (documented, watch).
• Recording: Claude Code transcript compaction can erase the /prepare-build announcement bookend #19: transcript-compaction bookend erasure — recording/tooling track, n=2 (run-08, run-09); separate from the spec/guide batch.
• PR Gate-semantics reform (F12): recommended-baseline rename, label timing frames, deviation-marked checklist, bare-(default) ban #23: under owner review; B-items assume it lands (they build on the F12 template).

🤖 Generated with Claude Code

[Carr1005]

Batch delivered via PR #25:

A-items (spec): A1 orientation block (overview scene + baseline-completion sentence, honesty-tightened: keys unlock live tests and live app use), A2 the D13 summary-scope row (first contradicted-category row), A3 D4 providers-not-model-names + CTX-D request-parameter perishables + §2 key wording, A4 inline heavy-setup notes (D11/D12). Plus the Section-2 pulls: fixture filename made importable; id-alphabet handled guide-side.

B-items (guide, all validated 3/3 on Opus-class regens): orientation template + reader test; heavy-option inline-weight rule; model-currency timing rule; id-alphabet mining rule (closed the 0/3 cross-model gap — 3/3 after); importable-filename rule; the scoping-contradiction worked example (scope row held 3/3; one regen produced the first exact-canon 13/13 Ledger).

Live-AC item — closed, with a find: first full keyed run. AC4/AC9/AC19/AC22/AC24 passed live; AC21 (the continuity demo) failed deterministically — the model is never told its thread id, guessed current_thread, and the summarize tool honestly consolidated nothing. Spec gained the R11 harness-known-identity constraint (living-document rule), the build was conformed, and AC21 passed on rerun: live suite 6/6. The guide generalization (harness-known identities are injected, never model-guessed) is the next pass's first candidate.

C-watchlist: unchanged (ceremony weight, baseline benchmark, ingestion idempotency, arXiv 429) — all await the scaled test.

🤖 Generated with Claude Code