diff --git a/.claude/skills/generate-spec/references/spec-generation-guide.md b/.claude/skills/generate-spec/references/spec-generation-guide.md index 8df9d62..c05b2ae 100644 --- a/.claude/skills/generate-spec/references/spec-generation-guide.md +++ b/.claude/skills/generate-spec/references/spec-generation-guide.md @@ -95,7 +95,7 @@ Either way — project, technique, or dependency — name **one** buildable targ Notebooks give you *what and how-configured*; transcripts give you *why and what-it-trades-off*. Mine both. **Pass A — Notebooks.** Extract: -- Every working parameter (chunk sizes, overlaps, k-values, temperatures, paths) **with its exact location**, distinguishing exploratory-notebook configs from the course's end-to-end *application* config. Parameters include **output-shape constraints the course's prompts impose** (length bands, required formats, reject-lists for generic outputs) — these are working parameters of the pipeline exactly as a numeric config is, and they vanish silently if mined only when numeric. When both exist, the app config is the recommended starting point (exception: a *contradicted near-equivalent lever* resolves by the §5.5 lever-value rule instead); record the rest as alternatives with their context. Never call a value "the default" if the materials contain more than one config. +- Every working parameter (chunk sizes, overlaps, k-values, temperatures, paths, identifier formats) **with its exact location**, distinguishing exploratory-notebook configs from the course's end-to-end *application* config. Identifier formats are parameters too, and their **alphabet is part of the value**, not just the length: an id the course generates as 8 hex characters is a different contract from "8 characters", and the alphabet is the half that silently drops when only the length is mined. Parameters include **output-shape constraints the course's prompts impose** (length bands, required formats, reject-lists for generic outputs) — these are working parameters of the pipeline exactly as a numeric config is, and they vanish silently if mined only when numeric. When both exist, the app config is the recommended starting point (exception: a *contradicted near-equivalent lever* resolves by the §5.5 lever-value rule instead); record the rest as alternatives with their context. Never call a value "the default" if the materials contain more than one config. - The API surface actually used: namespaces, class/function names, model names — and whether installs are **version-pinned**. State the era honestly (e.g. "installs unpinned, pre-1.0 namespace"). Never invent pins the notebook doesn't contain. - **Course-declared identifiers and constants** (store/table names, type enums, fixed labels the notebooks declare). Those that participate in a contract or schema stay **binding** in the spec; the rest land in **CTX as provenance** — a short name-map to the course's own terms — never as binding values. They exist so a learner can map the spec's concepts back to what the lessons show on screen; dropping them entirely severs that bridge. - Deliberately planted failure demonstrations (a duplicated input file, a query designed to fail) — these are curriculum, not accidents → they become rules. @@ -129,7 +129,7 @@ The **Decision Ledger** is the spec's second section (§6, right after the §0 p - **Category** — the §5.5 entry route that admitted the row (`design-argued | design-structural | realization | contradicted | learner`). Exactly one label, chosen by the precedence in the surfacing bar below; it determines the row's position in the fixed row order. - **Decision** — short name (e.g. "persistent store", "memory-core topology", "project"). - **Invariant** — what MUST hold to preserve the taught pattern. For learner-context rows it carries only the pattern's capability requirements on that dimension (§3) — typically empty for project/data/goal/scope-boundary, never invented. Write it precisely: this field doubles as the contract a future integration must satisfy. -- **Options** — realizations that satisfy the invariant, from course-faithful to tech-agnostic. The course's own technology always appears here even when it is not the default — and when it is not the default (a §3 branch-1 substitution), tag that entry **"(course default)"** in the cell text, so the §6.0 gate's labeling duty is deterministic: the build agent must never have to infer which option the course actually used. Never tag an Options entry with a bare "(default)": the Default column carries default-ness, and §6.0's two labels (provenance and recommendation) are the only sanctioned option labels. +- **Options** — realizations that satisfy the invariant, from course-faithful to tech-agnostic. The course's own technology always appears here even when it is not the default — and when it is not the default (a §3 branch-1 substitution), tag that entry **"(course default)"** in the cell text, so the §6.0 gate's labeling duty is deterministic: the build agent must never have to infer which option the course actually used. Never tag an Options entry with a bare "(default)": the Default column carries default-ness, and §6.0's two labels (provenance and recommendation) are the only sanctioned option labels. An option whose realization requires provisioning the learner may not have — a paid key, an admin-installed service, a large download — states that setup weight **inline in its own Options entry** (as prose inside or after its label, e.g. "(course default — heavy setup: …)"), not only in the Trade-off cell: the gate presents Options at choice time, and a weight buried in a cell the learner is not shown is a surprise deferred, not disclosed. - **Default (course-derived)** — **exactly one** buildable target (§3). For heavy-dependency rows, resolved by the §3 dependency precedence, with the branch stated. When the default is the course's *example realization* — a domain-specific instantiation (its demo data, its example tools) rather than the pattern itself — the Decision or Default cell must say so and mark it as expected to be swapped when the learner's project (the project row) differs: the invariant, not the example, is what must survive. - **Trade-off** — what switching costs (carries the course's *spoken* trade-offs: re-embedding, re-tuning, lateral-or-worse results). - **Owner** — a label only: `learner`, `course`, or `course+learner` (§9). Metadata; never gates behavior. @@ -142,7 +142,7 @@ The **Decision Ledger** is the spec's second section (§6, right after the §0 p - **`design-structural`** (pattern-structural) — choosing differently changes the system's *structure or semantics*: a pipeline stage added/removed/reordered, ownership moved between harness and model, a guarantee changed (reversibility, consistency), the data model or an interface reshaped. Litmus test: *redraw the architecture diagram after the change — did a box, arrow, owner, or guarantee change, or only a number inside a box?* Examples: store topology, the deterministic-vs-agent-triggered operation split, recoverable-compaction vs lossy-summarization. Counter-example: a retrieval `k` is load-bearing — set it absurdly and the pattern degrades — but no box, arrow, or guarantee changes; **sensitivity alone does NOT qualify** (such a value stays a body default unless another route admits it). - **`design-argued`** (course-argued) — the transcripts/slides explicitly present an alternative and argue a trade-off ("you could do X, but we do Y because Z"). Detection source: Pass B's mined spoken trade-offs (§4) — what the instructor argued about is what the course teaches *as a decision*. This route applies **even when the course's choice would otherwise be encoded as an invariant**: weaken the Invariant field to what must truly hold, make the argued position the Default, and carry the course's narrated argument — with its lesson citation — in the Trade-off cell. The default is always the course's argued position, so no-intake determinism (§3, §12.7/§12.8) is unaffected. **A row requires a genuinely two-sided argument.** When the mined "argument" is the course teaching X as best practice and demonstrating or warning against Y as the anti-pattern, do NOT emit a Ledger row offering X and Y as neutral options — a menu invites picking the warned-against side. Bake X in as a business rule (with an AC where testable); carry Y in the Trade-off/CTX narrative as the cited anti-pattern. Litmus: *did the course show Y working as an acceptable alternative, or only as the failure case?* Only the former earns a row. Judge "working" against how the course *frames* Y, not against the course's final goal: an alternative the course demonstrates working at reduced scope and then **moves beyond** (a baseline, a stepping stone, a simpler form it builds on) is NOT an anti-pattern — it remains a legitimate reduced-scope choice and keeps the subject a decision. Anti-pattern status requires the course to warn against Y as a *failure* (errors, degradation, broken behavior), not merely to outgrow it. And an "alternative" that is merely the taught approach's **degenerate or limiting case** — the behavior the taught mechanism itself produces when the workload is trivial (e.g. "with few items, retrieval trivially returns everything") — is NOT a second side: no row; the taught mechanism stays a business rule and the bound it enforces stays AC-tested. -- **`contradicted`** (course-contradicted) — the course itself set it inconsistently across lessons or between signature/docstring/call site (e.g. one distance metric in one lesson, another elsewhere; `k=3` in a signature but `5` in the call). A contradiction always elevates **provenance surfacing** — record it where the value is declared (in the default's own cell or body sentence, naming each side's source) and in CTX-C — but it earns a Ledger row only if it also passes the **stakes test, evaluated at initial choice time — before any build artifact exists**: would picking the other side *up front* change the system's structure, semantics, or a behavior guarantee (the `design-structural` litmus), or force rework? The cost of *changing the value later* (after data or artifacts exist) is §6 Ask First's business and is NEVER grounds for a row — nearly every value is expensive to change late, so late-change cost would promote everything. If yes → Ledger row. If no (near-equivalent, or a tunable knob) → a **body default**, not a row: pick the value by the **deterministic lever-value rule** — (1) the value the transcript narration states aloud in the lesson that introduces the concept; (2) if the narration names none or itself conflicts, the end-to-end application's config; (3) else the value used most often across the materials, earliest-taught on ties — cite the deciding branch next to the value, label it a **post-build lever**, and ask no build-start question about it. (For near-equivalent contradicted levers only, this rule takes precedence over Pass A's "app config is the recommended starting point.") Either way: any consistency invariant the contradiction threatens (e.g. one strategy across all stores, write and read) stays a business rule with an AC, and a lever whose change after data exists forces rework stays an Ask-First entry (§6). +- **`contradicted`** (course-contradicted) — the course itself set it inconsistently across lessons or between signature/docstring/call site (e.g. one distance metric in one lesson, another elsewhere; `k=3` in a signature but `5` in the call). A contradiction always elevates **provenance surfacing** — record it where the value is declared (in the default's own cell or body sentence, naming each side's source) and in CTX-C — but it earns a Ledger row only if it also passes the **stakes test, evaluated at initial choice time — before any build artifact exists**: would picking the other side *up front* change the system's structure, semantics, or a behavior guarantee (the `design-structural` litmus), or force rework? The cost of *changing the value later* (after data or artifacts exist) is §6 Ask First's business and is NEVER grounds for a row — nearly every value is expensive to change late, so late-change cost would promote everything. If yes → Ledger row. If no (near-equivalent, or a tunable knob) → a **body default**, not a row: pick the value by the **deterministic lever-value rule** — (1) the value the transcript narration states aloud in the lesson that introduces the concept; (2) if the narration names none or itself conflicts, the end-to-end application's config; (3) else the value used most often across the materials, earliest-taught on ties — cite the deciding branch next to the value, label it a **post-build lever**, and ask no build-start question about it. (For near-equivalent contradicted levers only, this rule takes precedence over Pass A's "app config is the recommended starting point.") Either way: any consistency invariant the contradiction threatens (e.g. one strategy across all stores, write and read) stays a business rule with an AC, and a lever whose change after data exists forces rework stays an Ask-First entry (§6). A worked example of a contradiction that **passes** the stakes test: a **storage-scoping choice** the course set both ways (records written with an owning-scope id in one lesson's code and without it in another's) — the two sides return different result sets to the same read, a semantics change at initial choice time, so it earns a row carrying both sides, rather than resolving as a lever. - **`realization`** — a heavy-dependency decision surfaced by the §3 dependency precedence (the row states which branch was taken and why). - **`learner`** — the learner-context dimensions (§3): project, data/inputs, goal, model/provider, environment, scope-boundary. @@ -175,12 +175,21 @@ The spec MUST contain these core sections, in this order (this is the seven-sect e.g. " chatbot" (precedence 1) or " app" (precedence 2). A learner retitles when they fill [project]. --> - + ## 0. Before you build — REQUIRED (do this first) @@ -240,7 +250,10 @@ The spec MUST contain these core sections, in this order (this is the seven-sect ## 5. Acceptance Criteria ★ (the oracle) + Given/When/Then table. Concrete inputs and asserted outputs only — no "typically". + Fixture module filenames must be importable identifiers in the build language (a + hyphenated .py file cannot be imported): an unimportable literal forces every + builder to either deviate from the spec's exact name or write loader shims.> ## 6. Standing Permissions (in force for the entire build) **Always** / **Ask First** / **Never** @@ -285,11 +298,11 @@ A Decision Ledger that no one is shown is worthless. A build agent's default beh - **Imperative and build-agent-addressed.** "You are the build agent. Before writing ANY code, you MUST…" — not a passive "review before build" header (which reads as a note to a human skimming and gets skipped). - **First.** It is spec section `## 0`, ahead of everything including the Decision Ledger, visually isolated. -- **An express lane for the baseline builder, asked first — and named honestly.** The gate's FIRST question offers exactly two paths: the **recommended baseline build** (every Ledger row resolves to its Default), or **customize** the decisions row by row. The express path is named "recommended baseline build" — never "build as-is with the course's setup" or any phrasing presenting it as the course's own configuration — because on substitution rows (§3 branch 1) the Default deliberately departs from what the course ran, so a course-default framing misattributes those rows: the same provenance error the labeling rule below prevents at option level, recurring at path level. The baseline option's text must carry a one-line explanation stating both halves: most rows resolve to the course's own choices, and wherever the course's choice needs setup the learner may not have (a paid key, an admin-provisioned service), a lighter equivalent stands in — with the step-5 checklist marking exactly where. Choosing the baseline skips the per-row questions entirely — the agent prints the full resolved-decision checklist and begins building; choosing customize enters the one-question-per-row loop. This is NOT an escape clause (§12.10): the user gave an explicit answer that covers every row — the same "defaults are fine" reply the determinism argument already sanctions — and the printed checklist still makes completeness visible. What stays forbidden is proceeding on *silence*. +- **An express lane for the baseline builder, asked first — and named honestly.** The gate's FIRST question offers exactly two paths: the **recommended baseline build** (every Ledger row resolves to its Default), or **customize** the decisions row by row. The express path is named "recommended baseline build" — never "build as-is with the course's setup" or any phrasing presenting it as the course's own configuration — because on substitution rows (§3 branch 1) the Default deliberately departs from what the course ran, so a course-default framing misattributes those rows: the same provenance error the labeling rule below prevents at option level, recurring at path level. The baseline option's text must carry a one-line explanation stating both halves: most rows resolve to the course's own choices, and wherever the course's choice needs setup the learner may not have (a paid key, an admin-provisioned service), a lighter equivalent stands in — with the step-5 checklist marking exactly where. Choosing the baseline skips the per-row questions entirely — the agent prints and writes the full resolved-decision checklist and begins building; choosing customize enters the one-question-per-row loop. This is NOT an escape clause (§12.10): the user gave an explicit answer that covers every row — the same "defaults are fine" reply the determinism argument already sanctions — and the printed checklist still makes completeness visible. What stays forbidden is proceeding on *silence*. - **Presents EVERY Ledger row.** Do not have the gate re-filter or tier rows — the §5.5 surfacing bar already decided what became a row, so every row is by construction worth showing. (Never key the gate on row numbers; row identities are course-specific.) - **One question per row — do not depend on a channel's item cap.** Interactive question tools cap how many questions fit in one call (Claude Code's `AskUserQuestion` allows at most a few), and a Ledger can have many rows. If the gate says "present every row" without saying *how*, the agent improvises — batching inconsistently, or (worse) presenting a first batch, getting a reply, and treating the build as unblocked while the rest are never asked (§12.11). Fix the mechanism deterministically: **one row = one question**, looped until every row is asked. One-per-row has no batch boundary to mis-track and its completeness is a plain count (N rows ⇒ N questions). **Never hardcode a numeric cap** ("4") into the gate — that is one environment's limit; state the capacity-relative rule instead so it holds for an uncapped or plain-text channel too. -- **Checklist-echo gates the build, not "a reply happened".** Keying resumption on "the user replied" fails the moment many rows exist: a reply to *some* rows satisfies it, and the rest are buried. Require a **visible artifact** instead — before any code, the agent prints a checklist of every row with its resolved value. A completion-driven agent cannot produce that checklist without having resolved every row, and the user/reviewer can *see* completeness. A printed artifact beats an internal "did I ask N times?" count precisely because the same agent that skips rows is the one that would judge the count. The checklist also carries **deviation marks**: each row's line states whether its resolved value is the course's actual choice or departs from it — a substitution row resolved to its Default departs by construction; a customize answer may too — naming the course's choice on every departing row. And on the baseline path, the agent walks the learner through the departing rows alongside the checklist (each row's course choice, and the reason its Default substitutes — the row's own branch note carries it), so the express lane never hides where the baseline differs from the course. -- **Labels the course's actual choice "(course default)"; the build agent's own "(Recommended)" flag may coexist.** "(course default)" is factual provenance — what the course actually did — and must always mark the option the course actually used. On a **substitution row** (§3 dependency-precedence branch 1) that is the course-faithful **Options** entry, NOT the substituted Ledger Default: labeling the substitute "(course default)" misattributes it (observed in real learner feedback — a SQLite+Chroma default presented as the course's choice when the course ran Oracle). Separately, the build agent may mark an option "(Recommended)" per its own judgment or its question tool's convention (Claude Code's `AskUserQuestion` suggests exactly this) — that is advice about *this* learner's project, a different claim from provenance, and it is allowed. The two labels compose: when the agent's recommended option differs from the course's choice, both labels appear, each on its own option; when the recommended option IS the course's choice, use the single combined label **"(Recommended - course default)"**. A recommendation never removes or moves the "(course default)" label. The two labels also differ in **when their claims are made**: the Ledger's Default column is the *generator's* recommendation, fixed at generation time on zero-setup grounds (buildable as-is with nothing the learner must provision — the same grounds §3's precedence uses); "(Recommended)" is the *build agent's* judgment at gate time, made with the learner's context (their earlier gate answers; keys or services they actually hold). With no gate-time reason to depart, the agent recommends the row's Default; with one (say, the learner holds the key a keyed-tool row's course option needs), "(Recommended)" may land on any option — the label that never moves is "(course default)". These two labels, plus their merged form, are the **only sanctioned option labels** anywhere the spec presents choices — gate questions and Ledger Options cells alike. Never label an option with a bare "(default)" or "(Default)": default-ness is carried by the Ledger's Default column, not by a label, and a third label re-opens the provenance-vs-advice confusion the two-label design exists to close (observed as direct owner confusion in review). +- **Checklist-echo gates the build, not "a reply happened".** Keying resumption on "the user replied" fails the moment many rows exist: a reply to *some* rows satisfies it, and the rest are buried. Require a **visible artifact** instead — before any code, the agent prints a checklist of every row with its resolved value. A completion-driven agent cannot produce that checklist without having resolved every row, and the user/reviewer can *see* completeness. A printed artifact beats an internal "did I ask N times?" count precisely because the same agent that skips rows is the one that would judge the count. The checklist is also **written** to a fixed name in the build folder (`resolved-decisions.md`): printed for the person present, written so the values the build was built from stay recoverable in a place a reader knows to look, rather than in whatever file the agent invents. It is a record, not an input — a later build re-runs the gate instead of reading a previous run's file. The checklist also carries **deviation marks**: each row's line states whether its resolved value is the course's actual choice or departs from it — a substitution row resolved to its Default departs by construction; a customize answer may too — naming the course's choice on every departing row. And on the baseline path, the agent walks the learner through the departing rows alongside the checklist (each row's course choice, and the reason its Default substitutes — the row's own branch note carries it), so the express lane never hides where the baseline differs from the course. +- **Labels the course's actual choice "(course default)"; the build agent's own "(Recommended)" flag may coexist.** "(course default)" is factual provenance — what the course actually did — and must always mark the option the course actually used. On a **substitution row** (§3 dependency-precedence branch 1) that is the course-faithful **Options** entry, NOT the substituted Ledger Default: labeling the substitute "(course default)" misattributes it (observed in real learner feedback — a SQLite+Chroma default presented as the course's choice when the course ran Oracle). Separately, the build agent may mark an option "(Recommended)" per its own judgment or its question tool's convention (Claude Code's `AskUserQuestion` suggests exactly this) — that is advice about *this* learner's project, a different claim from provenance, and it is allowed. The two labels compose: when the agent's recommended option differs from the course's choice, both labels appear, each on its own option; when the recommended option IS the course's choice, use the single combined label **"(Recommended - course default)"**. A recommendation never removes or moves the "(course default)" label. The two labels also differ in **when their claims are made**: the Ledger's Default column is the *generator's* recommendation, fixed at generation time on zero-setup grounds (buildable as-is with nothing the learner must provision — the same grounds §3's precedence uses); "(Recommended)" is the *build agent's* judgment at gate time, made with the learner's context (their earlier gate answers; keys or services they actually hold). With no gate-time reason to depart, the agent recommends the row's Default; with one (say, the learner holds the key a keyed-tool row's course option needs), "(Recommended)" may land on any option — the label that never moves is "(course default)". **Model currency is the canonical case of this timing split**: the spec names providers, protocols, and the course's own models (provenance facts, which never stop being true), but never bakes in "current best" model names — those rot between generation and the learner's build; recommending what is current when the learner actually builds is the build agent's gate-time job, exercised through this flag. These two labels, plus their merged form, are the **only sanctioned option labels** anywhere the spec presents choices — gate questions and Ledger Options cells alike. Never label an option with a bare "(default)" or "(Default)": default-ness is carried by the Ledger's Default column, not by a label, and a third label re-opens the provenance-vs-advice confusion the two-label design exists to close (observed as direct owner confusion in review). - **A hard stop, with no escape clause.** Presenting the decisions must end the agent's turn: it stops and waits for the user's reply before writing any code or touching any file. Do **not** give it a "proceed if there's no response" release valve — that is the loophole that re-opens §12.10: the agent *always* trivially has "no response" the instant it finishes presenting, so it reads the valve as immediate permission and the ask becomes theater. A narrower "proceed if launched non-interactively" escape is the same hole, smaller — drop it too; a completion-driven agent steers into whatever escape exists. Determinism is **not** at risk from waiting: it comes from every row having one course-derived default (§3, Known-Trap #7), so a user who replies "defaults are fine" lands on the identical target. Waiting changes *when* the agent proceeds, never *what* it builds. (If a genuinely headless harness cannot answer, the absent reply stalls that build — the harness's constraint to resolve by pre-answering, not a hole the spec should pre-drill.) - **Structured tool required when available; still portable.** If the environment has an interactive structured-question tool (Claude Code's `AskUserQuestion` or an equivalent), the gate MUST require using it — it is the reliability lever, not a mere suggestion. Name specific tools only as *examples* of the class, never as a hard dependency: a non-Claude agent (Codex, Cursor, plain chat) with no such tool must still comply by listing the rows in its reply and asking. So the rule is "use a structured question tool **if you have one**, else ask in plain text" — mechanism-*required* where possible, mechanism-*agnostic* in fallback. @@ -335,14 +348,15 @@ not because they are the right choice for this person's project. an answer (a chosen option, an explicit "use the course default", or the step-2 baseline answer, which resolves every row at once). Answers to *some* rows do NOT release the build; "no reply yet" is not an answer — wait for the user. -5. **Before the first line of code, print a resolved-decision checklist** — every Ledger row - with its final value (the user's choice, or its Ledger default), each line carrying a - deviation mark: `= course choice`, or `≠ course choice (course used: