Skip to content

feat(provenance): content-addressed verdict cache + report attestation#237

Merged
drewstone merged 2 commits into
mainfrom
feat/verdict-cache-attest
Jun 10, 2026
Merged

feat(provenance): content-addressed verdict cache + report attestation#237
drewstone merged 2 commits into
mainfrom
feat/verdict-cache-attest

Conversation

@drewstone

Copy link
Copy Markdown
Contributor

What

Track T6 of the 6-track program: judge-verdict caching (perf) + reproducibility attestation (provenance). Two new root modules, exports appended as one contiguous block at the end of src/index.ts (rebase-friendly).

src/verdict-cache.ts

  • canonicalJson(value) — stable stringify (recursively sorted keys). Throws on undefined / function / symbol / NaN / ±Infinity / bigint / Map / Set: ambiguity is an error, not a coercion. Honors toJSON so Dates canonicalize to ISO strings instead of {}. Strict/sync counterpart to pre-registration.ts's permissive async canonicalize/hashJson (cross-referenced in the doc).
  • contentHash(value) — hex sha-256 over the canonical JSON (node:crypto).
  • VerdictCacheStore (sync-or-async get/set) with inMemoryVerdictCache() and fileVerdictCache(path) — JSONL append + in-memory index; a corrupt or wrong-shape line throws at load with file:line, never skip-and-continue.
  • cachedJudge(judge, store, { judgeVersion }) — wraps any JudgeConfig generically (preserves TArtifact/TScenario + appliesTo; single JudgeScore import point so the spine PR's type dedupe rebases as a one-line change). Key = contentHash({ artifact: canonicalJson, scenarioId, judgeName, dimensions, judgeVersion }). judgeVersion is REQUIRED — silent judge upgrades must never serve stale verdicts. Hit path never invokes score(); stats() exposes { hits, misses }. Thrown judges are NOT cached (a cached failure would pin a transient outage forever).

LAW (stated verbatim in the module doc): cache JUDGE VERDICTS only — judging the same artifact with the same judge+rubric is pure. NEVER cache agent rollouts.

src/attestation.ts

  • attest(report, provenance)AttestedReport { reportHash, provenance, algorithm: 'sha256/canonical-json' } for ANY serializable report (campaign results, fuzz capsules, scorecards) — deliberately not coupled to any report schema. Provenance carries modelVersions, seeds?, priceTableHash?, codeSha, inputsHash?, caller-supplied createdAt (substrate stays clock-free).
  • verifyAttestation(report, attested){ valid, reason? } typed outcome: names the exact mismatch (unknown algorithm / hash mismatch / non-canonicalizable report).
  • Layering doc: cryptographic SIGNING is the consumer's layer; content-addressing is the substrate's.

Tests

28 new deterministic tests (no LLM, no RNG): canonicalJson key-order stability + every ambiguity throw; cache hit/miss across artifact / judgeVersion / rubric-dimension / scenario changes; fake-judge call-count proves the hit path never invokes score(); file-store roundtrip across instances + corrupt-line and wrong-shape loud failures; attest→verify roundtrip + single-field tamper → { valid: false, reason }.

pnpm typecheck / pnpm test (209 files, 2010 passed) / pnpm build all green. No version bump (release sequenced by program lead).

- canonicalJson: strict stable stringify (sorted keys; throws on
  undefined/function/symbol/NaN/Map/Set — ambiguity is an error, not a
  coercion) + contentHash (sync node:crypto sha-256). Strict/sync
  counterpart to pre-registration's permissive async canonicalize/hashJson.
- cachedJudge wraps any JudgeConfig generically: cache key covers artifact
  content, scenarioId, judge name, full rubric dimensions, and a REQUIRED
  judgeVersion (silent judge upgrades must never serve stale verdicts).
  Hit path never invokes score(); stats() exposes hits/misses. Caches
  judge verdicts ONLY — never agent rollouts (judging is pure; rollout
  caching destroys best-of-N diversity).
- VerdictCacheStore with inMemoryVerdictCache + fileVerdictCache (JSONL
  append + in-memory index; corrupt line throws with file:line).
- attest/verifyAttestation: reproducibility attestation for any
  serializable report (reportHash + modelVersions/seeds/priceTableHash/
  codeSha/inputsHash provenance, algorithm 'sha256/canonical-json').
  Content-addressing is the substrate's layer; cryptographic signing is
  the consumer's.
tangletools
tangletools previously approved these changes Jun 10, 2026

@tangletools tangletools left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Auto-approved PR — 9a68fe40

Blanket team auto-approval is enabled for this reviewer service.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.

tangletools · auto-approval · reason: blanket_auto_approve · 2026-06-10T10:39:38Z

@tangletools tangletools left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Auto-approved PR — a0ea326e

Blanket team auto-approval is enabled for this reviewer service.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.

tangletools · auto-approval · reason: blanket_auto_approve · 2026-06-10T10:55:07Z

@drewstone drewstone merged commit f57d590 into main Jun 10, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants