Skip to content

UK local release: publish a K=15, 150,000-household dense dataset in at most two builds (#1301) - #1307

Merged
juaristi22 merged 46 commits into
mainfrom
uk-local-release-1301
Oct 17, 2026
Merged

juaristi22 merged 46 commits into
mainfrom
uk-local-release-1301

Conversation

@juaristi22

@juaristi22 juaristi22 commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator

Part of #1301. This is the code half of the plan: a K=15, 150,000-household dense (local-area) release that can be published in at most two builds. The rulings are on #1124 (16 October) and below (17 October).

Summary

The plan's nine ordered commits and a tracking commit, then the fixes from the first review. After #1302 merged, the branch was rebased onto main and gained the re-review's fixes and #1302's remaining review items. The replies are in the comments.

  1. Holdout and incumbent comparison opt-in.
    • --rotated-holdout replaces --skip-holdout. --incumbent-diagnostic opts the incumbent steps back in.
    • The release contract treats the three incumbent files as all-or-none. Its core checks always run.
    • The manifest records what is not claimed: no_out_of_sample_claim, no_incumbent_comparison, engine_measurement, size_doctrine, and the reviewed exclusions the gates relied on. Exclusion reasons are copied verbatim from the committed registers, so they can carry national figures.
  2. Per-area support figures withheld; the disclosure walk warns.
    • Shipped files no longer carry per-area rows, Kish ESS or distinct sources.
    • The release contract walks every shipped JSON document and warns, without refusing the release, when it finds one of those figures. It looks in per-area records, code-keyed maps, columnar tables, lists, and minimum, maximum, median or other order statistics of a figure.
    • The walk:
      • matches whole support-figure names in any case, so a cross-grain relation's max_excess is not one;
      • treats region, nation and UK codes as no area;
      • reads key names and the shortfall wording only, not prose, other names or non-JSON files.
    • Where the warnings surface:
      • validate_release_dir returns them;
      • the assembler, the release pre-flight, publication and the sign-off --verify print them;
      • the pre-flight's --candidate-dir walks the three files the assembler ships byte for byte, so a finding shows after each build.
    • The drafter still refuses a figure in its own drafted entries, which go into a public PR.
    • Weights and per-cell estimates ship as before in the private release.
    • A failed refresh upload after a passed read-back no longer kills the build.
  3. 150,000 households and a lower L0 search.
    • calibrate() takes a budget_search_bracket. Callers on the default bracket keep the same bracket and cold path. On the warm path, a start outside the bracket now raises instead of clamping, and the search stops at a bound.
    • The UK size search runs in [1e-10, 10].
  4. Gate-only sign-off for local cells and areas.
    • uk_local_target_fit reads uk/local_target_fit_reviewed_exclusions.json.
    • Area-support exclusions no longer defer the area's cells.
  5. High-value homes region control, on the local line only (calibration_lines: ["local"]).
    • The North East and Wales rows over £2m sit below the 400-home floor, at about one household weight each. They are signed out of the region fan-out (ruling below); the same areas' £1m and £1.5m rows stay bound.
    • A row-level signed exclusion can now name a geography cell (row: {"dimension": "geography_id", …}).
    • Their £2m authority cells are all deferred, so the cross-grain reconciliation lets a leg that holds only deferred cells stand without a control, receipted under deferred_legs_without_control. A bound row on such a leg is still refused.
  6. Solve registers.
  7. Solve reuse between builds.
    • Gate-only registers leave the solve fingerprint.
    • --gate-review-date gives the calibrated gates their own clock.
    • --require-solve-reuse fails before the replay if:
      • there is no explicit --graph-store;
      • --out holds an earlier attempt (build.json, failure.json or a Logbook spool row);
      • the gate clock falls outside the window of an entry build 2 depends on: an in-line gate-only or degenerate-register entry, premature or expired, or a measure exclusion the solve applied. The refusal names the entries and the usable range.
  8. Sign-off drafting tool, tools/draft_uk_dense_signoff.py.
    • Public body: PR_BODY.md carries no local fit bands.
    • Free text: digits only in allowlisted references (microcosm#N, chronicle#N, uk-data#N, PolicyEngine/<repo>#N, with optional #issuecomment-N), PolicyEngine issue and pull request URLs, and ISO dates. --approved-by must be a GitHub login that starts with a letter and holds at most one run of digits.
    • Output location: it refuses an --out inside any git work tree.
    • National approvals: it never drops an approval that applies on every line. It rescopes it to the national line, and --verify --draft-receipt checks that only the lines changed.
    • Assembly deadline and gate-clock range: both count every entry build 2's gates hold to its window, bound_despite_support_floor and input-mass entries included.
  9. One regeneration pass on the pinned abf0921 feed, after rebasing onto Re-pin the UK Chronicle feed to abf0921 (#1300) #1302.
    • It covers the national and local references and memberships, both compile-parity receipts and the census.
    • The gate digests and the US spec-engine attestation do not move, and the count pins follow the regenerated files.

The national target kernel now also hashes measure_simulation.

From #1302's re-review, folded in since it merged:

  • Second-pass test: a real-rule test makes the second reconciliation pass refuse with off-surface cells present.
  • UC bridge wording: its texts say it bounds each grain's GB total.
  • Receipt rationales: the local receipt's signed UC rationales describe each area's paid claims, with ranges computed from the rows.
  • Streaming: the CGT test and the receipt tool stream the feed.
  • Fixture alignment: it builds its contract lookup once. The output is byte-identical and it runs in seconds instead of minutes, in the gate and the tool.
  • Gate replay test: a feed-free test replays the local compile-parity gate on the committed membership, so a stale receipt fails CI.
  • Bound check: the shared cross-grain operator refuses a bound of 1 or more.
  • Stock inputs: a feed-gated test checks the high-value home stock's inputs on the pinned feed.
  • Corrections: in the docs and the changelog.

Rulings (17 October)

  • The release disclosure walk warns, never refuses.
  • The two high-value homes region rows below the 400-home floor are deferred, not bound.
  • Local-cell sign-offs publish as drafted. Their reasons never state an estimate or an error.
  • Per-cell estimates and weights ship as they are in the private release (16 October). Public outputs apply the 10-source check.

Two-build flow

  1. Build 1:

    --release-role dense --release-candidate
    --review-date <D> --graph-store <store>
    --dataset-households 150000 --selection-initial-lambda 2.5e-8
    … --out <out-1>
    

    Then the pre-flight with --candidate-dir. Any national row, local cell or area that still fails is signed gate-only with tools/draft_uk_dense_signoff.py --run-dir <out-1>.

  2. PR-B carries register data and pins only. Merge it with a merge commit.

  3. Build 2: run at PR-B's head with build 1's arguments and the same --review-date, --graph-store and warm start. Add:

    • a new --out <out-2>;
    • --gate-review-date inside the range the draft printed;
    • --require-solve-reuse;
    • --logbook-prev-row-digest set to build 1's row digest.

    It recomputes only the calibrated gates, the export, the package and certification.

  4. Then: draft_uk_dense_signoff.py --verify, the pre-flight, assembly, validate_release_dir, and a tag-only publish (--tag-only --no-latest). Read every warning before publishing.

Merge order and what is not here

Merge order:

Engine: the policyengine.py certification pins whatever policyengine-uk main locks at build 1.

Not here:

Notes for review

  • Node keys from target compilation down move, so no older graph store is reused. That is intended before build 1.
  • National cuts certified before this PR fail validate_release_dir at the new head, as with every gates.json edit.
  • The national exclusion names are bound into the hash-joined candidate manifest. That manifest is unsigned, so it gives the same assurance as the manifest's measure-exclusion, holdout and size claims.
  • Peak memory (Re-pin the UK Chronicle feed to abf0921 (#1300) #1302's 1302-03): the build's own target load on abf0921 is being measured and goes in a comment. The regeneration tools peaked at 15.4 to 16.6 GB for the references and 25.5 GB for the receipt tool on both surfaces, so regenerate receipts with the machine to itself.

Testing

  • At e119867, locally:
    • The touched and surface-dependent test files pass.
    • The CI test plan verifies, and ruff is clean.
    • The census check is current, the gate digests equal their pins, and the US spec-engine coverage check passes.
  • The feed-gated tests ran locally against abf0921 (CI skips them), and so did the build's own target compilation. Results are in the comments.
  • Full suites run in CI.

🤖 Generated with Claude Code

@juaristi22

Copy link
Copy Markdown
Collaborator Author

Independent adversarial review of this PR at ea68813

Run with Claude Code on 16–17 October 2026 at ea68813: seven independent review lenses (correctness, method, generated artifacts, tests, claims against the diff, disclosure, a second fresh-eyes pass), then one to three skeptics per finding who tried to refute it; (k/n) is how many agreed. No builds and no feed-gated tests were run. The PR head has not moved since the review.

#1307 can merge on code and CI grounds. All 10 checks pass at ea68813, and about 2,300 tests on the touched surfaces pass locally. No finding shows wrong numbers in a build, or a release gate that fails open. Two things need settling before the PR is used for the release:

  • The sign-off drafter can put a sub-GOR fit band and unchecked free text into PR-B's public body and registers (1307-01, 1307-02). Both fixes are small, so do them here.
  • The merge order with Re-pin the UK Chronicle feed to abf0921 (#1300) #1302 needs fixing. Whichever order is chosen, the local compile-parity receipt must be regenerated on abf0921 at the final head before build 1, or build 1's preflight refuses (Cross-PR, X-s01).

Blocking

These don't block the merge itself. They block using the drafter for PR-B, which is only days away, so fix them in this PR.

  • The public PR-B body's stale-risk counts bracket named sub-GOR local estimates. CONFIRMED (2/2; one verifier rated it low).

    • Where: tools/draft_uk_dense_signoff.py:808-811 (stale_risk), :1552-1567 (the PR-body table prints Stale-risk and Near-bound for uk_local_target_fit), :1606-1608 (the "Disclosure: counts and gate ids only" line).
    • What goes wrong: say PR-B signs one local cell and the body prints Stale-risk 1. That cell's key (family/area_code/metric) is committed in the same patch, so a reader learns the LA or constituency estimate lies within about one point outside the 25% bound, against a published target. That is a survey-derived figure below GOR level with no disclosure check. The body's own disclosure line will reassure whoever does the manual scan.
    • Corrections from the verifiers:
      • Near-bound counts cover unsigned cells and never name one. Only Stale-risk brackets a named cell.
      • Only the operator can narrow the band (--stale-risk-margin, :2199, :2221-2222).
      • local_target_references.json carries no values. The targets are still published statistics.
      • Nothing is public yet, because the local register is empty at head.
    • Committing local keys as ">25% off" is in the approved UK local-area release: publish a K=15, 150,000-household dense dataset in at most two builds #1301 plan. One verifier still wants a ruling on it.
    • Fix: keep the local registers' Stale-risk and Near-bound counts in PRIVATE_review.csv and on stdout. In PR_BODY.md, print them for the national register only, or as present/absent.
  • The drafter's "no figures" guard fails open on reviewer free text that lands in public registers. CONFIRMED (2/2).

    • Where: tools/draft_uk_dense_signoff.py:1648-1657 (_pairs), :1675-1681, :1387-1407; packages/microcosm-data/src/microcosm/data/contract.py:5082-5085, :5123.
    • What goes wrong: _pairs refuses only a "%". The contract walk only catches the shape (rows|ESS|distinct sources) N <. A probe with synthetic strings showed these pass into the entry reasons and signoff.patch, which is then committed to the public repo:
      • a record or household reference with a weight
      • "N percent above its target"
      • "7 rows against the 50-row floor"
      • "ESS=3.2"
      • "sources 4 < 50"
    • The runbook's manual scan (docs/uk-dense-release-assembly-runbook-762.md:212-213) covers PR_BODY.md, not the patch. The runbook (:181-183) and the changelog fragment say the reasons "carry no figures" because the walk runs over them, which overstates what the walk checks.
    • Corrections from the verifiers:
      • --adjudication and --approved-by do go through the narrow walk; they only miss the "%" check.
      • Per-cell estimates passing is a recorded design choice (contract.py:5060). The record/weight and "percent" cases are not.
    • Fix:
      • Refuse any digit in --class-note, --area-name and --adjudication, except #NNNN and ISO dates, or pin --adjudication to a microcosm#NNNN ... pattern.
      • Reword the runbook and changelog.
      • Extend the manual scan to signoff.patch.

The Cross-PR finding X-s01 blocks build 1. See below.

Should fix

  • Two newly bound high-value homes region rows are below the PR's own 400-home floor, and each is its region's only £2m+ control. PLAUSIBLE (1/2).

    • Where: packages/microcosm-build/src/microcosm/build/uk/local_small_cell_rule.json:39.
    • What goes wrong: over_2m_by_region@E12000001 (North East) and @W92000004 (Wales) resolve below 400 homes from Land Registry-derived stock. The floor defers every £2m LA cell in both regions, so each row alone controls its region. Both rows bind and are tested release-blocking at 25% under uk_target_fit (full_gates.py:233-236, battery_bindings.py:1702-1711). Their loss weight is small under sqrt-count weighting. The rule text says a cell this size "is noise", yet keeps region rows out of its scope. No commit message, the body or a tool mentions this.
    • Why the skeptic refuted it:
      • The rule's scope is documented and deliberate: region rows are the top control.
      • A breach is handled, because the drafter signs any failing national row gate-only.
      • Pre-signing would fail build 1 as a stale exclusion if a row actually fits (terminal_gates.py:1078-1096).
      • The size selection protects each target's largest carrier, so "no carrier" can't happen.
    • What would settle it: build 1's fit on the two rows, plus whether each rests on at least 10 distinct source households.
    • Fix: don't pre-sign. Name the two rows in the rule text. Decide now whether they would be signed gate-only on a breach. María's ruling is needed on shipping their fitted estimates.
  • The sign-off draft deletes an unscoped uk_target_fit approval that the national line may still need. CONFIRMED (2/2).

    • Where: tools/draft_uk_dense_signoff.py:797-802 (first cited as :772); RegisterDraft.after() at :709-716.
    • What goes wrong: an in-force entry with no release_lines, which dense build 1 fits inside the bound, goes to draft.remove with only a receipt note ("also applies on the national line"). It is then dropped from the shared register. The next national build or certification fails on a row someone already signed.
    • Rescoping to ["national"] passes both lines in a probe. The tool's own dormant-unscoped path (:803-806) sends such entries to review; this path doesn't.
    • It's latent today because both target-fit registers are empty. Five of the last eight versions of the national register held unscoped entries, though.
    • Fix: draft release_lines: ["national"], or route the entry to review/not-signable. Add a test with an unscoped entry that the dense build fits.
  • The two-build tooling never surfaces the measure-exclusion expiries receipted at build 1. CONFIRMED (1/1).

    • Where: tools/draft_uk_dense_signoff.py:1880-1905 (gate_clock_range), :1979-2060 (verify); tools/preflight_uk_local_release_candidate.py:241, :354-357; solve_scope.py:29-42.
    • What goes wrong: gate_clock_range comes only from the gate-only registers. On this base ons.savings_interest_income expires 2026-10-22. Build 2 and --verify can pass on 10-22, and assembly on 10-23 then refuses. calibration_measure_exclusions.json is in the solve fingerprint, so renewing it then forces a third build.
    • Mitigations: the body says renewals must be on main before build 1, and build 1 logs a UserWarning within 7 days of an expiry. Neither the drafter nor --verify repeats the deadline.
    • Fix: cap gate_clock_range with build 1's receipted measure exclusions (and the degenerate register). Print min(expires_on) as the assembly deadline in PR_BODY.md and the receipt. Have --verify warn near it.
  • --require-solve-reuse is accepted without --gate-review-date. CONFIRMED (1/1).

    • Where: packages/microcosm-build/src/microcosm/build/uk_runtime/rowwise_cli.py:557-591; full_build_cli.py:1329-1332.
    • What goes wrong: build 2 needs build 1's --review-date. If it forgets the gate clock, the calibrated gates run on build 1's clock. Every sign-off entry approved after that date is premature: uk_target_fit, uk_local_target_fit and uk_local_area_support fail after the replay. A probe confirmed premature at 2026-10-20 and a pass at 2026-10-21. The re-run is cheap, but it breaks "at most two builds".
    • Caveat: an unsplit clock is a deliberate option, so an outright refusal may be too strict.
    • Fix: before the replay, refuse when any in-line gate-only entry is premature at the gate clock, or read the drafter receipt.
  • The body's build-2 recipe fails when followed. CONFIRMED (1/1).

    • Where: body line 39; full_build_cli.py:697-700, :1496-1497; docs/uk-full-build-graph.md:144.
    • What goes wrong with "the same arguments plus --gate-review-date and --require-solve-reuse":
      • A new --out with the default store points at an empty store, so the build stops with SolveReuseError at preflight. Re-running test_required_solve_reuse_on_an_empty_store_fails_at_the_preflight confirms it; the test passes.
      • A defaulted --review-date on a later day re-keys the solve.
      • Reusing build 1's --out doesn't raise SolveReuseError. It overwrites build 1's bundle, and --verify then fails on the attempt-id check (tools/draft_uk_dense_signoff.py:2077-2118).
    • The runbook's build-2 paragraph is correct, but its build-1 command passes neither --graph-store nor --review-date. The graph doc repeats the body's error.
    • Not checked: an aside on chaining --logbook-prev-row-digest for build 2.
    • Fix: pass --review-date and --graph-store explicitly in build 1 and repeat them in build 2, with a new --out. Correct the body, the runbook's build-1 command and the graph doc.
  • The obr.ni_employer exclusion also leaves the national objective, just before the frozen national build. CONFIRMED (1/1).

    • Where: packages/microcosm-build/src/microcosm/build/uk/calibration_measure_exclusions.json:527-536.
    • What goes wrong: the register has no line scoping, and the loader refuses one. load_uk_national_target_inputs (full_targets.py:411-426) therefore drops the row on the national line too.
    • The approved UK local-area release: publish a K=15, 150,000-household dense dataset in at most two builds #1301 plan says "off on every line" and the changelog says so, so the scope looks intended. The body still lists it under the local solve registers, and the adjudication cites the local plan.
    • A national run given a frozen register made before this PR fails closed ("re-derived register differs from the frozen scoring register", full_targets.py:437-447). If UK local release: publish a K=15, 150,000-household dense dataset in at most two builds (#1301) #1307 lands before the 20-21 Oct national build, derive that build's frozen register after it.
    • Fix: one sentence in the body and the adjudication saying it applies on both lines.
  • The body's reason for not hashing measure_simulation in the national target kernel is false. CONFIRMED (2/2; low).

    • Where: packages/microcosm-build/src/microcosm/build/uk_runtime/graph_national.py:473-493.
    • What goes wrong: the body (line 52) defers the fix "so as not to re-key" the national graph. At this head the national kernels re-key anyway: full_measure.py, full_targets.py, ledger_targets.py and local_target_census.py changed, and so did the country-spec fingerprint. The gap stays open. A later edit to measure_simulation alone leaves the national problem and gate nodes as cache hits.
    • Fix: add measure_simulation to _NationalKernel's source_hash now, with a test like the dense one.
  • PRIVATE_review.csv can be written inside another, committable worktree. CONFIRMED (1/1).

    • Where: tools/draft_uk_dense_signoff.py:1415-1420.
    • What goes wrong: the guard compares --out only against the tool's own checkout. A probe accepted an --out inside another checkout of the repo and one inside a sibling sign-off worktree. Nothing in .gitignore covers the file.
    • Fix: refuse any --out where git -C <out> rev-parse --show-toplevel succeeds. Add PRIVATE_review.csv and signoff-draft/ to .gitignore.
  • The small-cell rationale copied onto named LA cells reads like a 1-2-case count. CONFIRMED (1/1).

    • Where: local_small_cell_rule.json:39; tools/generate_uk_local_target_references.py:596-597.
    • What goes wrong: the generator copies "rests on one or two sample households" and "one household's placement decides" into 598 signed_rationale fields on named LA cells (plus 3 deferral rationales) in the committed membership. It is an expected ratio from admin stock, not a record count, so it isn't a real disclosure. The wording is the form CD171 forbids, though.
    • Fix: reword it as an expectation, e.g. "about two household weights at 150,000 households, so its fit would be noise".

Minor

  • The sub-1e-7 L0 smoke test is too weak to catch the regime the 150k search depends on. CONFIRMED (1/1).

    • Where: packages/microcosm-calibrate/tests/engine_free/shared/test_solve.py:790-812.
    • What goes wrong: on its synthetic 400-record fixture, gate mass stays saturated (minimum probability about 0.999) from 1e-6 down, so the monotonicity assertion holds on a flat curve. It still catches non-finite values or a drop above 1e-3, and it matches the issue's wording.
    • Fix: seed with contribution_initialization on a fixture well below saturation and assert a strict rise. Better, run one fixed-λ probe at 2.5e-8 from the 100k size checkpoint before build 1.
  • Kept high-value homes LA cells ship per-cell estimates with no enforced 10-source floor. CONFIRMED (1/1).

    • Where: local_small_cell_rule.json:39; gates.json uk_high_value_homes_support (diagnostic); calibration_run.py:81-95.
    • It isn't a regression: main bound more small cells, and the release is private and ships the H5. The point is advice for any public LA high-value homes output: reapply the 10-source check there, or record a release-manifest limitation that gives the number of such cells below the floor.
  • The release disclosure walker misses several per-area shapes. CONFIRMED (1/1).

    • Where: contract.py:5061-5152.
    • Shapes that pass: code-keyed maps under a figure key, columnar tables, geography_id/code identifiers, max/median names, numbers in lists, most prose, and unparseable *.json (skipped silently). No current writer emits these, so this is defence in depth.
    • Commit 95ca093 lists exactly the shapes it refuses. Only the body's "A contract walker refuses them" overclaims.
    • Fix: broaden the walker to cover these shapes, and fail on unparseable JSON.
  • The dense contract can't confirm the national uk_target_fit and uk_input_mass_parity exclusion names. CONFIRMED (1/1).

    • Where: contract.py:7709-7767.
    • What goes wrong: dropping a national name from both the coverage and the manifest limitation still validates. The same edit on a local cell is refused. Only a hand edit triggers it, and the docstring admits the gap.
    • Fix: pin a digest of the two name sets in signed or hash-joined evidence.
  • The obr.ni_employer reason says "not a weight question" while about half the gap is unexplained. CONFIRMED (1/1).

    • Where: calibration_measure_exclusions.json:528.
    • What goes wrong: the stated parts (Employment Allowance, calendar window, under-21 rate) add up to roughly half of the excess, and the rest is "under investigation". The verifier adds that the rest may be a conflict between the bound pay targets and OBR's NICs base rather than a weighting defect.
    • Fix: reword it, keeping the five phrases the test checks.
  • The PR body is stale against the head. CONFIRMED (1/1).

  • The changelog says default-bracket callers "search as before". CONFIRMED (1/1).

    • Where: changelog.d/1301-l0-search-bracket.fixed.md:1.
    • A warm start outside [1e-7, 10] now raises where it used to clamp, and the warm path stops differently. No production caller is affected (the US passes target_records: null).
    • Fix: reword it.
  • The scorer reference-count test can no longer catch drift. CONFIRMED (1/1).

    • Where: test_uk_local_candidate_scorer.py:244-252.
    • Both sides read the same key. The 22,986 pin at test_uk_battery_bindings.py:1680 still guards drift.
    • Fix: pin it here or cross-reference that pin.
  • "Names and reasons only, never the measured error" isn't guaranteed. CONFIRMED (1/1).

    • Where: tools/assemble_uk_dense_release_dir.py:308.
    • The input-mass register reason, which includes national totals and a percentage, is copied into the manifest. There's no disclosure issue.
    • Fix: reword commit and docstring claims of this kind going forward.
  • A release-contract fixture pairs a real area code with plausible support figures. CONFIRMED (1/1).

    • Where: packages/microcosm-data/tests/engine_free/uk/test_uk_dense_release_contract.py:1391-1470.
    • Fix: use a placeholder code (E06999999) or the E06000001 fixture.

Cross-PR

Disclosure check

  • Clean: the PR body, all 10 commit messages and the 11 changelog fragments. They carry only national figures, published OBR/HMRC numbers, policy constants and counts derived from Land Registry, VOA and census data.
  • Clean: the added values in the generated JSON (references, memberships, both receipts, uk_population_targets.json, the census) are admin-derived stock or region-tier rows, not survey values.
  • Clean: the new Orkney and Shetland area-support entries, the empty local_target_fit_reviewed_exclusions.json, and the obr.ni_employer reason, which is national only.
  • Clean: area_support_gate, weakest-area diagnostics, support-limited misses and the manifest support block no longer emit per-area rows, ESS or sources.
  • Clean: the release goes to the private policyengine/populace-uk-private.
  • Issues found: the drafter's public outputs (the two Blocking items and the PRIVATE_review.csv guard); the small-cell rationale wording in local_small_cell_rule.json:39; the release walker's coverage; kept high-value homes LA estimates for downstream public use; and the fixture at test_uk_dense_release_contract.py:1391-1470.

Tests and tools run

Engine-free pytest on the touched surfaces:

  • Solve scope, high-value homes region control, and calibrate test_informed_gates.py and test_solve.py: 228 tests, 0 failures.
  • High-value homes region control and local cells, test_uk_local_doctrine.py, test_informed_gates.py and the two new bracket tests in test_solve.py: 50 tests, 0 failures.
  • Pin tests (gate-battery contract pins, country spec, spec-engine loader, target-reference authoring, population targets, small-cell rule, national calibration, target references, battery bindings): 376 tests, 0 failures, 5 skipped (feed-gated).
  • Graph parity pins, test_acceptance_h_parity, the US multispine pool tool, local target census, cross-grain declarations, uk-data target parity, local doctrine, solve scope and high-value homes: 363 tests, 0 failures, 1 skipped (needs a uk-data tree).
  • Calibrate test_solve.py and test_informed_gates.py, data test_contract.py and test_uk_dense_release_contract.py, diagnostics test_schema.py: 556 tests, 0 failures.
  • Solve scope, full build CLI, dense sign-off draft, graph terminal, local release preflight, dense release assembler, and shared staging and gates: 625 tests, 0 failures.
  • Battery bindings, full certification, measure simulation, high-value homes, local candidate scorer, diagnostics, rowwise candidate, size evaluation, full calibration graph and full build preparation: 338 tests, 0 failures.
  • Full targets, terminal gates, weighted integrity, target references and compile modules, population targets, national calibration, local doctrine and gate battery, small-cell rule, incumbent surface evaluation, the shared spec tests and the untouched test_uk_national_graph: 429 tests, 0 failures, 5 skipped (feed-gated, in test_uk_target_references.py).
  • test_acceptance_h_parity: 2 of 2. One test_us_multispine_pool_tool test could not import policyengine_us in the engine-free environment, an environment limit; CI engine-us passed it.
  • Total over the touched engine-free files: 1,948 passed, 5 skipped, 0 failures.

Spot runs:

  • test_uk_solve_scope.py, calibrate test_solve.py, test_uk_dense_release_contract.py and test_uk_dense_signoff_draft.py: 355 tests, 0 failures.
  • The disclosure, support and sign-off tests in test_uk_dense_release_contract.py and test_uk_dense_signoff_draft.py: 56 tests, 0 failures.
  • test_required_solve_reuse_on_an_empty_store_fails_at_the_preflight: 1 test, 0 failures.
  • The two --require-solve-reuse / gate-clock tests: 2 tests, 0 failures.
  • test_a_bundle_naming_every_signed_exclusion_validates: 1 test, 0 failures.

Repo tools in check mode:

  • tools/census_uk_local_targets.py --check: current.
  • tools/census_uk_data_target_parity.py --check: current.
  • tools/ci_test_plan.py verify: ok, 731 files.
  • tools/spec_engine_coverage.py --check: pass, 42,184/42,184 fields, 41/41 inventory checks.

Probes, on synthetic inputs only, covered: the L0 search with a stubbed optimizer, base against PR; the smoke test's mass curve; walker and _pairs shapes; _refuse_inside_repo; draft_target_fit's unscoped removal, with gates before and after; measure-exclusion expiry at 10-20, 10-22 and 10-23; a premature gate clock; dropping a national name from the signed exclusions; the national kernel's hashed modules; and the assembler's limitation summaries.

Other checks:

Checked and found sound

  • The US default is numerically unchanged: no non-UK caller passes target_records, and the default bracket is still [1e-7, 10]. US seed_map, seed_protocol and spec hashes move, so US nodes re-key; numbers don't.
  • The cold L0 path matches main in 5 of 5 stub runs.
  • The warm-path fixes are correct, and no penalty is probed twice.
  • An infeasible 150k draw fails closed.
  • The preflight pins the bracket at {1e-10, 10}.
  • The λ extrapolation checks out (log-linear about 1.3e-8, power law about 2.3e-8 for 150k), and the 2.5e-8 ±0.5-decade warm start covers it.
  • Solve-scope design holds:
    • The four gate-only registers are read only by terminal gates.
    • The preflight kernel is keyed on the solve fingerprint, and the calibrated gate kernel on the full one.
    • --gate-review-date changes only the calibrated node.
    • resume='require' preflights every node key up front.
    • The national role refuses both new flags.
  • Local references drop exactly 2,323 high-value homes cells (25,309 to 22,986) and nothing else.
  • National references gain exactly the 30 high-value homes region rows (1,291 to 1,321). Their values match the derived resource, and the regions add up to the country and the LAs.
  • The kept LA counts (192/101/57) and the constituency counts below 400 match the derived facts.
  • The national line drops the calibration_lines=["local"] rows on every national compile path.
  • Removed test assertions were replaced by equal or stronger ones. No new test is a date bomb.
  • test_uk_national_graph's invariant still holds at head: latest approval 10-16, earliest expiry 10-22.
  • The staging read-back fix doesn't swallow real failures.
  • The holdout and incumbent opt-in agrees across the contract, assembler, preflight and scorer.
  • The committed gate-only registers are in the drafter's canonical form, with one pin site each.
  • Nothing pins a digest of calibration_measure_exclusions.json, so the post-regeneration commit leaves nothing stale.
  • The obr.ni_employer engine claims check out against policyengine-uk 2.132.6 and Model the Employment Allowance so ni_employer is net of it (split from #1918) policyengine-uk#2459.

Refuted during verification

  • 1307-09 (the warm search's ceiling probe at λ = 10 over-prunes and leaves the search one probe poorer): the UK size search runs with free mass, so the claimed failure doesn't happen. Base and PR settle at the same penalty.
  • 1307-17 (solve reuse is never tested through the real kernels): test_a_gate_only_sign_off_rekeys_the_calibrated_gates_and_nothing_before (test_uk_full_build_preparation.py:158) covers it.
  • Codex claim 1 (gate clock bypass through --review-date): the behaviour predates this PR, and the new gate clock is bounded on both sides.
  • Codex claim 3 (the assembler checks only kind on the full gate report): the report is sha-bound to the manifest, and the preflight requires releasable.
  • Codex claim 4 (a cold L0 search never probes the bracket ends): the cold path is unchanged, release builds warm-start, and the scenario is contrived.
  • Codex claim 5 (the incumbent score receipt isn't bound to the registry): it's moved code, now an opt-in diagnostic, and needs a hand-edited receipt.
  • Codex side claim that rowwise_geography's expected_rows reaches a dense release: it is used only by offline tools.

Not covered

  • The engine-UK tests on the UK local release: publish a K=15, 150,000-household dense dataset in at most two builds (#1301) #1307 tree (test_uk_full_graph_admission, test_uk_full_target_graph, tests/engine/uk/test_uk_graph_terminal). The review had no engine environment for this tree; CI engine-uk passed them.
  • A real two-build solve reuse against an actual graph store, and whether the re-keyed calibrated node reproduces build 1's per-row errors exactly, which the drafter's stale-risk rows depend on. Both need builds.
  • The body's claim that the feed-gated regeneration tests passed locally on 735820c. Running them would load the whole feed.
  • Whether the kept high-value homes LA cells and the two sub-floor region rows rest on at least 10 distinct source households (1307-03, 1307-12). Needs a build.
  • Downstream readers of UKAreaFit's now-optional support fields in other repos. Searched, not exhaustively.
  • --logbook-prev-row-digest chaining for build 2.

🤖 Generated with Claude Code

juaristi22 added a commit that referenced this pull request Oct 17, 2026
The dense release contract's disclosure walk refused three shapes of
per-area support figure: a figure under its name in a record with an
area_code or beneath an area key, a min_ or minimum_ name outside the
floors, and "rows 7 < 50" text. The #1307 review found shapes that slipped
past it: a code-keyed map under a figure's name, a columnar table, a
record identified by geography_id or code, figures in a list, max_ and
median_ names, and any *.json that did not parse, which the walk skipped.
No writer emits these today, but the walk is the last check before a
release leaves the machine, so it should not depend on today's writers.

The walk now refuses an area's rows, Kish ESS or distinct sources, as a
number or as numeric text, in a record identified by area_code,
geography_id or a code that holds an area code; under an area key beneath
the figure's name, through level keys and lists; in a table with an area
column, as parallel lists or pandas {row: value} columns; and as a
minimum, maximum or median of a figure, however nested, outside the
declared floors. A shipped *.json the walk cannot parse, decode or walk
now fails rather than being skipped. Failures still name where a figure
is, never its value. The text rule is unchanged.

The refusal tests' fixtures use placeholder codes (E06999999, E06999998)
where they paired a real local authority with plausible support figures.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22

Copy link
Copy Markdown
Collaborator Author

Response to the adversarial review

Eleven commits on top of ea68813, ending at b10840d. Every finding is either fixed or ruled on below. The touched tests pass locally, the CI test plan verifies, and ruff is clean. The PR body has been rewritten to match: the commit list, the build-2 recipe, the scope of obr.ni_employer, and the merge order.

Blocking

  • 1307-01, local fit bands in PR-B's body: fixed (6c843dd).
    • PR_BODY.md no longer prints Stale-risk or Near-bound for the local registers, not even present/absent.
    • The national register's counts appear in one sentence.
    • Local counts go only to stdout and PRIVATE_review.csv.
    • The body's disclosure line lists exactly what the body carries.
  • 1307-02, free text: fixed (6c843dd).
    • --class-note, --area-name and --adjudication may carry digits only inside issue references and ISO dates. % is still refused.
    • --approved-by must be a GitHub login.
    • The checks run before the build is read, and the review's probe shapes are tests.
    • The runbook and changelog now say what the two checks cover and miss. The manual scan covers signoff.patch as well as PR_BODY.md.

Should fix

  • Sub-floor region rows (North East and Wales, £2m): not pre-signed (a4e0109).
  • An unscoped national approval deleted by the draft: fixed (c0e0f9a).
    • An all-lines uk_target_fit entry is never dropped. It is rescoped to ["national"], or, when it is out of its window on a row that still breaches, kept rescoped and sent to a hand fix.
    • --verify --draft-receipt checks that the rescope was committed as drafted.
  • Measure-exclusion expiries: fixed (c0e0f9a).
    • The draft reads build 1's receipted measure exclusions from the candidate manifest, or from the target compilation's surface when a structural gate stopped the manifest.
    • It also reads the degenerate register's local-line entries and the gate-only windows.
    • It caps the gate clock range and prints the earliest expiry as the assembly deadline, with the entries that set it.
    • --verify warns two days before the deadline and fails after it.
  • --require-solve-reuse without a usable gate clock: fixed (be7cb0e).
    • It refuses before the replay when any in-line gate-only entry is premature at the gate clock, and names the earliest gate clock that works.
    • An unsplit clock is still allowed when nothing is premature on it.
  • The build-2 recipe: fixed (be7cb0e, c0e0f9a, the body).
    • --require-solve-reuse also refuses without an explicit --graph-store, and refuses an --out that already holds a build.
    • The runbook, the graph doc and the body pass --review-date and --graph-store explicitly in build 1 and repeat them in build 2 with a new --out.
    • Build 2 chains --logbook-prev-row-digest to build 1's row digest. The Logbook refuses two rows on one predecessor in the uk/local scope.
  • obr.ni_employer on the national objective: stated (8f17654, the body).
    • The adjudication, changelog and body say the exclusion applies on both lines.
    • They also say the national build's frozen scoring register must be derived from a head that carries it.
  • measure_simulation in the national kernel: fixed (8c4b670). _NationalKernel now hashes it, with a test.
  • PRIVATE_review.csv inside another worktree: fixed (6c843dd). Any --out inside a git work tree is refused, in both draft and verify. .gitignore covers PRIVATE_review.csv and signoff-draft/.
  • The small-cell rationale wording: fixed (a4e0109).
    • The rationale is now an expectation: "about two household weights at the ruled 150,000 households, so its fit would be noise".
    • The membership was regenerated on 735820c. Only rationale strings moved; references, the local receipt and every count are byte-identical.

Minor

  • The sub-1e-7 smoke test (a3a2f87).
    • The gates are now seeded with contribution_initialization, below saturation, and the test asserts at least 10 records of open mass per decade from 1e-7 to 1e-10.
    • Mutations it catches and the old test missed: a solver that floors the penalty at 1e-7, and a larger Adam eps.
    • The per-target loss scales mimic a large pool; the test comment says why.
  • High-value homes LA estimates and the 10-source floor: agreed as advice for public outputs. The dashboard's disclosure layer applies it. Nothing to change in the private release.
  • The release disclosure walk (66c871e). It now covers code-keyed maps, columnar tables, area-code records, numbers in lists and min/max/median names, and it fails on any shipped JSON it cannot parse. The body no longer overclaims.
  • National exclusion names (4e7beba).
    • The candidate manifest now records the names each signed-exclusion gate stood on, and the contract refuses coverage or limitations that differ.
    • The manifest is unsigned, so this gives the same assurance as its measure-exclusion, holdout and size claims. A signed binding would need the gate-report replay to carry the names; that is not done here.
  • obr.ni_employer reason: reworded (8f17654). The stated parts explain roughly half of the excess; the rest is in the pay level. The five tested phrases are kept.
  • Stale PR body: rewritten.
  • L0 changelog: reworded (b10840d).
  • Scorer reference count: pinned literally (b10840d), with a pointer to the battery-bindings pin.
  • "Names and reasons only": reworded (05e31d3). Reasons are copied verbatim and can carry national figures.
  • The fixture's real area code: now placeholder codes (66c871e).

Cross-PR

🤖 Generated with Claude Code

@juaristi22

Copy link
Copy Markdown
Collaborator Author

Re-review of this PR at b10840d

Run with Claude Code on 17 October 2026, after the eleven commits answering the first review, the response comment and the rewritten body.

  • How: five independent reviewers re-checked every first-round finding and every claim in the response, attacked the new code, ran the touched tests, checked the new public text against the UK Data Service rules, and simulated the merges with Re-pin the UK Chronicle feed to abf0921 (#1300) #1302 and the stacked PRs.
  • Not run: builds and feed-gated tests.

Verdict

Not ready for build 1. Almost every first-round finding is fixed, and CI is green (10/10). But one fix, 66c871e's widened disclosure walk, now refuses every real dense release at assembly. There is also a disclosure question about the drafter's national counts that needs María's ruling before build 1.

Blocking

  • The widened release disclosure walk refuses real dense releases (high).
    • Confirmed independently by two reviewers and again on the committed code.
    • Where: packages/microcosm-data/src/microcosm/data/contract.py:5085-5088, _UK_SUPPORT_EXTREME_KEY_RE:
      ^(?:min|minimum|max|maximum|median)_[a-z_]*(?:households|rows|sources|sample_size|ess)$
      
      The prefixes max and maximum are new. The ess ending also matches words ending in "excess".
    • The real writers put two such keys into the shipped rowwise_candidate_manifest.json, which is required and walked:
      • cross_grain.subset_relations[i].max_excess and solve.cross_grain.subset_relations[i].max_excess. Every tested relation writes one (national_reconciliation.py:745,824 → graph_targets.py:325/352/719 → graph_terminal.py:1764/1916). On main since 77c7acf (10-11).
      • identity.spine.spine_provenance.spine_gate_report.payload.gates.uk_stage_was_wealth_coherence.details.maximum_owner_share_without_main_residence_excess. This is a gate parameter echoed in the details (stage_health.py:2412). On main since a2ddbb9 (10-01).
    • Evidence:
      • The committed reconciliation extract (docs/evidence/uk-target-surface-1123/reconciliation.json), placed as the manifest places it, gives 0 refusals at ea68813 and 40 max_excess paths at b10840d.
      • The spine provenance from two earlier local dense runs passes the old walker and fails the new one.
      • End to end through the contract tests' own write_dense_bundle, it gives one contract failure.
    • Why CI is green: the fixtures' candidate manifests carry no subset_relations or spine gate report. --verify walks only the signed local report, and the preflight never walks.
    • Effect: build 2 completes, then assembly refuses (tools/assemble_uk_dense_release_dir.py:1084; upload validates again).
    • Fix:
      • Anchor the pattern to whole support-figure stems, e.g. …_(?:nonzero_households|nonzero_source_households|assigned_households|rows|sources|distinct_sources|effective_sample_size|ess)$ with ess as a whole token. Or require an area context.
      • Add a contract test that walks a real-shaped candidate manifest, with subset_relations and a spine gate report.
      • Ideally, have the preflight or --verify walk the run directory's to-be-shipped JSON, so a false refusal shows at build 1.
    • Timing: contract.py keys only the certification kernel, so fixing it before build 1 costs nothing. Assembly, upload and consumers each validate with their installed microcosm-data, so the fix has to ship in this PR.

For María's ruling (disclosure; nothing is published yet)

  • Public sign-offs of the two below-floor high-value homes region rows (medium).
    • The rows are North East and Wales at £2m (over_2m_by_region@E12000001 / @W92000004).
    • Both are below the PR's 400-home floor, which its rule text puts at about two household weights at 150,000 households, so each row's weighted total would be expected to rest on fewer than 10 unweighted records.
    • If build 1 breaches one, the drafter signs it as a national uk_target_fit row. If the breach is within a point of the bound, the public PR_BODY.md prints "National target-fit rows: 1 stale-risk" (synthetic probe: stale_risk_count = 1 for that key). With the key committed, that places the row's weighted estimate within one point outside the 25% bound, against its published target.
    • Even without the count, the committed reason would say the estimate is more than 25% off its target.
    • The same reasoning applies to kept high-value homes LA cells between 400 and about 2,000 homes (about 10 weights). It also applies to local-cell entries in general, whose committed reasons say the cell breached the 25% bound, and the response doesn't give a ruling on that.
    • Options:
      • exempt rows below the floor from public gate-only sign-off by a declared rule, and assess them privately;
      • pool the two region rows;
      • have the drafter keep the national counts out of PR_BODY.md whenever a signed row is below the floor.
    • Decide before build 1.

Should fix (low)

  • The free-text rule (1307-02) is only partly fixed.
    • _ISSUE_REFERENCE_RE (tools/draft_uk_dense_signoff.py:177-179) treats any #N or word#N as an issue reference. Synthetic probes accepted:
      • household #4521 carries weight #312
      • record #1234 decides the cell
      • #7 rows against the #50-row floor
      • household 1234-05-06 (date-shaped)
      • #25%
    • The login regex (:182) accepts 4521 and rows-7-of-50.
    • In the other direction, it refuses legitimate adjudication text: microcosm#1301 (issue comment 6232359992), …#issuecomment-… and full GitHub URLs (the repo's usual citation), K=15, 150,000 households, plan codes like c9 and R5, and a bare 2025.
    • Fix: allowlist repo prefixes (microcosm#, PolicyEngine/<repo>#), refuse a bare #N, and allow issuecomment-\d+ anchors. The manual scan stays the backstop.
  • The pre-replay guard checks "premature", not "expired".
    • Where: rowwise_cli.py:627-645 and :707-727.
    • An in-line entry in force at the solve clock but expired at the gate clock is admitted. The gate then fails it after the replay.
    • The degenerate register isn't checked, and the upper end of the drafter's gate-clock range is printed but never enforced.
  • The occupied---out guard keys only on build.json.
    • Where: full_build_cli.py:1406.
    • A failed build-2 attempt leaves failure.json and a Logbook spool row, and a retry into the same --out is admitted.
    • --verify then refuses the run dir ("the Logbook spool names 2 attempts"), and the assembler picks rows[-1] by digest order.
    • Fix: also refuse an --out with failure.json or any spool row.
  • be7cb0e's tests don't pin the line rule for three of the four registers. With _in_line dropped for the local target-fit, input-mass and area-exclusion registers, both of its tests still pass.
  • The drafter's deadline and gate-clock range disagree with the gate (latent).
    • National-scoped bound_despite_support_floor entries are dropped (tools/draft_uk_dense_signoff.py:2328-2331, :2502-2506), though the gate checks all of them (battery_bindings.py:1317-1330).
    • Input-mass entries are left out.
    • Nothing is affected today: the only bound_despite entry is unscoped, and input-mass entries expire April 2027.
  • The walker also refuses regional and national aggregates (latent).
    • _UK_AREA_KEY_RE (contract.py:5076) matches region, nation and UK GSS codes, and an area_code/geography_id key marks any mapping as an area.
    • So {"ess": {"E12000001": …}} is refused, which CD171 allows at GOR level. No current writer emits it.
  • The walker still lets several shapes through. Prose ("…has 7 rows"), variant names (n_rows, upper-case ESS, smallest_/lowest_, rows_min), nested {figure: {min, p10, median}} summaries, non-JSON files, and files outside sha256sums.txt. That's defence in depth, but the body's claim is broader than the walk.
  • A real LA code is paired with synthetic support figures in new lines.
    • tests/engine_free/uk/test_uk_dense_signoff_draft.py:897 and :936-937, and the contract.py:5127 docstring.
    • The existing SUPPORT/LOCAL fixtures in the same test file do the same.
    • The response says "now placeholder codes", but that only covers test_uk_dense_release_contract.py. Placeholders (E06999999/E06999998) would be safer here too.

Nits

  • docs/uk-full-build-graph.md:160-166's build-2 command omits --logbook-prev-row-digest; the runbook and body include it. Build 1 already passes one, so for build 2 "add" really means "replace".
  • --verify --draft-receipt checks that a rescoped entry is national-only, but not that it is otherwise unchanged.
  • The runbook's PR_BODY.md contents list omits "national entries for review".
  • changelog.d/1301-uk-local-release-pins.changed.md and body item 9 say 735820c. They become stale at the planned regeneration on abf0921.

First-round dispositions

Most are fixed in code; the reviewers verified each one.

  • Fixed:
    • 1307-01: no local bands in PR_BODY.md.
    • All-lines uk_target_fit entries are rescoped, never dropped.
    • measure_simulation is hashed in the national kernel.
    • The merge order is stated.
    • obr.ni_employer is stated to apply on both lines.
    • The build-2 recipe and its refusals.
    • The sub-1e-7 smoke test: the two named mutations now fail it.
    • An --out inside any git work tree is refused.
    • The small-cell rationale wording: the membership changes only 2,323 signed_rationale + 6 rationale strings, and references, both receipts and the census are byte-identical.
    • The national exclusion names are bound (unsigned, as stated).
    • The texts, the L0 changelog, the scorer pin, the verbatim-reasons wording and the contract tests' fixture codes.
    • The assembly deadline and the premature gate clock.
  • Decided and documented: 1307-03 (the two below-floor region rows), with the outcome depending on build 1. María's ruling that per-cell estimates ship in the private release appears only in the response comment, not on UK local-area release: publish a K=15, 150,000-household dense dataset in at most two builds #1301.
  • Partly fixed: 1307-02, the free-text rule (above).
  • Fix introduces a new problem: the widened walker (Blocking above).
  • Not addressed, deliberately: 1307-12 (advice for public outputs).

Cross-PR

Disclosure check

No breach in what changed since the first review: the eleven commit messages and their added lines, the runbook, the changelog, the register reasons, the regenerated rationale strings, the body and the response. The forward-looking question is above.

Tests and tools run

  • At b10840d:
    • 1,404 engine-free tests, 0 failures, 6 skipped.
    • 409 more on the touched files and the solve-reuse subset, 0 failures.
    • tools/ci_test_plan.py verify ok; tools/census_uk_local_targets.py --check current.
  • Mutations: the smoke test catches the 1e-7 floor and the Adam-eps mutations; be7cb0e's line-rule mutation survives.
  • CI at b10840d: 10/10 green.
  • Engine-UK tests were left to CI.

🤖 Generated with Claude Code

juaristi22 and others added 22 commits October 17, 2026 15:14
…1301)

A K=15 dense release has to be publishable before the Budget in at most
two builds. The five-fold rotated holdout repeats the dense solve five
times, and no candidate has ever passed the incumbent-surface
assessment, so both stood between a green gate battery and a release.
They become report-only diagnostics, and the release says what it does
not claim instead.

--rotated-holdout replaces --skip-holdout, with no alias: a dense build
skips the holdout unless asked and records the skip, and the national
line, which has no holdout node, refuses the new flag and keeps
recording skip_holdout false. The pre-flight accepts an unrequested
holdout and still wants five folds and a finite mean when one was run.
The scorer reads the active reference count from the membership file
and scores a candidate with no holdout. The size evaluation runs its
incumbent steps only behind --incumbent-diagnostic. Certification
readiness stops gating on the scorecards or a skipped holdout.

The contract's dense check is split so that the candidate's own joins
(dataset, diagnostics, gate report, clean code, Ledger pins, measure
exclusions) always run; previously they lived behind the incumbent
files and would have been skipped with them. The three incumbent files
ship all together or not at all. Their identities and the pinned row
inventory refuse; their measured limits are recorded in the manifest,
not enforced. The coverage carries holdout.claim, incumbent.compared
and a size block, and the manifest's limitations must match them. The
assembler takes --incumbent-manifest as optional, requires the full
gate report, from which it names the reviewed exclusions the gates
stood on (names and reasons only, never the measured error), and prints
a --tag-only publish command.

The holdout node's code hash moves, so the dense solve re-keys; that is
intended before build 1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A dense release ships the signed local gate report, both diagnostics
files and the candidate manifest, and all four carried per-area support
figures: an area's rows, Kish ESS and distinct source households. The
area-support gate put them in each reviewed exclusion and in its failure
lines and reported each level's minima; the weakest-area diagnostics and
the support-limited misses listed them per area; the manifest's support
block held the minima. The signed report cannot be redacted after the
build, so the gate itself now names only the floors an area misses,
records a reviewed exclusion as its shortfalls and its signed record, and
counts the areas checked and below the floor by level. The diagnostics
rows and the support-limited misses carry support_below_floor instead,
UKAreaFit's figure fields become optional so earlier diagnostics still
parse, and the manifest's support block counts areas. The figures stay in
the run's area-support table, which no release ships. Weights and
per-cell estimates ship as before.

The dense contract walks every shipped JSON document, the release
manifest's strings included, and refuses a support figure in a per-area
record, a minimum of one outside the declared floors, or a shortfall
worded with its measured value. A refusal names where the figure sits,
never its value.

The full gate kernel now also hashes gates and terminal_gates, so an edit
to a gate evaluator or a register loader re-keys the gate nodes. This
commit re-keys them, which is intended before build 1.

After a passed read-back, the best-effort refresh upload could fail and
set UPLOAD_FAILED, which the delivery contract refuses beside a passed
read-back. The refusal escaped the dense driver before it recorded the
staging delivery and spooled the Logbook row, so the build exited 1 and
the assembler then refused the run. Once read-back has given its verdict,
the verdict owns last_error_code; a failed refresh still shows as an
upload attempt without a success.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…#1301)

The local size doctrine asked for 100,000 households, which K=15 cannot
reach: at the solver's L0 floor of 1e-7 the size search's open mass fell
short, and the 100,000-household attempt then spent its third probe
re-running 1e-7 on a bracket that had collapsed onto the floor. The #1124
ruling, as amended for #1301, sets 150,000 households at K=15 with the
draw, refit floor and refit penalty unchanged, and names only the gates
and the pre-flight as publication requirements; the holdout and the
incumbent comparison are opt-in diagnostics since the first #1301 commit.

calibrate() takes a budget_search_bracket whose default stays [1e-7, 10],
so the US contract and every other caller search exactly as before. The
UK size search passes its own bracket, from the new
UK_LOCAL_L0_SEARCH_FLOOR (1e-10) up to the solver's ceiling, through
select_uk_dataset_size's default, which the holdout folds inherit. The
floor is a parameter of the search, draw, refit and holdout nodes, since
the doctrine module is not in their code hash, and the size receipt
records the bracket the search ran in, read from the search's own
receipt; the pre-flight checks it on a sized run.

The search itself refused nothing and wasted probes. A warm start outside
the bracket was clamped to the bound while the receipt recorded the value
asked for, so --selection-initial-lambda 2.5e-8 silently became 1e-7; it
is now refused, in the solver, in the size module, in the graph config and
when the command is parsed, together with a warm start given without
--dataset-households. A warm start is probed at exactly the value given.
No penalty within 1e-6 decades of a measured one is probed again. A probe
on the floor or ceiling that asks to go past it stops the search as
search_floor_reached or search_ceiling_reached, and a bracket narrowed onto
a measured penalty as bracket_exhausted; both receipts record the stop and
the bracket. The warm edge probe follows the first probe's own steer: a
warm start on the floor that asked for a smaller penalty was sent half a
decade upward.

Editing solve.py, dataset_size.py, local_doctrine.py and full_measure.py
re-keys the dense solve, the size nodes, target compilation and the
holdout, which is intended before build 1. The calibrate parity pins and
the US spec-engine pins that attest solve.py move with the final pin pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A failing local cell or a below-floor area found in build 1 has to be
signable before build 2 without changing the solve; otherwise the sign-off
re-keys and re-runs the whole dense build. Neither could be:
uk_local_target_fit read no register at all, and the local reference
generator built its support-floor cell deferrals from
local_area_support_exclusions.json, so signing an area there would also
have deferred its cells.

uk_local_target_fit now reads its own register,
uk/local_target_fit_reviewed_exclusions.json, committed empty and keyed by
the gate's family/area_code/metric cell names. An in-force entry lifts the
fence for its cell only and the cell stays bound in the solve; stale,
expired and not-yet-in-force entries fail. The gate evaluates entries on
the review date the other register-reading gates share rather than the
wall clock, honours release_lines, and records evidence_absent without a
review date. The size evaluation keeps signed cells on its red-row ledger,
since they still breach the bound.

The area-support exclusions become gate-only. The binding honours
release_lines, so an entry for another line stands aside unchecked, while
the bound-despite-floor approvals stay window-checked on every line. A
constituency key is honoured like a local-authority one, and the assembler
names each exclusion's own grain in the release limitations. The Isles of
Scilly and City of London cell deferrals and the families bound despite
the floor move into the generator as SUPPORT_FLOOR_DEFERRED_AREA_IDS and
SUPPORT_FLOOR_BOUND_FAMILIES. Lockstep tests keep the deferred areas inside
the register's local-authority exclusions and the bound families equal to
its approvals, and a perturbation test signs synthetic entries at both
grains and checks that the generator's deferrals do not move. The committed
local references and membership regenerate byte-identical from the pinned
feed.

The gates.json edit moves the gate-battery contract digests, which are
re-pinned with the other derived pins in the final pin pass. The new
resource joins the country package and so re-keys target compilation and
everything below it, which is intended before build 1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Most high-value homes cells at constituency and local-authority grain sit
below about one household weight at the release size, so the family needs
a control above them. Three two-level targets
(microcosm.high_value_homes.over_{1m,1_5m,2m}_by_region) bind the derived
Land Registry stock at the nine English regions and Wales. They copy the
local targets' measurement block and record-set selector, so each row is
the cross-grain control for its area cells and the three no_higher_control
local routes become stale and go.

The rows must not bind on the national line: there main_residence_value
is the WAS donor's, not a Land Registry placement. A new contract key,
calibration_lines, names the lines a target binds on. The region rows
declare ["local"]. load_uk_national_target_inputs, the release-cut
certifier's calibration reference and the UC matrix diagnosis drop them
right after the compile, before reconciliation, the measure exclusions and
the frozen-register check, so the national registry version is unchanged;
register_completeness.calibration_line receipts the drop on both lines.

Every national compile now merges the derived facts as the local ones
already did (both full_targets loaders, the certifier, the parity tool's
national receipt with an HVCTS ledger_only rationale, the national
reference generator, the incumbent-surface evaluation), so all committed
references compile on every line. The generator's region-tier fan-out now
honours area_scope by level, giving Wales at country level and no
Scottish or Northern Irish cell, and records a fan-out outcome.

The bound values are per-household measures in measure_simulation, read
like the bus boardings: the stored main_residence_value and tenure_type
through high_value_home_mask at each threshold's frame-basis value, as the
local metrics and the release gates read them. A persisted column of any
of them is refused.

The regenerated references, membership, parity receipts, census and
registry_parity accounting follow in the pin commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every register edit that changes the compiled problem has to be on the
branch before the first local release build, because the second build
reuses that solve and may change only gate-only registers.

obr.ni_employer becomes a measured diagnostic in the calibration measure
exclusions. The engine's ni_employer is gross of the Employment Allowance,
which OBR's receipts are net of, and the row's calendar-2025 window takes a
quarter of FY2024-25 that the engine prices on FY2025-26 rules. Modelling
the allowance alone would close only part of the gap. The entry tracks
policyengine-uk#1918 until the drafted issue that splits the allowance out
of it is open, and takes the three-month window of the obr.vat
engine-factor diagnostic. The UC support tool's protected probe stops
asking for the row, which a new run's matrix no longer carries.

The high-value homes region rows now control that family on the local
line. The local reference generator therefore defers its constituency
level whole (high_value_homes_constituency_grain_deferred), after the
English band-H precedent, with a roster guard on the 575 England and Wales
seats. A new signed small-cell rule, hvcts_la_household_floor, defers the
local-authority cells under 400 homes, about two household weights at
150,000 households. In the generator's second pass the rule finds no
active constituency cell, and the region rows are not local targets, so it
reaches the authority cells only. The family's binding adjudication now
records where it binds, and that the seat rank correlations, out of sample
once the seats are deferred, decide its renewal. The three
no_higher_control local routes already went with the region control.

Orkney and Shetland are pre-signed gate-only exclusions of the
area-support floor, scoped to the local line, so the first build needs no
sign-off for them. Their cells stay bound and the solve does not move.

The regenerated local references, membership and compile-parity receipt,
and the counts that move with them, follow in the pin commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The local release is built at most twice: build 1's failing cells and
below-floor areas are signed in gate-only registers, and build 2 at the
sign-off's head must recompute only the calibrated gates, the export, the
package and certification. Today every UK register feeds the country
fingerprint, which the target kernels, the preflight, the holdout and the
spine gates hash, so a sign-off re-keys everything from target compilation
down and build 2 is a full rebuild of 10 to 16 hours.

Those nodes now key on a solve fingerprint,
ResolvedCountrySpec.fingerprint_excluding over every packaged resource but
uk_runtime.solve_scope.UK_GATE_ONLY_RESOURCES: the target-fit, local
target-fit and input-mass registers and
local_area_support_exclusions.json, which the gate-only area-support
change left without a solve reader. The excluded names are bound under
their own domain tag. The source preflight runs under its own kernel,
uk.full-gates.preflight@1, because the dense solve takes its report as an
input; the calibrated gates keep the full fingerprint, so a sign-off
re-keys them and nothing before them. Each gate kernel refuses the other's
phase. The degenerate register stays solve-scoped, since the release
contract pins its digest, and the national graph is unchanged.

--gate-review-date gives the calibrated gates their own clock, by default
--review-date. It may not precede the solve clock or, when given, be later
than today, and only the calibrated node carries it, so a later sign-off
date re-runs the gates alone. The export bindings record both clocks.

--require-solve-reuse replays the spine and preflight phases and then
every strict ancestor of uk.full.gates.calibrated with resume="require"
before calibration, and turns a store miss into SolveReuseError naming the
missing nodes, so a build 2 whose inputs drifted fails in minutes instead
of silently re-solving. It is refused with --resume forbid, and the
national role refuses both flags.

test_uk_solve_scope.py holds the scoping sound from both sides. An AST scan
of the build source and tools allows only reviewed modules to name a
gate-only register, and none of the target-compile, solve, spine or
generator modules. Every gate that reads one is a terminal gate. Editing
each one leaves every solve kernel's hash unchanged while the calibrated
gates' moves, and a solve-scoped edit moves them all. The preparation test
checks the same on the node keys of a real prepared graph, and the engine
admission test traps every gate-only read through target compilation and
the preflight. Node keys from target compilation down move once; no pin
does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A dense release candidate blocked only by national target-fit rows, local
target-fit cells or areas below the support floor can be released by a
second build once those failures are signed in gate-only registers. Writing
that sign-off by hand from a blocked build is slow and easy to get wrong:
an entry that is premature at build 2's gate clock, a stale entry left in
place, or a figure copied into a reason.

tools/draft_uk_dense_signoff.py reads build 1's run directory (the full
gate report, the signed local gate report, the target and area-support
tables and the Logbook row) and checks them against one another and
against the checkout's gates.json. It drafts release_lines ["local"]
entries for every signable failure, the removal of every entry build 2
would fail and the register pins (test_uk_terminal_gates.py for the two
target-fit registers, test_uk_weighted_integrity.py for the area
exclusions), as a git-apply patch with a counts-only PR body. Any other
release-blocking failure makes the build unsignable.

The local reference generator keeps its own support-floor constants,
lockstep-tested against the area-support register, so the tool never edits
the bound_despite_support_floor block and never retires or renews the
exclusions of the areas whose cells the generator defers. If build 2 would
fail one of them, the area gate is reported unsignable with what to do: a
stale entry needs the generator, and so the solve, to move; an expired one
is renewed by hand. It never touches the degenerate register either, whose
digest the contract pins.

Reasons cite the build-1 attempt id and the full gate report's sha256,
carry no figures and pass the release contract's disclosure walk, since
build 2's signed local report ships them. Per-row errors and per-area
support figures go only to a private review file outside the repository.
A checkout whose gates.json wires no local target-fit register reports
local cells as unsignable.

--verify checks that build 2 stood on exactly the committed entries, that
no release-blocking gate failed and that its signed local report passes
the dense contract and its disclosure walk. With --build-1 it also reports
whether build 2 reused build 1's solve.

The three fixed-clock target-fit tests in test_uk_terminal_gates.py now
read the register as the national line does, so local-line sign-offs
approved after their clock leave them unchanged. The runbook's
between-builds section now holds the second-build steps too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The earlier #1301 commits changed what the committed surfaces derive from
and left every regenerated file and moved pin to one pass, so that each is
computed once, from the final code, against the pinned 735820c feed.

The high-value homes region targets join the contract's registry-parity
scope as unmapped declarations, since uk-data has no such metric. The
national references and membership gain their 30 region-tier cells, which
the national line still drops after the compile; the national
compile-parity receipt records them as ledger_only. The local references,
membership and compile-parity receipt lose exactly the high-value homes
cells that the constituency-level deferral and the signed 400-home
authority floor now defer, and nothing else; the local census names the
region rows in the family's fence, description and source. Under the
doctrine's target-weight rule the compiled local-line surface still meets
both family loss-share floors.

The gates.json edit that wired the local target-fit register moves the
gate-battery policy, manifest and fingerprint pins and the dense line's
gate digests; the spine, calibration-seam and release-cut part digests are
unchanged. calibrate()'s budget_search_bracket moves the US seed-protocol
and seed-map hashes, the spec-engine coverage attestation, the loader's
golden vector, the multi-spine pool tool's spec hash and the calibrate
parity case's implementation hash and node key, re-pinned on
arm64/darwin/py3.14; its pinned bytes are unchanged. The count pins move
to the regenerated files, and the local generator's contract-refusal test
gets a constituency roster the new 575-seat deferral mask accepts, as it
already had for the council-tax masks.

National cuts certified before this commit fail validate_release_dir at
the new head, as with every gates.json edit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The engine issue that splits the Employment Allowance out of
policyengine-uk#1918 is now open as policyengine-uk#2459, so the
obr.ni_employer measure exclusion tracks it, as its adjudication said it
would once the issue was open. #1918 keeps the under-21 and apprentice
zero rates.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s sign-offs (#1301)

--require-solve-reuse admitted three requests that cost the two-build
release its second build, and each surfaced late. Without --graph-store
the store defaults to <out>/.graph-store, and build 2 writes a new --out,
so the store is empty and the build stopped with SolveReuseError only
after preparing its inputs. Pointed at build 1's --out, nothing stopped
it: publication replaced build 1's bundle, the one its sign-off was
drafted from and that draft_uk_dense_signoff.py --verify --build-1 reads.
And without --gate-review-date the calibrated gates evaluate on build 1's
--review-date, so every sign-off approved since fails as premature, but
only after the solve replay, and a third build follows.

validate_cli_args now refuses, before anything runs, a solve-reuse build
without --graph-store or with one that is not a directory, and one whose
gate clock (--gate-review-date, else --review-date) precedes the
approved_on of any entry the calibrated gates would hold in force. The
refusal names the earliest gate clock that puts them all in force. Those
entries come from the four gate-only registers through the gates' own
loaders and line rule (battery_bindings._in_line, on the local line),
plus the area-support register's bound_despite_support_floor approvals,
which that gate window-checks on every line. An unsplit clock stays
admitted whenever nothing is premature on it, and a build that reuses no
solve is unchanged. The dense attempt refuses an --out that already holds
a bundle (its build.json completion marker) before the attempt opens, as
the national line refuses an occupied --out, so nothing is written into
build 1's directory; dry runs are not refused.

rowwise_cli becomes a reviewed reader of the gate-only registers, limited
to gate_only_signoffs, and a lockstep test holds that function to
UK_GATE_ONLY_RESOURCES. The solve-reuse tests stand synthetic registers in
for the committed ones, so none depends on what is signed on the day it
runs, and the replay test now follows the recipe: build 1's store and a
new --out.

The two-build recipe in docs/uk-full-build-graph.md now has build 1 pass
--review-date and --graph-store, and build 2 repeat both with a new
--out, --gate-review-date and --require-solve-reuse; the guard lists the
new refusals.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The national targets apply their measure exclusions, and the national
problem resolves its measures, in measure_simulation, but
_NationalKernel's implementation hash named full_targets and full_measure
and not the module they call. An edit to the measures alone therefore left
the national targets, the problem and the nodes after them as cache hits
computed by the old code. The dense target kernels already hash the module.

Leaving it out would not have kept the national graph's keys: they move at
this head anyway, since full_measure, full_targets, ledger_targets and
local_target_census changed and so did the country-spec fingerprint. So
the fix costs no extra cache miss. A test like the dense one holds both
national kernels to the module.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The sub-1e-7 smoke test ran on gates at the default init_mean, which stay
near 0.999 from 1e-6 down, so its open mass crept up by under a hundredth
of a record a decade and the monotonicity check held on a flat curve. It
also passed when the solver lifted every penalty to 1e-7, the floor the
size search now goes below, and when Adam's eps was raised to 1e-6, which
stops penalties below 1e-8 from steering.

The test now seeds the gates with contribution_initialization, as the UK
size search does, which starts them at an open mass of about 210 of 400,
and asserts that each decade from 1e-7 to 1e-10 opens at least ten more
records of mass. Informed gates alone do not get there on 400 records:
each record carries about 1/400 of a target, a pull on its gate that
swamps penalties this small, and below 1e-7 the steps sit within seed
noise, with a step falling on three of four seeds. Loss scales 3e5 times
the targets cut that pull to about 1e-8, and at the UK solve's learning
rate of 0.15 the open mass runs 3.5, 42, 211 and 247 across the four
penalties on this machine. Over 20 optimizer seeds on two fixture draws
the smallest step was 32. Both mutations above now fail the test.

Test-only: solve.py and the calibrate parity pins do not move.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The dense release contract's disclosure walk refused three shapes of
per-area support figure: a figure under its name in a record with an
area_code or beneath an area key, a min_ or minimum_ name outside the
floors, and "rows 7 < 50" text. The #1307 review found shapes that slipped
past it: a code-keyed map under a figure's name, a columnar table, a
record identified by geography_id or code, figures in a list, max_ and
median_ names, and any *.json that did not parse, which the walk skipped.
No writer emits these today, but the walk is the last check before a
release leaves the machine, so it should not depend on today's writers.

The walk now refuses an area's rows, Kish ESS or distinct sources, as a
number or as numeric text, in a record identified by area_code,
geography_id or a code that holds an area code; under an area key beneath
the figure's name, through level keys and lists; in a table with an area
column, as parallel lists or pandas {row: value} columns; and as a
minimum, maximum or median of a figure, however nested, outside the
declared floors. A shipped *.json the walk cannot parse, decode or walk
now fails rather than being skipped. Failures still name where a figure
is, never its value. The text rule is unchanged.

The refusal tests' fixtures use placeholder codes (E06999999, E06999998)
where they paired a real local authority with plausible support figures.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ence (#1301)

A dense release names every reviewed exclusion the uk_target_fit,
uk_local_target_fit and uk_input_mass_parity gates stood on, and the
contract checked the local target-fit names against the signed local gate
report. The two national gates' names come from the full gate report,
which does not ship, so the contract could only hold the manifest's
limitations to the coverage's own list: a hand edit that dropped a
national name from both still validated, where the same edit to a local
cell was refused.

The build's candidate manifest projection now records, for each of the
three gates, the sorted names it stood on, dormant entries left out as the
assembler leaves them out, under reviewed_exclusions_stood_on. The
release ships that manifest and hash-joins it to the dataset, the
diagnostics and the signed report, and the contract now holds the
coverage's names to the record gate by gate, so the limitations follow
it; a candidate without the record is rebuilt. The record repeats names
the limitations already ship. Like the measure-exclusion, holdout and size
records the contract already reads from it, the manifest is unsigned: an
edit to it as well would still pass.

graph_terminal.py is in the implementation hash of the preflight and
calibrated gate, holdout, export, package and inspection-export kernels,
and contract.py in the certification kernel's, so the dense graph re-keys
from uk.full.gates.preflight on, the dense solve included, as the earlier
#1301 edits to graph_terminal.py do. No national kernel hashes either
file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The dense assembler's docstrings, the runbook and the changelog said a
reviewed-exclusion limitation carries "names and reasons only, never the
measured error". The gate's own measurement of the row is indeed not
copied, but the reason is the register's reviewer prose, copied verbatim
up to 300 characters, and it can carry figures: the input-mass register's
reasons cite national totals and a percentage. Nothing published is wrong
today; the wording promised more than the code checks, so it now says what
is copied and that a reason may cite figures of its own.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ext figures (#1301)

The drafter's PR body printed stale-risk and near-bound counts for every
register. A local cell signed just beyond the 25% bound has its
family/area_code/metric key in the same patch, so a stale-risk count there
tells a reader that a named local authority or constituency estimate lies
within a point of the bound: a sub-GOR figure with no disclosure check. Only
the national register's counts now reach the PR body and the receipt. The
local registers' counts go to the tool's standard output and stay as per-row
flags in PRIVATE_review.csv, and the body's disclosure line now lists exactly
what the body carries.

Reviewer free text was checked only for '%', and the release contract's
disclosure walk catches only an area's support worded as "rows N <". A
household reference with its weight, "N percent above its target" or "7
rows against the 50-row floor" passed into the committed reasons.
--class-note text, --area-name names and --adjudication may now carry digits
only in issue references and ISO dates, and --approved-by, which lands in
every entry, must be a GitHub login; all four are checked before the build
is read. The runbook and the changelog fragment say what the checks cover
and what they miss, and the runbook's manual scan now covers signoff.patch
as well as PR_BODY.md.

--out was refused only inside the tool's own checkout, so the private review
file could land in another checkout or a sign-off worktree. Any --out inside
a git work tree is now refused, found by git rev-parse from --out or its
nearest existing parent, and .gitignore covers PRIVATE_review.csv and
signoff-draft/.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…#1301)

The sign-off draft removed a national target-fit entry that applies on
every line whenever build 2 would fail it on the local line, for instance
because dense build 1 fits the row inside the bound, and replaced one with
a local-only renewal when build 1 still breaches the row and the entry is
out of its window. The register is shared with the national line, which
may still stand on that approval, so the next national build or
certification would fail on a row someone had already signed. Such an
entry is now never removed or replaced: it is rescoped to release_lines
["national"], otherwise unchanged, and the receipt says why. An expired one
also goes to review for the national line to renew or retire. Kept beside
newer approvals it breaks the national graph test's review invariant, so
the draft is then reported unsignable until that is done. A row build 1
still breaches under such an entry needs a hand sign-off, because a
register key holds one entry and no local-only entry fits beside it.
--verify checks that every rescope was committed as drafted. Local-cell
entries are still removed, since the national line reads no local cell.

The gate-clock range came only from the gate-only registers, so build 2 and
--verify could pass on the day a measure exclusion expired and assembly
refuse it the next, while renewing the exclusion changes the solve and
forces a third build. The drafter now reads the measure exclusions build
1's solve ran under from its candidate manifest or, when the gates stopped
build 1 before the export, from the target compilation's materialized
surface, and takes the local line's degenerate-register entries from the
checkout. The range ends at the earliest expiry among these and the
in-line gate-only entries, which the receipt and the PR body print as the
assembly deadline, and a --gate-clock past a solve-scoped expiry is
refused. --verify fails once the deadline has passed and warns from two
days before it.

The runbook's build-1 command now passes --review-date and --graph-store,
whose defaults do not carry over to build 2, and build 2 repeats both with
a new --out, --gate-review-date and --require-solve-reuse. It also chains
its Logbook row on build 1's: the build does not check this, but the
Logbook store and the archive export refuse a fork or a second genesis on
the uk/local chain.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… rows it leaves bound (#1301)

The hvcts_la_household_floor rule said a cell under 400 homes "rests on
one or two sample households" and that "one household's placement
decides" whether it passes, and the constituency deferral said a cell
"resting on one or two sample households is fitted by noise". These are
expectations from Land Registry-derived stock at 150,000 households, not
counts of sample records, but they read like counts. The generator also
copies both texts into the committed membership: the signed_rationale of
598 authority cells and 1,725 constituency cells, and six signed
deferrals. Both texts now state the expectation: a cell below 400 homes
is about two household weights at 150,000 households, so its fit would
be noise.

The rule text now also names the two region rows below the floor:
microcosm.high_value_homes.over_2m_by_region@E12000001 (North East) and
@W92000004 (Wales). The rule defers every £2m authority cell in both
regions, so each row is its region's only £2m control. Region rows are
the family's control and outside the rule's scope by design, so both
stay bound. Each is signed gate-only (uk_target_fit, release_lines
["local"]) only if build 1 measures it past 25%. Neither is signed in
advance, because an entry on a row that fits fails the gate as stale.
The changelog says the same.

The local references, membership and compile-parity receipt were
regenerated on the pinned 735820c feed. Only the 2,329 rationale strings
in the membership move. The counts stay at 22,986 active of 27,115
candidates. The references and the receipt are byte-identical, and the
census check reports current.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The measure-exclusion register has no line scoping, so the obr.ni_employer
entry drops the row from the national objective as well as the local one.
Its adjudication now says so, and says what that means for the frozen
national build. A national run given a frozen scoring register derived
before this entry fails closed, because the re-derived register differs,
so that build's frozen register is derived from a head that carries the
entry.

The reason no longer calls the gap "not a weight question". The parts it
names (the Employment Allowance, the calendar-2025 window and the under-21
zero rate) explain roughly half of the excess. The rest sits in the pay
level and is under investigation. It may be a conflict between the bound
pay targets and OBR's NICs base rather than a weighting defect. The
phrases test_employer_nics_are_a_measured_diagnostic checks are kept. The
solve-registers changelog now says both.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The L0 bracket changelog said callers on the default bracket "search as
before". They keep the same bracket and the same cold path. On the warm
path, though, a start outside the bracket now raises instead of being
clamped, and the search now stops at a bound instead of bisecting toward
it.

test_the_frozen_count_is_the_committed_active_reference_count compared
two reads of the same membership key, so it could not catch drift. It
now pins 22,986, the count test_uk_battery_bindings.py pins as
candidate_targets from the committed local references, and says the two
pins move together.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ase (#1301)

The dense contract's disclosure walk runs only after both builds: at
assembly, in the release pre-flight, at publication and in the sign-off
verify. A refusal there could cost a re-run of the gates for a finding a
reviewer can weigh. validate_release_dir now returns the walk's findings
as warnings, and an empty list on every other line. The dense assembler
prints them and lists them in its summary, the release pre-flight prints
them without failing, publication repeats them as Python warnings, and
draft_uk_dense_signoff.py --verify reports a figure in build 2's signed
local report as a warning.

The walk over the drafted entries still refuses the draft: re-drafting
costs nothing, and those entries are committed to the public repo in the
sign-off PR.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 and others added 6 commits October 17, 2026 16:09
The drafter scopes an all-lines national target-fit entry that build 2
would fail on the local line to the national line, unchanged otherwise,
because the national line still stands on it. --verify --draft-receipt
checked that the committed entry was national-only, but not that the
sign-off left the rest of it alone, so an extended expiry or a reworded
reason would have changed the national line's approval under cover of
the local line's sign-off.

The receipt now keeps each rescoped entry as committed before the
sign-off (rescoped_from: committed register text, not build 1's
evidence), and --verify fails when any field but release_lines differs
from it in the committed register, naming the entry and the fields. A
receipt that does not record the entry fails too, asking for a new
draft. The module docstring says what --draft-receipt checks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
)

The drafter's assembly deadline and gate-clock range disagreed with the
gates they predict. The area-support gate window-checks every
bound_despite_support_floor family whatever its release lines
(battery_bindings._evaluate_area_support), but the drafter kept only the
families on the local line; and the input-mass gate holds its in-line
entries to their windows on build 2's gate clock, but the drafter left
them out. Nothing is affected today (the only bound family applies on
every line, and the input-mass entries expire in April 2027), but a
national-scoped family or an input-mass entry expiring first would have
let the receipt print a range build 2 then fails.

gate_clock_entries now gathers, for draft and --verify alike, the
local line's entries of each wired register after the sign-off, every
bound family, and the local line's input-mass entries, which the tool
reads (input_mass_in_line) but never drafts. Their expiries join the
measure exclusions and degenerate entries in the deadline, and their
approvals, with the degenerate register's, start the range. An
input-mass entry out of its window at the gate clock makes the draft
unsignable until it is renewed by hand, as an out-of-window bound family
already did, so the printed range is never empty.

Tests cover a national-scoped family and an input-mass entry at each end
of the range, an out-of-line input-mass entry standing aside, the
unsignable case and --verify's deadline; dropping either source fails
them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
)

The pre-replay guard of --require-solve-reuse checked only that build 2's
gate clock put every in-line gate-only sign-off in force from its
approval. An in-line entry in force on build 1's clock but expired on
build 2's was admitted, and the calibrated gates failed it after the
replay. The degenerate register, solve-scoped but evaluated by those
gates on the gate clock, was not checked at all, and the end of the
gate-clock range the sign-off drafter prints (its assembly deadline) was
never enforced, so a gate clock past an expiry the release depends on
went through.

The guard now reads every window the drafter bounds build 2's gate
clock by (solve_reuse_windows): the gate-only registers' entries the
calibrated gates hold on the local line, the degenerate register's local
entries, and the calibration measure exclusions the solve applied
(--measure-exclusions, else the packaged register), which assembly and
upload check on the day they run. A gate clock before any approval or
after any expiry is refused before anything runs. The refusal names the
entries out of their window and the usable range: on or after the latest
approval, on or before the earliest expiry, the latest usable gate clock.
When the windows do not overlap it says no gate clock is usable, and an
expired degenerate entry or measure exclusion is flagged as needing a
third build, since renewing one changes the solve.

The earliest-expiry arithmetic is shared: rowwise_cli.earliest_expiry
is what the drafter's assembly_deadline returns. A lockstep test holds
the drafter's gate_clock_entries to the guard's windows on the same
synthetic registers, line rule included.

The solve-reuse tests now stand in the degenerate register and the
measure exclusions too, so none depends on what is committed on the day
it runs. The premature test's "in force" target-fit entry had in fact
expired before the solve clock; it now covers the gate clock it names.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
)

The guard reads each register through the calibrated gates' line rule
(battery_bindings._in_line), so an entry scoped to the national line
neither refuses nor admits build 2's gate clock. Only the target-fit
register's rule was pinned: with _in_line dropped for the local
target-fit, input-mass or area-exclusion register, the guard's tests
still passed.

A parametrized test now holds every register the guard reads to the
rule, the degenerate register included, for a premature and for an
expired entry: scoped to the national line, the entry is set aside and
the gate clock admitted; on the local line, or on every line, the same
entry refuses it. Dropping the line rule for each of the five registers
in turn fails the test, and only the target-fit mutation failed an
earlier test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The dense attempt refused a solve-reuse build into an --out that held a
bundle, but keyed only on its build.json completion marker. A failed
build-2 attempt leaves failure.json and a Logbook spool row instead, so
a retry into the same --out was admitted and spooled a second attempt's
row beside the first. draft_uk_dense_signoff.py --verify then refuses
the run directory (its spool names two attempts), and the assembler
takes the last row by file name.

refuse_occupied_solve_reuse_output now refuses an --out holding
build.json, failure.json or any logbook-spool/*.json row, naming what it
found, still before the attempt opens and without writing into the
directory. Other files, and a spool without rows, are not an attempt.
The --require-solve-reuse help names the three markers.

Tests refuse each marker on its own, admit the same directory without
one, and run a failed second build whose retry into its --out is refused
with the directory unchanged; keying on build.json alone fails them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The runbook, the graph doc and the two changelog fragments now say what
the preceding commits enforce: the allowlisted references, URLs and
login shape the free-text guard accepts; the input-mass and
bound-despite-floor entries in the gate-clock range and the assembly
deadline; build 2's refusal of a gate clock outside that range and of an
--out holding any earlier attempt; and --verify's check that a rescoped
entry changed only its release lines.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Oct 17, 2026
On main after #1302: 25,327 compiled cells. The 18 Scottish BRMA rows are
ledger-only by construction, and no signed value moves. The UC drift
rationales now read their fixture/ours bands off the rows, and the support
floor's deferred cells carry their own signed rationale. The release-blocking
preflight gate uk_ledger_compile_parity_local_incumbent_2025 reads this
receipt, so whichever of #1305, #1306, #1307 and #1308 lands last
regenerates it on the combined tree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 and others added 12 commits October 17, 2026 16:42
…ls (#1300)

Every committed surface has off-surface cells, so the first reconciliation
pass only records the UC bridge's factor and the second pass is the one that
checks the 2% bound. Until now every refusal test ran without off-surface
cells and so exercised the first pass only: turning the check off on the
second pass, or restoring the old guard that skipped the second pass when no
off-surface row joined the frame, left every test green.

The new test runs the real rule with off-surface cells present: an unrelated
cell that adds no row (a GB caseload of 300 over 200 constituency households
is still refused), a UC cell that joins the frame (refused at the same 1.5),
and a UC cell that leaves the authorities short of the caseload (refused at
authority grain). Each of the two mutations fails it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The bridge's bound reads each lower grain's total under its control: for
the GB Universal Credit caseload, the constituencies' and the authorities'
sums over Great Britain. It does not bound the nations and English regions
inside a grain: the exact-signature group that puts each leg's authorities
on its constituencies runs first and has no bound. The texts said "a leg"
and "each grain on its own input total", which read as a per-leg bound.

No behaviour changes. The shared operator's docstring, the UC bridge's
declaration reason, the #1300 changelog line and the authority-tier test's
name now say the bound applies to each grain's GB total.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ir ranges (#1300)

The local compile-parity receipt's UC drift rationales still described the
all-claims series: a uniform national rescale with fixture over ours in a
fixed narrow band across 982 rows, and a children scatter range taken from
that series. Since the abf0921 re-pin each area cell counts its own paid
claims, so fixture over ours moves with each area's paid share. María has
signed off on the rewording.

The texts now say so, and each quotes the 5th to 95th percentile of fixture
over ours, to three decimals, with the row count, computed from the rows it
covers when the tool writes the receipt (the totals rows; the four child
bands together), so the range cannot go stale. The gate does not read
reasons; the next regeneration picks the texts up. Unit tests check the
quoted ranges on synthetic rows and that other rows keep their rationales.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two more readers took the 7.4 GB feed whole. The CGT partition test read the
file into one string and parsed every row; it now streams the lines and
parses only the CGT rows. Every selector and ratio operand it compiles pins
an hmrc.cgt_* source concept and none names a fact key, so no other row can
match and the compile is the same. It also accepts CHRONICLE_UK_FACTS, as the
other feed-gated helpers do, and skips as before when neither is present.

The receipt tool read the file into one string and split it before parsing.
It now streams the lines; both compiles read every row, so it still keeps
the parsed rows, but no longer holds the text and its lines beside them. A
unit test checks it returns the same rows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
align_uk_local_registry_parity_fixture looked up each fixture row's contract
target through _uk_local_metric_targets, which reads, parses and checks the
whole UK population contract. The local fixture has about 23,500 rows, so the
local compile-parity gate, the receipt tool and any replay spent three to four
minutes here. The table is now built once per fixture and passed to each
lookup; the aligned fixture is byte-identical (checked against the previous
code on the committed fixture), and alignment takes well under a second.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… test (#1300)

The release-blocking preflight gate uk_ledger_compile_parity_local_incumbent_2025
compares each signed value in the local receipt with the live compile, so a
receipt the last regeneration left behind blocks every full or dense build
and the release-cut certifier, and no CI test noticed (microcosm#1302 review).

The local reference generator writes a reference for every cell that compiles
on the pin and records each cell's compiled value in the membership, so the
test rebuilds the compile from the committed references and membership (and
checks the two agree), then runs the declared gate entry through its UK
binding with nothing unsupported. It needs no feed and takes a few seconds.
A second test moves one signed value and checks the replay names exactly it.

On this branch the generated surfaces are left stale until they are
regenerated on abf0921, so the replay fails here until then. Against main's
membership and references it passes, and a corrupted receipt value fails it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…unt (#1300)

The re-pin procedure named uk/high_value_home_stock.json, which does not
exist: the stock is uk/high_value_home_stock_facts.json, with its value knots
and uk/high_value_home_stock.provenance.json. The abf0921 history entry said
the local-authority energy selection reads gas by gas_year, which implies the
selector pairs fuel with period type; it accepts both period types for both
fuels, and its expected row count and the per-fuel reads do the pairing, as
the changelog already says (1302-18). It said the bridge reads each grain on
its own input total; it bounds each grain's GB total. And it and the
changelog said the 3,508 UC cells' signed values move in the local receipt:
3,507 change, and one now equals the incumbent and leaves the receipt.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The energy module's docstring said consumption is calendar 2024 kWh. Since
the abf0921 re-pin the gas rows are DESNZ's gas year 2024, which the
sub-national lookups pair with calendar 2024 (SUBNATIONAL_PERIOD_TYPES);
electricity stays on calendar years. Docstring only.

The other wording the re-review names in this module (the module docstring's
account of the Northern Ireland interim, UK_ENERGY_LEVEL_BASIS and the
interim text in the household-share receipt) is left as it is: the two
stacked changes that retire the interim, #1305 and #1306, rewrite those same
lines, and UK_ENERGY_LEVEL_BASIS is stamped into each kWh cell's metadata, so
it moves with their regeneration.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The shared operator accepted any finite positive max_factor_shift, and only
the UK declarations loader required one below 1. A bound of one or more
accepts every factor below one, since a negative factor is refused anyway,
so it bounds one side only. _validate_rule now refuses a bound outside
(0, 1), as the loader does.

Every caller already passes a bound in range: the UK declarations (validated
by their loader), the fan-out sum bridges (no bound) and the tests; nothing
outside the UK builds a CrossGrainBridge. The refusal test adds bounds of
one and above, and a bound just below one is accepted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The stock is built from feed facts (the census tenure record sets, the OBR
non-owner-occupied share, the HPSSA seat cells), and the re-pin procedure
asks for a check that they are unchanged on a new pin, but nothing ran it.
Its provenance still names the 735820c feed it was built from.

A feed-gated test now streams the pinned feed, runs the builder's own
readers over every row that carries one of their selectors' strings, and
fingerprints what they return. It compares that with the fingerprint
recorded for the feed the provenance names, and asks for a rebuild if they
differ. The fingerprint recorded for 735820c was computed on that feed and
is the same on abf0921, so the stock stands and, as the procedure says, its
provenance keeps naming 735820c. A second test, run without the feed, checks
that the feed the provenance names has a recorded fingerprint, so a rebuilt
stock records its own. The procedure names the test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Small-cell deferrals always join the reconciliation frame, but they were
tested only with the bridges' bounds lifted. Two tests now run them on the
real rule: a deferral that brings the authorities' GB total up to the
caseload is accepted, the bound authority keeping its value, and one that
leaves the total short is refused at authority grain. The committed surface
has no UC deferrals, so nothing moves today.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
After rebasing onto the #1300 re-pin, the references, memberships, both
compile-parity receipts and the census are regenerated once from the
final code against the pinned abf0921 feed, in the order the re-pin
procedure gives. The rebase had kept main's version of each generated
file at every conflict, so nothing in them was merged by hand.

National: the two high-value homes region rows signed out below the
floor leave the references and the receipt's ledger_only rows and are
recorded as signed_excluded, 1,319 active references of 1,341
candidates. Local: the references are byte-identical to the rebased
ones; the receipt drops the 2,323 deferred high-value homes cells, which
leaves the union the #1302 re-review computed, and restates its 3,507
UC rationales in the new wording; the membership carries the floor
rule's new text. No other value or row moves. The census check is
current, the gate digests and the US spec-engine attestation are
unchanged, and the count pins move to the regenerated files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22
juaristi22 force-pushed the uk-local-release-1301 branch from b10840d to e119867 Compare October 17, 2026 16:03
@juaristi22

Copy link
Copy Markdown
Collaborator Author

Response to the re-review

This PR is rebased onto main after #1302 and regenerated on the abf0921 feed. On top of the earlier commits it carries the fixes below and #1302's remaining review items. The body is rewritten to match.

Blocking

  • The widened walker matched non-support keys: fixed (fd5ac44).
    • The extreme pattern now names whole support-figure stems, before or after the statistic (min_rows, rows_min, smallest_ess). So max_excess and the spine gate's maximum_owner_share_without_main_residence_excess no longer match.
    • Walked over the 15 October dense run's three byte-for-byte files, the old pattern gave 41 false paths in the candidate manifest. The new one gives none, and every real finding in that pre-withholding run is still caught.
    • A contract test walks a real-shaped candidate manifest, with cross-grain relations and a spine gate report, and expects no warning.
    • The pre-flight's --candidate-dir now walks those three files (9eda754), so a finding, or a false match, shows right after build 1.
  • Separately, María ruled that the walk warns rather than refuses (f2a2602).
    • validate_release_dir returns its findings as warnings.
    • The assembler, the release pre-flight, publication and --verify print them and carry on.
    • Even a false match can no longer stop assembly after build 2.
    • The drafter still refuses a figure in its own drafted entries, which go into a public PR.

María's rulings

  • The two high-value homes region rows below the floor are deferred (4850088).
    • The North East and Wales rows over £2m are signed out of the region fan-out by two row exclusions in target_reference_signed_exclusions.json; a row-level exclusion can now name a geography cell.
    • Neither is bound, so no public sign-off can name them. The same areas' £1m and £1.5m rows, which count those homes, stay bound.
  • Local-cell sign-offs publish as drafted. Their reasons never state an estimate or an error.
  • Per-cell estimates and weights ship as they are in the private release. This is the 16 October ruling, now restated in the body.

Should fix (low)

  • Free text and logins (7b0a56d).
    • Digits are allowed only in allowlisted references: microcosm#N, chronicle#N, uk-data#N and PolicyEngine/<repo>#N, each with optional #issuecomment-N anchors.
    • Also allowed: PolicyEngine issue and PR URLs, and ISO dates of real days.
    • Every probe you listed is refused, and the anchor and URL forms are accepted.
    • --approved-by must start with a letter and hold at most one run of digits.
  • The pre-replay guard (0cedaf8).
    • It refuses a gate clock outside the window of any entry build 2 depends on: an in-line gate-only or degenerate-register entry, premature or expired, or a measure exclusion the solve applied.
    • The refusal names the entries and the usable range.
    • The range shares earliest_expiry with the drafter's assembly deadline, with a lockstep test.
  • The occupied --out (c0974d5). failure.json or any Logbook spool row now refuses it, as build.json did.
  • The line rule for every register (fc556cd). A parametrized test covers each register, premature and expired; dropping _in_line for any one fails it.
  • The deadline and gate-clock range (27e00c7). They now count every bound_despite_support_floor family whatever its line, and the local line's input-mass entries. An input-mass entry out of its window makes the draft unsignable until renewed by hand.
  • Regional and national aggregates (fd5ac44). Region, nation and UK codes no longer mark an area, and neither does an area_code holding only those.
  • Shapes the walker missed (fd5ac44).
    • It now also matches names in any case, n_rows, and order statistics keyed under a figure ({"ess": {"p10": …}}).
    • Prose, other names and non-JSON files remain out of reach. The docstring, the runbook and the body now say so, and the manual scan stays.
  • Placeholder codes (f432382, fd5ac44). The drafter's tests and the walker's docstring use placeholder codes now.
    • The shared build fixtures in test_support do the same pairing and are outside this file. They are left for a follow-up.

Nits

  • The graph doc's build-2 command replaces build 1's --logbook-prev-row-digest with build 1's own row digest (47650d2, 94bf591).
  • --verify --draft-receipt checks that a rescoped entry changed only its release lines (723d470).
  • The runbook's PR_BODY.md list names the national entries for review (47650d2, 94bf591).
  • The pins changelog and body item 9 now describe the abf0921 regeneration (e119867).

From #1302's re-review

Folded in here, since #1302 merged (48ef616..b0a6914):

  • Second-pass refusal test: a real-rule test makes the second reconciliation pass refuse with off-surface cells present. Both named mutations fail it.
  • GB-total wording: the UC bridge's docstring, declaration reason, changelog line and test name say the bound covers each grain's GB total.
  • Receipt rationales: the local receipt's signed UC rationales describe each area's paid claims, and quote P5–P95 ranges computed from the rows when the receipt is written.
  • Streaming: the CGT test and the receipt tool stream the feed.
  • Fixture alignment: the local parity fixture alignment builds its contract lookup once. The output is byte-identical, and the step takes under a second instead of minutes, in the gate and the tool.
  • Gate replay test: a feed-free test replays the local compile-parity gate on the committed membership. A second test moves one signed value and expects exactly that row back.
  • Bound check: the shared cross-grain operator refuses a bound of 1 or more.
  • Stock inputs: a feed-gated test checks the high-value home stock's inputs on the pinned feed. They are identical on 735820c and abf0921, so the stock is not rebuilt and its provenance still names 735820c, as the procedure says.
  • Small-cell deferrals: they are tested through the real bound.
  • Doc corrections: the re-pin doc's file names, gas wording and receipt count (3,507 move, 1 leaves).
  • Left to the stacked PRs: the energy wording that Place UK private rent and BRMA at the assigned area; bind Scotland's rents at BRMA grain (#1262, #1090) #1305 and UK local surface before the Budget build: high-value homes OBR leg, per-vehicle fuel flag, Northern Ireland energy share and housekeeping (#1278) #1306 rewrite. This PR changes only the gas-year docstring.
  • Re-freeze (1302-07): the frozen national scoring register is an input of national builds, not a file in this repo. It has to match the next national run's head and review date, so it is re-frozen before that run, after Port the uk-data wave to policyengine-uk 2.134.4: stamp duty, Move to UC, national home values (sixth #1095 PR) #1308. The dense builds do not read it.
  • Memory (1302-03): the build's own target load on abf0921 is being measured locally now; the figure follows in a comment. For scale, the regeneration tools, which also read the whole feed, peaked at 15.4 GB (national references), 16.6 GB (local references) and 25.5 GB (the receipt tool on both surfaces, 23.9 GB on the local one alone), so the receipt tool needs the machine to itself.

Cross-PR

Tests

  • At e119867, locally:
    • The 37 touched and surface-dependent test files pass, and so do the other callers of validate_release_dir. Feed-gated tests skip without the feed.
    • tools/ci_test_plan.py verify is ok, and ruff check and ruff format --check are clean on every Python file this PR changes.
    • tools/census_uk_local_targets.py --check is current.
    • The gate digests, computed as test_gate_battery_contract_pins.py computes them, equal the pins, and tools/spec_engine_coverage.py --check passes in a US environment.
  • Each new guard test fails under the mutation it targets: the second-pass bound, the line rule for every register, an expired gate clock, an occupied --out, a rescoped entry, and the bound-despite-floor and input-mass windows.
  • The feed-gated test files are running locally against abf0921, and their results follow in a comment. CI is running at e119867.

🤖 Generated with Claude Code

@juaristi22

Copy link
Copy Markdown
Collaborator Author

Third look at this PR, at e119867

Run with Claude Code on 17 October 2026, after the rebase onto main (post #1302), the fixes for the second review, #1302's folded-in review items (labelled #1300), the regeneration on abf0921 and the second response.

Verdict

Every blocker from the second review is fixed in code, and so is every should-fix item except the walker's missed shapes (partly fixed). #1302's folded-in items are fixed except streaming (partly).

  • Tests: 1,900 and 1,328 tests on the touched files pass, and every targeted mutation is caught.
  • CI: 10/10 green.
  • Regeneration: consistent. Every move against main is explained by the PR's code, and both compile-parity gates replay with 0 failures.
  • Nothing in the code blocks build 1, build 2, assembly or publication.

Two things need settling before build 1: the timeline, and a disclosure ruling on five more region rows.

Timeline

At this head the drafter's and guard's usable gate-clock range ends on 22 October, the ons.savings_interest_income expiry. The next-earliest is 2 December.

Unless #1308, which carries the renewal, lands first, build 2, --verify, assembly and upload must all finish by 22 October. #1308 is rebased onto main (8739557) and its CI is green, but it is still a draft. The renewal commit sits directly on main and merges cleanly with #1305, #1306 and this PR. If #1308 isn't approved by about 20 October, split the renewal into its own PR.

For María's ruling before build 1 (disclosure)

  • Five more high-value homes region rows stay bound, with fewer than 10 records expected.
    • The rows: North East and Wales at £1.5m; Yorkshire and the Humber, East Midlands and West Midlands at £2m.
    • From their published Land Registry-derived stock, each is expected to rest on about 3–8 average household weights at 150,000 households. That is an expectation from the stock, not a count of frame records.
    • 4850088's reason for signing out the North East and Wales £2m rows applies to them too. If build 1 breaches one, the drafter writes a public national uk_target_fit entry saying it "breached the target-fit release bound". The national stale-risk count in PR_BODY.md can then bracket it.
    • The response's pre-PR-B private check covers only local-authority cells under 2,000 homes.
    • Options:
      • extend the private check to national rows under about 2,000 homes;
      • keep the national counts out of PR_BODY.md when a signed national row is a high-value homes region row;
      • or sign these rows out too.
    • Related: any breaching local cell resting on fewer than 10 records has the same exposure through its "breached the bound" reason.
  • The walker's new region exemption ("a region's support is a GOR-level aggregate") holds for a region's total support. It doesn't hold for region × threshold cells like these five. So a per-cell support figure keyed by a region code in a shipped JSON file would draw no warning.

Low

  • The warnings never reach machine-readable output. --preflight-only prints {"valid": true, "published": false} without them (they go to stderr), every step exits 0, and nothing in the published tag records that warnings were weighed. Add them to the verdict JSON. Optionally require an acknowledgement.
  • The free-text allowlist still lets digits through.
    • Accepted, though they carry figures:
      • household microcosm#4521 carries weight microcosm#312
      • the area has uk-data#7 rows
      • PolicyEngine/rows#7 against PolicyEngine/floor#50
      • URL path segments, an issuecomment anchor followed by a count, and date-shaped 2045-12-31
      • the logins rows-7, ess7 and w312
    • Refused, though legitimate: a lower-case owner, #discussion_r…/#pullrequestreview-… anchors, and blob URLs with digits in the path (the runbook's own file name).
    • Fix: a fixed repo list, a date window near today, and review anchors accepted.
  • The walk still misses some shapes.
    • Nested order statistics first ({"min": {"rows": …}}).
    • distinct_sources and sample_size as record keys: they are extreme stems, but they are missing from _UK_AREA_SUPPORT_FIGURE_KEYS.
    • lad_code or geography columns, pandas split orient, *_by_area maps, num_rows, row_count, kish_ess, an upper-case AREA_CODE key, and "1,234" as text.
    • min_distinct_source_households, which the old pattern caught, is no longer caught; no writer emits it.
    • All seven pre-withholding writer shapes are still caught.
  • Pre-flight walk gaps.
    • The opt-in incumbent files, which ship byte for byte, aren't walked.
    • It parses with plain json.loads, without the contract's NaN rejection, and doesn't catch RecursionError.
    • No test drives --candidate-dir through to the warning output or puts a figure in the diagnostics or signed report.
    • No test drives the assembler with a finding.
  • A build-1 retry into the same --out spools a second Logbook row, and the drafter then refuses the run dir. The runbook has no retry guidance for build 1.
  • The body's peak-memory warning isn't attached to the bullet telling Place UK private rent and BRMA at the assigned area; bind Scotland's rents at BRMA grain (#1262, #1090) #1305/UK local surface before the Budget build: high-value homes OBR leg, per-vehicle fuel flag, Northern Ireland energy share and housekeeping (#1278) #1306 to regenerate. It doesn't say that 25.5 GB can exceed the build machine's memory, and it omits 23.9 GB for the local surface alone. Suggest "regenerate the receipts one --surface at a time with nothing else running".
  • Bisectability: commits 87d310b..b0a6914 fail the new replay test until e119867. That only matters if the PR isn't squash-merged.

Nits

  • "At about one household weight each" (the North East and Wales £2m rows: the 4850088 message, local_small_cell_rule.json and so the membership texts, the changelog and body item 5) is an accurate expectation from the published stock. On its own it can read as a measured count of frame records. Safer: "their published stock is about one average household weight at the ruled 150,000 households (an expectation from the stock, not a count of frame records)".
  • docs/uk-full-build-graph.md:107 still says the contract "refuses" such a file.

Still open from earlier rounds

Dispositions

  • Second review, all fixed:
    • the walker blocker: no false paths on any real-shaped file, it still flags everything the old pattern rightly flagged, and it is now warn-only plus a pre-flight walk;
    • the below-floor £2m rows, superseded by 4850088;
    • the free text and logins (with the residuals above);
    • premature or expired gate clock;
    • occupied --out;
    • the line rule for every register;
    • the deadline and range;
    • regional and national aggregates;
    • placeholder codes in the drafter tests;
    • all the nits.
  • Second review, partly fixed: the shapes the walker missed.
  • Re-pin the UK Chronicle feed to abf0921 (#1300) #1302 items:
    • Fixed: the second-pass refusal test, the GB-total wording, the UC rationales from the rows (0.937–0.973 over 980, and 0.815–1.237 over 2,527, recomputed), the gate replay test, the shared bound below 1, the stock-inputs test, small-cell deferrals and the doc fixes.
    • Partly fixed: streaming. The receipt tool still keeps every parsed row.

Cross-PR

Disclosure check

What changed since the second review (the new commits' messages and the lines they change) carries no survey-derived figures. The forward-looking question is above.

Tests and tools run

  • Engine-free, at e119867: 1,900 tests, 0 failures, 7 feed-gated skips; and 1,328 tests, 0 failures, 7 skips.
  • tools/ci_test_plan.py verify ok; the census --check and the coverage manifest --check current; test_gate_battery_contract_pins.py 20/20.
  • The PR's own test_uk_local_compile_parity_receipt.py: 5/5.
  • National and local gate emulation: 0 failures each.
  • Mutations: all caught (second-pass bound, guard line rule, expired branch, failure.json, input-mass and bound-family windows, rescope check, degenerate entries).

🤖 Generated with Claude Code

@juaristi22
juaristi22 merged commit ff3f348 into main Oct 17, 2026
10 checks passed
juaristi22 added a commit that referenced this pull request Oct 17, 2026
…process (#1278)

Second review of microcosm#1306, items A and B.

The OBR leg still raised on a matching row with a missing or non-numeric
value, or a period or dimensions field that is not a mapping, and passed
NaN or negative counts as available. Such a row is now reported: a count
must be finite and non-negative, a field that is not a mapping reads as
empty, and an unreadable share leaves the band comparison standing.

uk_terminal_only_code_hash is memoised, so an edit to the high-value
homes files while a build runs cannot fail the calibrated gates after the
solve. Its docstring now says what it buys: an edit to
high_value_home_gates.py never re-solves, but high_value_homes.py also
feeds the target kernels, so an edit there still does. The hash test
covers the preflight kernel #1307 added, which inherits the calibrated
kernel's hash, and the memoisation.

Two comments are corrected: the regional split withholds regions until
the withheld ones rest together on the floor, and its threshold follows
the derived stock while the bands follow OBR's. The evidence summary,
not the gate details, carries the frame total.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Oct 17, 2026
)

Third review of microcosm#1307. The drafter's free-text check accepted
figures dressed as citations (a household and weight beside microcosm
references, uk-data references counting rows, references to made-up
PolicyEngine repositories, counts after anchors, far-off dates and
logins carrying a number), and refused legitimate ones (a lower-case
owner, review anchors, file URLs with digits in the path).

Digits may now stand only in a citation of a repository on a fixed list,
matched in any case: an issue reference, an issue or pull request URL
optionally anchored on a comment or review with GitHub's ten-digit id,
or a file URL. A citation running into a word, or right next to a noun
a figure counts or belongs to, is refused, and an anchor stands only in
a URL. ISO dates must lie within a year of the drafting date. Logins
with digits must be on a fixed list. The sign-off fragment #1307 added
now describes these rules.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22

Copy link
Copy Markdown
Collaborator Author

Follow-up: the local feed checks

The reply said these would follow. They ran locally against abf0921.

They caught a build failure, now fixed (89cc6b9)

  • What failed. The build's target compilation refused the regenerated surface. This is load_uk_full_target_inputs, then compile_uk_full_target_surface, as uk.full.target_compilation runs them. The error: cross-grain inconsistency 'exact:uk.household.count:…:region_over_la' has unparented lower-grain leg(s) ['E12000001', 'W92000004'].
  • Why.
    • The North East and Wales £2m local-authority cells are all deferred by the 400-home floor.
    • The small-cell rule adds deferred cells to the reconciliation frame, so that they take their share of a region control.
    • With the two region rows signed out (4850088), their legs had no control, and the unparented-leg check stopped the compile.
    • Build 1 would have failed at target compilation. Nothing in CI compiles the full surface on the feed.
  • The fix.
    • The local surface now passes the reconciliation the area rows that bind nothing (its frame-only rows).
    • A leg on which every open row is one of them, for targets licensed empty there, needs no control. It is receipted under deferred_legs_without_control instead of refused.
    • These are still refused as orphans: a bound row on such a leg, a frame-only row without the licence, and a row of a target the leg is not licensed for. The Scottish authority of an England-only target is still refused too.
    • A new shared test covers each case.
  • Result. The compile now passes on abf0921. The new receipt names exactly two legs, E12000001 and W92000004, for the £2m local-authority target, and nothing else. The compile also warns that ons.savings_interest_income expires on 22 October; Port the uk-data wave to policyengine-uk 2.134.4: stamp duty, Move to UC, national home values (sixth #1095 PR) #1308 carries the renewal.

Peak memory (1302-03)

The build's target load on abf0921 peaked at 21.9 GiB and took about 20 minutes. The compile after it took 30 seconds within the same peak. So target compilation alone needs most of a 32 GB machine.

Feed-gated tests

  • With CHRONICLE_UK_FACTS set to the abf0921 artifact, all 12 feed-gated UK test files pass. That includes test_committed_surfaces_regenerate_from_pinned_feed on both surfaces, and the new stock-input check.
  • Two tests in test_uk_target_references.py read only the default .codex-work path, not CHRONICLE_UK_FACTS: the childcare and bus compile, and the DfE extended-sum refusal. Both pass with the feed linked there.
  • Every test file that exercises the reconciliation passes at 89cc6b9.

🤖 Generated with Claude Code

juaristi22 added a commit that referenced this pull request Oct 18, 2026
)

From the #1305 review:
- Both private-rent routes in uk/cross_grain_declarations.json declare
  their level miss. The rows bind PIPR's level while the node holds each
  region at its lower FRS level, so with tenure bound, calibration closes
  the gap only by moving weight toward higher-rent renters (item 03). The
  BRMA route names its own composer, uk_private_rent_brma_mean_to_total
  (item 28).
- The compile-parity tool routes the support floor's signed deferrals
  before the per-metric maps, so E06000053 and E09000001 no longer read as
  Northern Ireland's (item 31). Its UC drift rationales read their
  fixture/ours bands off the rows, as #1307's code does (item 09).
- The parity census names PIPR's July 2026 edition, the record set the
  selector resolves (item 32).
- The Scottish overlap note states the layer's correction counts and where
  they live: 9 filled, 59 corrected by the postcode lookup, 2 Balloch
  overrides (item 22).
- A pinned digest of the atomic supports' (area, FRS region) pairs lets CI
  check the committed layer against supports it does not mount; the mounted
  test re-derives it (item 17). A ladder-free test checks the crosswalk's
  BRMA level against the committed layer (item 30).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Oct 18, 2026
On main after #1307 and #1306: 27,267 rows, 23,004 of them compiled. The 18
Scottish BRMA rows are ledger-only by construction, and no signed value
moves. The 40 cells of the support floor's two authorities, E06000053 and
E09000001, carry the floor's own signed rationale; nothing else changes.
The release-blocking preflight gate
uk_ledger_compile_parity_local_incumbent_2025 reads this receipt, so
whichever of #1305 and #1308 lands second regenerates it on the combined
tree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant