Skip to content

Re-pin the UK Chronicle feed to abf0921 (#1300) - #1302

Merged
juaristi22 merged 9 commits into
mainfrom
uk-chronicle-repin-1300
Oct 17, 2026
Merged

juaristi22 merged 9 commits into
mainfrom
uk-chronicle-repin-1300

Conversation

@juaristi22

@juaristi22 juaristi22 commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator

Re-pins the UK Chronicle feed to PolicyEngine/chronicle abf0921 (PolicyEngine/chronicle#359, which answers chronicle#326, #339, #356, #357 and #358) and makes the fixes #1300 lists, so that nothing Microcosm binds stops resolving or silently changes meaning. Values move only for the UC area rows. The private-rent work for #1262 stacks on this branch.

Fixes #1300.

Summary

  • UC households by area count paid claims (chronicle#339). Chronicle now splits each area-month by Stat-Xplore's Payment Indicator and retires the all-claims cells. dwp.uc.households_by_area (both grains) and the four child bands add payment_indicator to their dimensions and bind Yes; the three-plus band's summed operands name it too. The area cells and the GB caseload that controls them now both count paid claims, so each cell takes its own area's paid share instead of the GB one, and the bridge national_uc_caseload_vs_uc_households_by_area moves from a factor of 0.921 to about 1.001 (the gap is about the size of DWP's "Unknown" area).
  • A factor bound on cross-grain bridges. CrossGrainBridge gains max_factor_shift, read from uk/cross_grain_declarations.json. The UC bridge declares 0.02, so a factor more than 2% from one is refused: the two sides would no longer measure one quantity.
    • Each lower grain is bounded on its own input total. The authority tier is otherwise rescaled onto the constituencies by an exact-signature group first, which would leave the bridge a factor of one whatever the authorities said.
    • A surface with off-surface cells is bounded on the pass that carries them, not the first one.
    • The shared operator validates the bound (finite and positive) and records it, with the factor it read, in each leg's receipt.
  • OBR table 4.9 lines carry DWP's geography (chronicle#357 item 4). TARGET_ID_GEOGRAPHY_PINS stamps eight lines Great Britain and four England and Wales, and the obr.state_pension diagnostic, which dwp.state_pension.amount requires, reads its Great Britain fact. No value changes.
  • DESNZ gas runs on gas years (chronicle#356).
    • Northern Ireland's rows now share Great Britain's concepts, so both sub-national vendor selections pin Great Britain (geography_vintage: publication_2025_nspl).
    • The local-authority selection reads gas by gas_year and electricity by calendar_year, at domestic_scope: all_domestic.
    • The energy stage and the local energy appliers name each fuel's period type (energy_pricing.SUBNATIONAL_PERIOD_TYPES). Gas year Y pairs with calendar year Y, as the rows were read before Chronicle stamped the gas period, so the gas-connected shares and the GB levels do not move.
    • 56 regional and 347 local-authority gas rows come back under new keys with the same values.
  • The Northern Ireland energy interim stays for now. Great Britain's share of UK households still stands in for its share of the Energy Trends level. The Northern Ireland facts are now in the feed, on a basis DESNZ calls not directly comparable, and María ruled on 16 October that the interim stands until a change stacked on this one derives Great Britain's part from them. The adjudication, its census fence and the code say so.
  • Schema digest. The v4 consumer-fact digest moves to d953306d… (the gas_year period type). The NZ pin moves at its own next re-pin, and docs/uk-chronicle-feed-repin.md now names the two tests that restate the UK pin.

Regenerated

In the procedure's order, on current main:

  1. the vendored resources (only the two DESNZ ones change rows, keys only);
  2. the national references and membership;
  3. the local census;
  4. the local references and membership;
  5. the validation levels' source_feed;
  6. the release input coverage manifest (the CGT resource's digest);
  7. the charter-H2 fixture (twelve stage contract hashes);
  8. the two OBR incapacity-benefits rows in the captured feed-rows fixture, which Chronicle restamped Great Britain;
  9. the local compile-parity receipt. The release-blocking preflight gate uk_ledger_compile_parity_local_incumbent_2025 compares its signed values with the live compile, so every dense or full build and the release-cut certifier read it. Only the 3,508 UC cells move: 3,507 signed values change, and one cell now matches the incumbent and leaves the receipt.

What moves

The membership reports show which facts the references resolve:

  • National: every reference resolves the fact it resolved before, except the twelve restamped OBR lines (same values).
  • Local: every reference resolves the fact it resolved before, except the 3,508 UC area cells (now paid claims) and the 346 local-authority gas-consumption cells (restamped as gas years, same values).
  • The UC values (DWP Stat-Xplore). Against the all-claims cells, the GB constituency total falls 8.0%: households with no children 10.5%, one 6.6%, two 4.6%, three or more 3.0%. The childless share falls from 53.6% to 52.1%. After the bridge, each cell moves by its own area's paid share, between -1.7% and +1.3% for the middle 90% of constituencies.

Status counts are unchanged on both surfaces: 1,291 national and 25,309 local active references, none dropped.

Not run locally

On María's call, to unblock the stacked work:

  • The full target surface was not recompiled. The first full compile on this pin, including the new 2% bound on the UC bridge, is the next build's.
  • Peak memory on the larger feed. The dense and full target load (load_uk_full_target_inputs) and the release-cut certifier were not measured on this pin, which has 1,499,301 rows. The national arms ran at this head with flat peak memory, but those two paths keep more of the parsed feed alive.
  • Most feed-gated tests were skipped, PR CI skips them too, having no feed. They rerun both generators or the national compile parity. One feed-gated test did run: it compiles the childcare and bus references from the feed. It showed that dfi_ni.bus.passenger_receipts expected 150,082,817.49 where the feed and the committed membership give 152,500,000. That expectation was stale on main too (it fails there on the old feed), and is updated here.

Follow-ups

  • Re-freeze the uk-candidate-eval scoring register before the next national solve. The twelve restamped OBR lines move the national register version, which hashes each reference's metadata, though no value moves.
  • The high-value home stock still names 735820c in its provenance. Its feed inputs (TS054 tenure, the OBR non-owner share, the HPSSA seat cells) are the same 26,212 facts on abf0921, so it was not rebuilt. The re-pin procedure now lists the check.

Tests

Touched and fixture-reading tests pass locally:

  • the cross-grain bridge bound:
    • refusals above and below one, accepted exactly at the bound;
    • a lower grain that an exact-signature group rescales first;
    • bounds that are not finite and positive;
    • a pass that records the bound without refusing.
      Three ledger-target tests ran synthetic UC controls at 2 to 4 times their area cells through the real bridge, so they now use controls within 1%. One keeps the old 2× case as a refusal, and a UK control at 0.92 of its cells is refused too;
  • the off-surface UC cells. Most of these tests lift the bound to show where shares go at factors far from one. Three run on the real rule: a caseload far from its constituencies and an incomplete authority tier are refused, and an off-surface authority that completes the tier is accepted on the last pass;
  • the feed pin and the shared identity;
  • the UC selectors and operands, and the profile-parity selectors;
  • the energy lookups. The gas-year fixtures spell out the feed's period types, the committed meter counts cover the roster, and a meter row on another period type is refused by name;
  • the census, validation levels, benefit scope and state pension;
  • the references compiled from the captured rows;
  • the coverage manifest, vendoring and the country spec;
  • the bridge loader, which now names the declaration kind of a bad bound.

Four feed-gated test helpers hash the feed in chunks instead of reading all 7.4 GB into memory, and the council-tax one parses only the MHCLG rows.

🤖 Generated with Claude Code

juaristi22 and others added 3 commits October 16, 2026 14:29
PolicyEngine/chronicle#359 answers chronicle#326, #339, #356, #357 and
#358. Three selector families change identity with it:

- UC households by area: Chronicle splits each area-month by Payment
  Indicator and retires the all-claims cells. The five UC area targets
  bind payment_indicator: Yes (the three-plus operands name it too), so
  the cells count paid claims like the GB caseload that controls them.
  The bridge's factor moves from 0.921 to about 1.001, and a new
  CrossGrainBridge.max_factor_shift (0.02 on this bridge) refuses a
  factor far from one.
- OBR table 4.9: twelve DWP lines are stamped Great Britain or England
  and Wales (TARGET_ID_GEOGRAPHY_PINS), and the obr.state_pension
  diagnostic reads its GB fact. No value changes.
- DESNZ: the two sub-national vendor selections pin Great Britain, and
  the local-authority one reads gas by gas_year at all_domestic.
  SUBNATIONAL_PERIOD_TYPES names each fuel's period type in every
  lookup (gas year Y with calendar year Y), so no value moves.

The Northern Ireland energy interim stays, reworded, until the change
stacked on this one reads the share from the new NI facts (María's
ruling of 2026-10-16). Regenerated: the vendored resources, national
and local references and memberships, local census, coverage manifest,
charter-H2 fixture, and two captured OBR rows. Synthetic UC bridge tests
now use controls within the bound, and a feed-gated bus expectation
that was stale on main is updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
test_chronicle_184_uc_and_obr_references_compile_fail_closed compiled
obr.universal_credit_in_cap against fixture facts stamped United Kingdom;
the reference now pins Great Britain, the geography Chronicle stamps
OBR table 4.9's DWP lines with.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e legs (#1300)

The off-surface tests leave Scotland's authority tier incomplete on purpose,
so the GB caseload spreads at factors far from one, which the bridge's new
2% bound refuses. They are about the shares, so a fixture lifts the bound;
a new test shows the bound refusing the same incomplete tier.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22

Copy link
Copy Markdown
Collaborator Author

Independent adversarial review of this PR at 4c51843

Run with Claude Code on 16–17 October 2026 at 4c51843: seven independent review lenses (correctness, method, generated artifacts, tests, claims against the diff, disclosure, a second fresh-eyes pass), then one to three skeptics per finding who tried to refute it; (k/n) is how many agreed. No builds and no feed-gated tests were run. The head has not moved since the review.

Not ready to merge as it stands. The re-pin checks out: the feed identity, vendored resources, both memberships, the OBR geography pins and the DESNZ gas period types all hold. One thing blocks it. The local compile-parity receipt was not regenerated, so the release-blocking preflight gate uk_ledger_compile_parity_local_incumbent_2025 fails on all 3,508 UC area rows. Any dense/full build or release-cut certification on main after this merges will refuse until that receipt is regenerated. The PR body says "Nothing reads it at build time", which is false. The national seam build is unaffected: the control arm reported in #1308 passed at this exact head.

Blocking

  • Stale local compile-parity receipt fails a release-blocking preflight gate on all 3,508 UC rows (1302-01; high; CONFIRMED, 3/3 skeptics).
    • Where: packages/microcosm-build/src/microcosm/build/uk/ledger_compile_parity_local_incumbent_2025_signed_differences.json (not in the diff); gate declared at uk/gates.json:43-53 (preflight, release_blocking).
    • What happens:
      • The PR moves every UC area cell from all claims to paid claims but skips step 6 of the re-pin procedure (docs/uk-chronicle-feed-repin.md:46-47).
      • battery_bindings.py:1756-1782 loads the receipt. gates.py:446-544 then compares each signed ledger_value to the live compile at abs_tol=1e-9 (default set at gates.py:192).
      • The full graph always compiles local 2025 (full_targets.py:92, :204-208) and supplies it to preflight (graph_terminal.py:683-690).
      • A failed release_blocking gate blocks in every posture (gate_battery.py:537-544). artifact_permitted goes False (full_gates.py:356-369). The dense build then refuses before solving (full_build_cli.py:1952-1962, graph_calibration.py:463-465). The certifier raises at release_certification.py:344-345.
      • Example cell (DWP admin aggregate): dwp.uc.households_by_area@E14001063 is signed at the all-claims value 8791.25 but now compiles to the paid-claims mean 8051.17.
    • Evidence:
      • Several reviewers replayed the repo's own _signed_difference_check_failures / ledger_compile_parity_gate. Base 866fee4 gives 0 failures; head gives 3,508 (980 households_by_area plus 632 for each of the four child bands; 3,507 "changed ledger_value" and 1 "stale").
      • A jq join gives 3,508 of 3,508 UC drift rows equal to base membership and 0 equal to head. All 21,798 non-UC rows match both.
      • No CI test catches it: test_uk_target_references.py:1487 is national-only and feed-gated.
    • False claim: PR body line 42, "Nothing reads it at build time". The repin doc's history entry (docs/uk-chronicle-feed-repin.md:213-215) does not repeat that claim. It only says the UC rows show all-claims values until UK private rent at the assigned area (#1123 item 4, "PR B"): what is in place and the pieces still missing #1262.
    • Fix:

Should fix

  • The 2% max_factor_shift on the UC bridge never applies at local-authority grain (1302-02; medium; CONFIRMED, 2/2, one voter would grade it low).

    • Where: packages/microcosm-build/src/microcosm/build/cross_grain.py:552-563.
    • What happens:
      • Each UC lower row gets both a bridge key and an exact-signature key (cross_grain.py:171-196).
      • The exact-signature group exact:uk.benefit_unit.count:…:constituency_over_la has bridge_id None, so it has no bound. It sorts ahead of the bridge's country_over_la group (:256-271), and claims are keyed per group identity (:304-335).
      • LA rows are therefore rescaled onto the constituency tier with no bound. The bridge's LA group then sees a factor of about 1.0.
      • Probes (synthetic values, real uk_local_target_surface):
        • GB 202 over constituencies 100+100, Scottish LA at 60 or 10: accepted, with the LA scaled by 1.68 or 10.1.
        • GB 200 with LAs at 50/50: accepted.
        • Only a constituency-tier mismatch is refused.
      • Committed evidence on main (docs/evidence/uk-target-surface-1123/reconciliation.json) already shows this shape: the exact group at about 0.921 and the bridge's LA factor at 0.9999999999999999.
      • The codebase names this trap itself, in the _assert_uk_fanout_sum_agreement docstring (uk_runtime/ledger_targets.py:2851-2862).
    • What limits it: one selector serves both grains (uk_population_targets.json:1435-1460), and missing cells must be reviewed absences. No wrong numbers today.
    • Fix: bound each grain on raw, pre-reconciliation totals, as the fan-out check does. Or apply the bridge's bound to the exact-signature group that rescales its lower side. Until then, reword the declaration (cross_grain_declarations.json:52-60, "by constituency and local authority"), the body and the changelog to say constituency grain only.
  • The new refusal test and commit 4c51843 credit a cause the fixture does not exercise (1302-04; medium; CONFIRMED, 2/2).

    • Where: packages/microcosm-build/tests/engine_free/uk/test_uk_off_surface_cells.py:422-426; fixture docstring at :348-357.
    • What happens:
      • The test name, the docstring and the commit message ("a new test shows the bound refusing the same incomplete tier") all say the Scottish authority tier causes the refusal.
      • The actual refusal is GB 300 over constituencies 100+100, factor 1.5. The test's own comment (:423-424) says so.
      • The test raises the same error with a complete authority tier, and with no authority rows at all. With GB 200, the incomplete tier passes and S12000033 moves from 60 to 100.
      • The test asserts only match="max_factor_shift", so any regression at LA grain passes. It duplicates the constituency coverage in test_uk_ledger_targets.py:1449-1455.
    • Fix: rename it to what it shows (for example …_refuses_a_caseload_far_from_its_cells) and fix the docstring. Or rebuild it on a 2%-consistent control with a short LA tier once the LA-grain item above is fixed, and assert on the leg named in the message.
  • The 12 OBR restamps move the national register version, so the frozen scoring register needs a re-freeze, and nothing says so (1302-07; low; CONFIRMED, 1/1).

    • Where: uk_runtime/full_targets.py:250-258 (and :418-428).
    • What happens:
      • TargetRegistry.version hashes asdict(spec) including metadata (microcosm-calibrate/.../registry.py:184-191, :268-282).
      • _ledger_metadata (microcosm/build/ledger_targets.py:4270-4310) stamps the fact key and geography.
      • Compiling obr.esa base vs head gives the same value but versions 3d94c7fa2fb2 and 878eec697c63.
      • Loading with --register-json pointing at the old frozen register raises "Re-derived full national register differs from the frozen scoring register".
      • The 00b4b14 history entry (docs/uk-chronicle-feed-repin.md:104) recorded this consequence for a similar metadata-only move. The abf0921 entry does not.
    • Fix: note it in the body and the history entry, and add "re-freeze the uk-candidate-eval scoring register" to the pre-national-solve follow-ups.
  • No test checks refusal below one, the direction the bound exists for (1302-05; low; CONFIRMED, 2/2).

    • Where: tests/engine_free/shared/test_cross_grain.py:117; check at cross_grain.py:556.
    • What happens:
    • Fix: add a below-one refusal case, including a UK case at about 0.92. For a case at the boundary, note that abs(0.9-1.0) is 0.09999999999999998, so pick values that pin the strict inequality.

Minor

  • Full-build feed memory on the larger pin (1302-03; PLAUSIBLE, low; 1/2, the intent voter refuted).

    • Where: ledger_artifact.py:392-432.
    • What happens: the loader keeps every parsed row. Tracemalloc on 1% samples estimates about 13.2 GiB on 735820c and about 22.0 GiB on abf0921.
    • Counter-evidence: the national arms on abf0921 actually ran. The control arm at this exact head (the one Port the uk-data wave to policyengine-uk 2.134.4: stamp duty, Move to UC, national home values (sixth #1095 PR) #1308 reports) exited 0; the reviewers read its targets-phase maxrss as 12.48 GB, against 11.8-12.2 GB for arms on 735820c. Facts are dropped when the node returns (graph_national.py:495-526, graph_targets.py:246-262).
    • What would settle it: a measured run of load_uk_full_target_inputs (dense/full path) and of tools/certify_uk_release_cut.py on abf0921. The certifier keeps artifact alive through its battery (certify_uk_release_cut.py:254-284) and has only been measured on the 352k-row 1ee7dfe feed (8.06 GB).
    • Fix: add a line to "Not run locally".
  • The bound is checked on the first reconciliation pass, before off-surface cells join (1302-06; low; CONFIRMED, 1/1).

    • Where: uk_runtime/ledger_targets.py:2644 (first pass) and :2726-2727 (second pass); the check at cross_grain.py:552-563 runs in both.
    • What happens:
      • Probe: GB 300, bound cells 170+100, off-surface 30. It is refused at 1.111, yet the full-frame factor is 1.0 (it passes with the bound lifted).
      • Today the only off-surface cells inside the GB legs are 2 English LAs, far below 0.1% of the caseload. The 29 NI cells are outside the legs.
    • Fix: defer the bound to the final pass.
  • The HVCTS stock provenance still names 735820c, and the stock is not in the re-pin procedure (1302-08; low; CONFIRMED, 1/1).

    • Where: uk/high_value_home_stock.provenance.json:288-292.
    • What happens: nothing reads ledger_feed back. The verifier compared the builder's feed inputs (TS054 tenure, OBR non-owner share, HPSSA) across both pins: 26,212 lines each, with identical sorted sha256. So no numbers move this time.
    • Fix: add the builder to the procedure, either as a step or as a recorded "inputs unchanged" check, or add a test that compares ledger_feed to the pin.
  • Off-surface UC tests run with the bound monkeypatched away (1302-09; low; CONFIRMED, 1/1).

    • Where: test_uk_off_surface_cells.py:347-373, applied at :430, :458 and :490.
    • What happens: no test sends off-surface or deferral cells through the production 0.02 bound. The verifier showed consistent values pass with the real bound (synthetic values: GB 200, S12000033 60 plus off-surface S12000034 40).
    • Fix: keep one off-surface test on the real rule, and mention the lifted bound in the body. The committed membership through the real bound gives a factor of 1.00105, so this pin should not refuse.
  • Stale UC rationale texts (1302-10; low; CONFIRMED, 1/1).

  • DESNZ selector notes still say "until UK local energy, area classification and Welsh bus facts: DESNZ Northern Ireland and sub-local consumption, rural-urban lookups, BUS04 country rows (PolicyEngine/microcosm#1207) chronicle#356 lands" (1302-11; low; CONFIRMED, 1/1).

    • Where: uk/uk_population_targets.json:31644 (electricity) and :31683 (gas). The PR edited the gas note but kept the clause.
    • The approved_on part of the original finding is overstated: the continuation date is already recorded at local_binding_adjudications.json:79.
    • Fix: reword both notes to match the fence.
  • Twelve OBR notes still say "The Chronicle fact still carries the UK code K02000001" (1302-12; low; CONFIRMED, 1/1).

    • Where: uk_population_targets.json:2232, 2290, 2616, 2684, 2752, 2820, 2878, 2946, 3058, 3138, 3193, 17410. abf0921 stamps these K03000001/K04000001. Text only.
  • CrossGrainBridge.max_factor_shift is not validated in the shared operator (1302-14; low; CONFIRMED, 1/1).

    • Where: cross_grain.py:948-1002 (_validate_rule) and the field at :55.
    • What happens: NaN or inf never refuses, and a negative or 0.0 bound refuses everything. Leg receipts (:597-619) omit the bound, while national_reconciliation.py:534 records it. Only the UK JSON loader validates it, so no current path is affected.
    • Fix: validate in _validate_rule and record the bound in the receipts.
  • Feed-gated test helpers load the whole feed into memory (1302-c01; low; CONFIRMED, 1/1; this predates the PR, which makes it worse).

    • Where: test_uk_council_tax_targets.py:526-528, :549 (full parse plus national compile); test_uk_hmrc_rate_band_targets.py:91; test_uk_state_pension_targets.py:180; test_uk_pension_credit_targets.py:147. All use sha256(feed.read_bytes()).
    • What happens: the council-tax test parses about 22 GiB of rows on abf0921 (sampled at about 16.1 KB/row).
    • Fix: hash in chunks (hashlib.file_digest), and filter by source before json.loads.
  • Nit: wrong declaration kind in an error message (1302-15). uk_runtime/cross_grain_declarations.py:136-141: a bad bound on a plain bridge raises "UK fan-out sum bridge … must declare max_factor_shift in (0, 1)". No negative test covers it.

  • Nit: energy test fixtures take their period type from the constant under test (1302-16). test_uk_domestic_energy_targets.py:77 uses SUBNATIONAL_PERIOD_TYPES[fuel], and nothing runs _meter_rows_by_roster_area on the committed resource. With a re-stamped gas row the error blames "an authority without a gas network" (domestic_energy_targets.py:400-408), not the period type. The committed resource is correct today: 350 areas, 347 with both fuels.

  • Nit: the changelog fragment gives no size for the value moves (1302-17; changelog.d/1300-uk-chronicle-repin-abf0921.changed.md:3). It does give the bridge factor. Missing are the raw constituency UC total change (about -8.0%, children_0 about -10.5%), the per-cell spread after the bridge (about -1.7% to +1.3%, P5-P95) and the 0-children share shift (about -1.5 points). These are public DWP aggregates.

  • Nit: the LA vendor selection takes the cross product of period types (1302-18). ledger_fact_vendor_selections.json:1300-1320 accepts both period types for both fuels. Only expected_row_count 697 and the runtime reads enforce the gas/gas_year pairing. The body and changelog wording implies the selector does it.

Cross-PR

Disclosure check

The disclosure lens and every verifier found no UKDS breach in #1302's public text or in the lines it adds or changes. What was checked:

  • The PR body, the three commit messages, the changelog fragment and the docs/uk-chronicle-feed-repin.md addition. Every figure is DWP Stat-Xplore, OBR, DESNZ, ONS LFS, DfI NI or a feed/reference count.
  • The lines this PR changed in the generated local references and membership: selector fields, and DWP UC and DESNZ cells, only.
  • The census, validation levels, release manifest and the 20+ vendored resources: identity headers only.
  • The DESNZ resources: keys and period type only.
  • The national files: geography and keys only.
  • The charter-H2 fixture: 12 hashes.
  • The test fixtures: synthetic values or OBR table 4.9 rows.
  • Receipt-writing code: adds no threshold code, and its leg receipts carry only official/admin controls.

Tests and tools run

All runs were at head 4c51843 unless noted.

  • Correctness lens: test_cross_grain.py, test_uk_off_surface_cells.py, 3 test_uk_ledger_targets.py nodes and test_uk_domestic_energy_targets.py: 81 tests, 0 failures, 0 skipped. Plus synthetic probes of the LA-grain bound (1302-02) and the first-pass check (1302-06).
  • Method lens: 88 tests (cross-grain, off-surface, domestic energy, benefit scope), 0 failures, 2 skipped (feed-gated). A replay of the repo's ledger_compile_parity_gate against the committed local receipt: base 0 failures, head 3,508.
  • Artifacts lens:
    • Drift tests (chronicle feed, vendoring, validation levels, release coverage manifest, cross-grain declarations): 53 tests, 0 failures, 1 skipped.
    • tools/ci_test_plan.py verify: ok. tools/census_uk_local_targets.py --check: current. tools/graph_acceptance_burndown.py --verify: ok, 41 green.
    • Streamed sha256 of the abf0921 feed matches the pin.
  • Tests lens:
    • test_cross_grain.py, test_uk_chronicle_feed.py, test_uk_domestic_energy_targets.py, test_uk_full_targets.py: 81/81.
    • test_uk_ledger_targets.py, test_uk_off_surface_cells.py, test_uk_population_targets.py: 106/106.
    • test_uk_national_calibration.py, test_uk_target_references.py, test_uk_energy_pricing.py: 92 (87 passed, 5 feed-gated skips).
    • Vendoring, benefit scope, local census, local validation levels, release coverage manifest, state pension, consumption resources and fan-out sum bridges: 85 (79 passed, 6 skipped).
    • test_country_spec.py, test_uk_release_input_coverage.py: 141/141.
    • Ladder targets, local targets, UC source periods, local geography and local rowwise: 110 (109 passed, 1 skipped).
    • Mutation removing abs(): 126 passed, 0 failed. Mutation disabling the check: 3 failed, which confirms the tests reach the code.
    • A synthetic probe of the authority tier, and a run of the committed UC cells through the bounded bridge (they pass at 1.00105).
  • Claims lens: 8 test files/nodes: 129 tests, 0 failures. The repo's _signed_difference_check_failures on the committed receipt: 3,508 failures. Tracemalloc estimates on 1% feed samples.
  • Second fresh-eyes pass: test_cross_grain.py, test_uk_off_surface_cells.py, test_uk_domestic_energy_targets.py and 3 test_uk_ledger_targets.py nodes: 81 tests, 0 failures. A synthetic probe of the LA bound, and a check of the committed receipt against the head membership (3,508 of 3,508 UC rows fail, 0 of 15,767 non-UC). uk_national_gate_manifest() gives 19 entries, including the local parity gate.
  • Verifiers:
    • A gate replay: base 0 failures, head 3,508.
    • A mutation check for 1302-05, and synthetic probes for 1302-02, 1302-04, 1302-06 and 1302-14.
    • Memory re-estimates: 13.20 / 22.04 GiB (1302-03) and 16,118 B/row (1302-c01).
    • test_uk_off_surface_cells.py: 20 tests, 0 failures. The single node test_the_uc_bridge_bound_refuses_an_incomplete_authority_tier passes.
    • Fixed-string feed greps for the HVCTS inputs: 26,212 identical lines on both pins.
  • Not run: engine-gated tests (CI engine-uk passed) and every whole-feed tool.

Checked and found sound

  • The pin identity: facts sha256 2b64fecf…, manifest sha256 b90fb1b8…, 1,499,301 rows, schema digest d953306d…. It matches the abf0921 artifact, and the two tests that restate it agree.
  • Of the 26 vendored resources, 24 change only the feed header. The 2 DESNZ resources keep identical values, with gas re-stamped to gas_year (56 regional, 347 LA rows).
  • National membership: only the 12 OBR lines change fact key, with no value moves. They are stamped GB (8) or E&W (4), as TARGET_ID_GEOGRAPHY_PINS declares. The model-side scoping was already GB or E&W.
  • Local membership: statuses are unchanged (25,309 active). Values move only on the 3,508 UC cells, and all of them bind payment_indicator: Yes, including the 3plus operands. One constituency's 12-month paid mean matches the feed.
  • The UC bridge on the committed surface is 1.00105 at constituency and about 1.0011 at LA, so the build should not trip the new bound at this pin.
  • SUBNATIONAL_PERIOD_TYPES is used at every sub-national read. The Energy Trends level (quarter) is unaffected.
  • The obr.state_pension diagnostic key has the same value at K03000001.
  • The national compile-parity receipt still matches the head membership (1,261 rows). The OBR restamp cannot fail that gate, because its keys are (name, period).
  • The restamp leaves the target-weight nation grain and the cross-grain, fan-out and band declarations unchanged.
  • The charter-H2 fixture moves exactly 12 stage-contract hashes, and recomputation matches.
  • The dfi_ni.bus.passenger_receipts expectation, updated here to 152.5m, was already stale on main.
  • uk_target_surface_local_default_2025 is a structural check, so 1302-01 is the only local preflight gate this PR makes stale.
  • The PR adds no new expiry dates or wall-clock tests.

Refuted during verification

  • 1302-13 (the NI energy interim justified by the electricity basis): dropped. The pinned NI gas row carries its own DESNZ "not directly comparable" caveat (meter-profile classification against GB's 73,200 kWh threshold).
  • 1302-03: narrowed from medium to PLAUSIBLE low. The national arms on abf0921 ran at this head with flat peak memory, and the facts are dropped after their node.
  • 1302-01: sub-claim corrected. docs/uk-chronicle-feed-repin.md:213 does not say "nothing reads it". Only PR body line 42 does.
  • 1302-11: sub-claim dropped. The continuation date is recorded in the adjudication text, and nothing gates local adjudications on approved_on.
  • 1302-c01: sub-claim narrowed. Concurrent xdist workers are hypothetical (xdist is not in addopts), and macOS would more likely swap than kill.
  • X-s01: sub-claim corrected. The "third build / re-solve" cost is overstated: the dense path refuses at preflight, before the solve.
  • X-m02: refuted as a merge-specific defect by one voter. It reduces to 1302-01.
  • X-s03: refuted as a plan contradiction by one voter. R0 folding was planned with a fallback, and Port the uk-data wave to policyengine-uk 2.134.4: stamp duty, Move to UC, national home values (sixth #1095 PR) #1308's engine is now 2.134.4.

Not covered

  • A full UK target compile on abf0921, including the new 2% bound on a registry compiled from the feed rather than from the committed membership.
  • The feed-gated tests on abf0921: both regeneration tests, national compile parity, state pension, pension credit, pension-age, council tax, CGT, income-tax rate classes, HMRC rate bands and the vendoring drift test. Only the childcare and bus test has run (by the PR, per its body).
  • Byte-for-byte regeneration of the national and local references and memberships from abf0921. Only spot checks were done.
  • A rebuild of the HVCTS stock against the new pin (its inputs were shown to be unchanged).
  • Engine-gated tests at this head (charter-H2 parity, tests/engine/uk). The only evidence is CI.
  • Consumers outside this repo, such as the uk-candidate-eval scoring register.
  • Memory of the release certifier and of the dense/full build on the new pin.

🤖 Generated with Claude Code

juaristi22 and others added 6 commits October 17, 2026 11:39
The release-blocking preflight gate uk_ledger_compile_parity_local_incumbent_2025
compares each signed value with the live compile, so a receipt left on the
old pin blocks every dense or full build and the release-cut certifier. Only
the 3,508 UC area cells move: 3,507 signed values change, and one cell now
matches the incumbent and leaves the receipt.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The UC bridge's 2% bound read each leg after earlier groups had rescaled it.
The exact-signature group rescales the authority tier onto the constituencies
first, so the bridge's authority factor was one whatever the authorities
said. The bound now reads each lower grain's input total. A surface with
off-surface cells is bounded on the pass that carries them; the first pass
records the factor without refusing. The shared operator validates the bound
(finite and positive), and each leg's receipt records it with the factor it
read.

Tests: refusals below one and exactly at the bound, a grain another group
rescales first, invalid bounds, the deferred check, a UK control at 0.92 of
its cells. The off-surface refusal test is renamed for what it shows, and the
incomplete-authority-tier test is rebuilt on a consistent control, with one
off-surface case on the real rule. The UK loader names the declaration kind
of a bad bound.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A gas meter row stamped by calendar year read as a missing fuel, and the
gas-share applier blamed an authority without a gas network. The meter read
now refuses a row on another period type by name. The test fixtures spell out
the feed's period types instead of reading the constant under test, and a
test reads the committed meter counts over the roster.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Four feed-gated helpers read the whole 7.4 GB consumer feed into memory to
hash it, and the council-tax one parsed every row. They now hash it in
chunks, and the council-tax helper parses only the MHCLG rows: the compile
reports every other reference unsupported.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The twelve OBR table 4.9 notes name the Great Britain or England and
  Wales code the fact now carries.
- The two DESNZ notes describe the Northern Ireland interim as it stands
  past the re-pin.
- The re-pin procedure says both compile-parity receipts are read at build
  time, lists the high-value home stock check, and records that a metadata
  move re-versions the national register.
- The abf0921 history entry and the changelog fragment record the
  regenerated receipt, the UC value moves, the stock's unchanged inputs,
  the scoring-register re-freeze, and the period-type pairing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Re-worded with María's sign-off of 2026-10-17 (microcosm#1302 review):

- uc_unit_vs_household_grain: the area series count paid claims since the
  abf0921 re-pin, as the national rows do, so the bridge moves the areas by
  about 0.1% and refuses a factor more than 2% from one at either grain. The
  adjudication records the rewording; its all-claims clause goes.
- desnz_northern_ireland_interim_share: the interim stands until a change
  stacked on the re-pin derives Great Britain's part from Northern
  Ireland's facts, without naming a formula. Its census fence says the same,
  and the census is regenerated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22

Copy link
Copy Markdown
Collaborator Author

Addressed in 6715824, item by item:

Blocking

  • 1302-01. The local compile-parity receipt is regenerated on abf0921. Only the 3,508 UC cells move: 3,507 signed values change, and one cell now matches the incumbent and leaves the receipt. Nothing else moves. The body no longer says nothing reads it, and the re-pin procedure now says both receipts are read by release-blocking preflight gates.

Should fix

  • 1302-02. The bound now reads each lower grain's input total, so the authority tier is bounded even though the exact-signature group rescales it first. The declaration's "by constituency and local authority" is now true, and it says so.
  • 1302-04. The old test is renamed test_the_uc_bridge_bound_refuses_a_caseload_far_from_its_cells and asserts the constituency leg. test_the_uc_bridge_bound_refuses_an_incomplete_authority_tier is rebuilt on a consistent control (GB 200 over 100 + 100) with Scotland's authorities at 60, and asserts the LA leg at 1.25.
  • 1302-07. Noted in the body, the history entry and the changelog. Re-freezing the uk-candidate-eval scoring register is listed as a follow-up.
  • 1302-05. Below-one refusals are added. Shared: factors of exactly 0.75 and 1.25, refused at a bound of 0.2 and accepted at 0.25. UK: a control at 0.92 of its cells.

Minor

  • 1302-03. Listed under "Not run locally".
  • 1302-06. The bound is checked on the pass that carries the off-surface cells. The first pass records the factor without refusing.
  • 1302-08. The procedure lists the stock check. The builder's 26,212 input facts are identical on both pins, which the history entry records.
  • 1302-09. test_the_uc_bridge_bound_counts_off_surface_cells_on_the_last_pass runs your consistent case (GB 200, S12000033 60, off-surface S12000034 40) on the real rule, and the body mentions the lifted bound.
  • 1302-10. The partition reason and the Declare cross-grain target reconciliation for the UK local surface: national wins, sub-national rescales (#495 increment 6 companion) #802 doc are reworded. With María's sign-off, so are the two signed adjudications: uc_unit_vs_household_grain now describes the paid-claims series, and desnz_northern_ireland_interim_share (X-s02) no longer names a formula. The interim's census fence matches, and the census is regenerated.
  • 1302-11 and 1302-12. Both DESNZ notes and all twelve OBR notes are reworded.
  • 1302-14. _validate_rule refuses a bound that is not finite and positive. Leg receipts record the bound, the factor read on input totals, and whether it was checked.
  • 1302-15. The loader names the declaration kind, and a negative test covers it.
  • 1302-16. The fixtures spell out the feed's period types. A test reads the committed meter counts: 350 authorities, 347 with both fuels. A meter row on another period type is now refused by name.
  • 1302-17 and 1302-18. The changelog gives the sizes and no longer implies that the selector pairs fuel with period type.
  • 1302-c01. The four helpers hash the feed in chunks, and the council-tax one parses only the MHCLG rows.

Cross-PR. #1305, #1306 and #1308 rebase onto this. #1305 also regenerates its own local references after its Northern Ireland change (the 694 kWh cells). Which Northern Ireland formula to keep (X-s02) is still open; #1302's texts no longer pick one.

🤖 Generated with Claude Code

@juaristi22

Copy link
Copy Markdown
Collaborator Author

Re-review of this PR at 6715824

Run with Claude Code on 17 October 2026, after the five commits answering the first review (d8493a3 to 1ced3d3) and 6715824.

Verdict

This PR is ready to merge on its own terms.

  • The first review's blocker is fixed.
    • The local compile-parity receipt is regenerated on abf0921. The repo's own gate replay now gives 0 failures against the committed membership; at 4c51843 it gave 3,508.
    • A second reviewer rebuilt the receipt from the head membership with the tool's own code and got a byte-identical file.
    • Only ledger_value moves, on 3,507 UC rows. One cell (children_1@E14001205) now equals the incumbent and leaves the receipt. Every other row is identical.
  • CI is green at 6715824: 10 of 10.
  • What's left: one medium test gap, a few low items and nits, and the ordering of the stack, which is now the critical path.

Should fix

  • No test makes the second reconciliation pass refuse (medium; test gap).
    • In production every surface has off-surface cells (1,598 on the committed membership), so only the second pass checks the bound (uk_runtime/ledger_targets.py:2654-2655, :2737-2738).
    • All the refusal tests run without off-surface cells, so they only exercise the first pass.
    • Two mutations each leave every one of the 96 bound tests green, and GB 300 over 100+100 plus an unrelated off-surface cell is then accepted:
      • turn the bound check off on the second pass;
      • restore the old if off_surface_rows: guard.
    • Fix: add one real-rule refusal with off-surface cells present. For example, _uc_surface([_off("local_authority", "E06000001", 5.0)]) must raise the constituency 1.5 message.

Low

  • The bound checks each grain's GB total, not each leg.
    • Where: cross_grain.py:556-580.
    • The exact-signature group still rescales each leg's authorities with no bound. Synthetic probe on the real rule: GB 200, constituencies 100/100, authorities Scotland 60 / Wales 140. It is accepted, with Scotland ×1.667 and Wales ×0.714, while both bridge legs read 1.0.
    • No wrong numbers today: the committed per-leg factors are 1.00103–1.00107.
    • The response's "the declaration's 'by constituency and local authority' is now true" holds for GB totals only.
    • Fix: bound the exact group's per-leg factor on bridged lower sides, or reword four places to "each grain's GB total": the docstring at :45-49, the declaration reason, changelog line 3 and the test name at test_uk_off_surface_cells.py:429.
  • The regenerated receipt carries a stale signed UC rationale.
    • The constants at tools/build_uk_ledger_compile_parity_signed_differences.py:428-447 still say "fixture/ours 0.8925-0.9003 across all 982 total rows", a "uniform national payment-distribution rescale".
    • At head the 980 rows run P5–P95 0.937–0.973, and each area now takes its own paid share. The children text has the same problem.
    • Text only: the gate doesn't read reasons. It is signed text, so it needs María's sign-off, then the next regeneration picks it up.
  • Two more whole-feed readers.
  • 1302-03 is only partly fixed. The body now names the peak-memory risk of the dense/full target load and the certifier on 1,499,301 rows, but nobody has measured it before 20–21 October.
  • Make the gate replay a test. A replay of the gate on the committed membership needs no feed and runs in under a minute. As a test, it would catch the next stale receipt, which UK local surface before the Budget build: high-value homes OBR leg, per-vehicle fuel flag, Northern Ireland energy share and housekeeping (#1278) #1306 will create when it rebases.

Nits

  • 1302-18 is partly fixed. The PR body ("The local-authority selection reads gas by gas_year…") and docs/uk-chronicle-feed-repin.md:213-214 keep the cross-product-implying wording; only the changelog was fixed.
  • The energy code still says "reads the share".
    • Where: the module docstring at domestic_energy_targets.py:15-20, UK_ENERGY_LEVEL_BASIS at :84-91 and the interim text at :220-228.
    • 6715824 made the adjudication formula-neutral, but these strings still describe the share method.
    • UK_ENERGY_LEVEL_BASIS is stamped into each kWh cell's metadata, so reword it together with the Northern Ireland change's regeneration.
  • Gas is now gas year 2024, but domestic_energy_targets.py:25-26 still says calendar 2024 kWh.
  • Wrong file name. docs/uk-chronicle-feed-repin.md:57 names uk/high_value_home_stock.json, which doesn't exist. The real files are high_value_home_stock_facts.json and .provenance.json.
  • Off by one. docs/uk-chronicle-feed-repin.md:229 says "the 3,508 UC cells' signed values move". In fact 3,507 move and 1 leaves the receipt.
  • The shared operator accepts a bound of 1 or more. Only the UK loader enforces a bound below 1.
  • The HVCTS stock provenance still names 735820c. The procedure now lists the check, but nothing runs it automatically.
  • Small-cell deferrals are still only tested with the bound lifted. The committed surface has 0 UC deferrals, so nothing is affected today.

First-round dispositions

  • Fixed: 1302-01, -02 (as GB totals; see Low), -04, -05, -06 (code; the test gap is under Should fix), -07 (documented; the re-freeze itself is still owed), -08 (as a recorded check), -09, -10 (at 6715824), -11, -12, -14, -15, -16, -17 and -c01.
  • Partly fixed: 1302-03 and -18.
  • No longer applies: 1302-13 (refuted in the first review).
  • Mutation checks:
    • Dropping abs() now fails 2 tests; in the first review the same mutant passed everything.
    • Reading the bound off the rescaled factor fails 2.
    • Dropping validation fails 5.
    • Reverting the first-pass change fails 1.
    • Two second-pass mutations fail nothing (Should fix).

Cross-PR, now the critical path

Disclosure check

The new public text is clean. That covers the six new commit messages, the regenerated receipt (values and two header counts only), the changelog, the docs, the selector notes, the body and the response comment. All figures are DWP, OBR, DESNZ or MHCLG admin figures, or synthetic test values.

Tests and tools run

  • At 1ced3d3:
    • 573 tests, 0 failures, 10 feed-gated skips.
    • A further 739 across three batches, 0 failures.
    • tools/ci_test_plan.py verify ok; tools/census_uk_local_targets.py --check current; tools/graph_acceptance_burndown.py --verify ok.
  • At 6715824: census --check current, ci_test_plan.py verify ok, and 87 census, rowwise and energy tests pass.
  • CI at 6715824: 10/10 green.
  • The feed was only stream-searched; nothing loaded it whole.

🤖 Generated with Claude Code

@juaristi22
juaristi22 merged commit 1f1cfa6 into main Oct 17, 2026
10 checks passed
juaristi22 added a commit that referenced this pull request Oct 17, 2026
The register still compared Microcosm with uk-data as it stood on 1
September (8629dbb). uk-data's main is now a43e8c5 (15 October), 136
commits on. Re-extracted there, the inventory has 57 modules. obr.py,
hmrc_salary_sacrifice.py and tests/test_target_registry.py changed, and
three sources are new. The re-extraction changes no existing concern's
classification; one is corrected by hand (below).

New concerns:
- dwp_housing_benefit.py (uk-data#490) splits DWP's Housing Benefit
  line by age. The pension-age half is ported: Microcosm binds
  dwp.hb.amount_pension_age and the pension-age caseloads. The
  working-age half, net of supported and temporary accommodation, is
  routed to #867 and fenced against binding the full line.
- dwp_pension_credit.py (uk-data#510) is ported: obr.pension_credit
  (stamped Great Britain by #1302) and the dwp.pension_credit rows.
- ons_labour_market.py (uk-data#529) is a blocked source: the feed has
  no LFS employment series.

The five obr.py line citations follow the functions that moved. Two
verdicts are corrected. Band D entered uk-data's LA loss only as the
multiplier in England's housing/council_tax_net proxy, never as a
column. The council-tax outlier guard is a ported compile contract: each
authority's cells compile from its own row, and the local compile-parity
preflight pins every one. uk-data#482 and #484 are still open; a second
re-pin follows when they merge.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Oct 17, 2026
The local-authority energy rows are levelled onto the LCFS stage's Energy
Trends level, a United Kingdom total. DESNZ's sub-national figures cover
Great Britain, so Great Britain's part of the level was taken at its
share of UK households (ONS LFS), an interim fenced until Chronicle
carried Northern Ireland's consumption. abf0921 (#1302) brought those
rows, and María ruled on 16 October that the change stacked on the
re-pin reads the share from them.

Great Britain's part of the level is now its share of DESNZ's
sub-national domestic consumption with Northern Ireland's row added (all
domestic meters, 2024), so each fuel's factor is the UK level over the
UK sub-national total. Northern Ireland's electricity runs on financial
years from April and its gas on its own gas years. Both labelled 2024,
they pair with Great Britain's 2024 as Great Britain's gas year pairs
with its calendar-year electricity. DESNZ calls the Northern Ireland
figures not directly comparable with Great Britain's, and the basis text
and the receipt say so.

desnz_local_energy_facts.json now vendors the two Northern Ireland rows
in place of the four LFS household totals; its 697 meter rows are
unchanged. The resource was cut with the tool's own vendor_resource from
a grep superset of the pinned artifact's lines, under #1302's pin. The
desnz_northern_ireland_interim_share adjudication and its census fence
retire, and the two energy contract rows' notes say how Northern Ireland
is taken out.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Oct 17, 2026
)

Review of microcosm#1306, item I.

The local compile-parity receipt, which the release-blocking preflight
reads, still carried the retired household-share factor on its 694 kWh
rows. Regenerated on abf0921 at this head, exactly those rows move (348
electricity, 346 gas, ledger_value only), each by its fuel's new factor
over the interim's: 0.99901 and 1.01582. The local membership report
moves on the same cells' holds and resolved values. Everything else in
both files, and the local references, is byte-identical to #1302's
full-feed regeneration.

Both generators hold the whole feed in memory, which this machine cannot.
They ran on a cut of the pinned artifact, digest checked against the pin:
the constituency and local-authority rows matching a local contract
selector. No local reference reads operands outside its own selector, so
the cut holds every candidate, as the unchanged rows confirm.

María signed off a rewording of the desnz_subnational_consumption_basis
adjudication and its census fence on 2026-10-17. One sentence on
Northern Ireland's periods and comparability is added, with dates
unchanged, so the caveat stays under an adjudication now the interim's
fence is gone. The census also changes:
- DESNZ's two Northern Ireland publications are family sources,
  verified 2026-10-17.
- The electricity note says Northern Ireland is published separately.
- The module docstring and the verified-on comment no longer describe
  the interim.

I's fragment says why the share stands over the plan's subtraction
(María's ruling of 2026-10-17): the sub-national statistics run about
9% (electricity) and 12% (gas) above Energy Trends, and the share keeps
both nations on that one basis. The re-pin history entry points here.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Oct 17, 2026
On main after #1302: 25,327 compiled cells. The 18 Scottish BRMA rows are
ledger-only by construction, and no signed value moves. The UC drift
rationales now read their fixture/ours bands off the rows, and the support
floor's deferred cells carry their own signed rationale. The release-blocking
preflight gate uk_ledger_compile_parity_local_incumbent_2025 reads this
receipt, so whichever of #1305, #1306, #1307 and #1308 lands last
regenerates it on the combined tree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Oct 17, 2026
After rebasing onto the #1300 re-pin, the references, memberships, both
compile-parity receipts and the census are regenerated once from the
final code against the pinned abf0921 feed, in the order the re-pin
procedure gives. The rebase had kept main's version of each generated
file at every conflict, so nothing in them was merged by hand.

National: the two high-value homes region rows signed out below the
floor leave the references and the receipt's ledger_only rows and are
recorded as signed_excluded, 1,319 active references of 1,341
candidates. Local: the references are byte-identical to the rebased
ones; the receipt drops the 2,323 deferred high-value homes cells, which
leaves the union the #1302 re-review computed, and restates its 3,507
UC rationales in the new wording; the membership carries the floor
rule's new text. No other value or row moves. The census check is
current, the gate digests and the US spec-engine attestation are
unchanged, and the count pins move to the regenerated files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Oct 17, 2026
The register still compared Microcosm with uk-data as it stood on 1
September (8629dbb). uk-data's main is now a43e8c5 (15 October), 136
commits on. Re-extracted there, the inventory has 57 modules. obr.py,
hmrc_salary_sacrifice.py and tests/test_target_registry.py changed, and
three sources are new. The re-extraction changes no existing concern's
classification; one is corrected by hand (below).

New concerns:
- dwp_housing_benefit.py (uk-data#490) splits DWP's Housing Benefit
  line by age. The pension-age half is ported: Microcosm binds
  dwp.hb.amount_pension_age and the pension-age caseloads. The
  working-age half, net of supported and temporary accommodation, is
  routed to #867 and fenced against binding the full line.
- dwp_pension_credit.py (uk-data#510) is ported: obr.pension_credit
  (stamped Great Britain by #1302) and the dwp.pension_credit rows.
- ons_labour_market.py (uk-data#529) is a blocked source: the feed has
  no LFS employment series.

The five obr.py line citations follow the functions that moved. Two
verdicts are corrected. Band D entered uk-data's LA loss only as the
multiplier in England's housing/council_tax_net proxy, never as a
column. The council-tax outlier guard is a ported compile contract: each
authority's cells compile from its own row, and the local compile-parity
preflight pins every one. uk-data#482 and #484 are still open; a second
re-pin follows when they merge.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Oct 17, 2026
The local-authority energy rows are levelled onto the LCFS stage's Energy
Trends level, a United Kingdom total. DESNZ's sub-national figures cover
Great Britain, so Great Britain's part of the level was taken at its
share of UK households (ONS LFS), an interim fenced until Chronicle
carried Northern Ireland's consumption. abf0921 (#1302) brought those
rows, and María ruled on 16 October that the change stacked on the
re-pin reads the share from them.

Great Britain's part of the level is now its share of DESNZ's
sub-national domestic consumption with Northern Ireland's row added (all
domestic meters, 2024), so each fuel's factor is the UK level over the
UK sub-national total. Northern Ireland's electricity runs on financial
years from April and its gas on its own gas years. Both labelled 2024,
they pair with Great Britain's 2024 as Great Britain's gas year pairs
with its calendar-year electricity. DESNZ calls the Northern Ireland
figures not directly comparable with Great Britain's, and the basis text
and the receipt say so.

desnz_local_energy_facts.json now vendors the two Northern Ireland rows
in place of the four LFS household totals; its 697 meter rows are
unchanged. The resource was cut with the tool's own vendor_resource from
a grep superset of the pinned artifact's lines, under #1302's pin. The
desnz_northern_ireland_interim_share adjudication and its census fence
retire, and the two energy contract rows' notes say how Northern Ireland
is taken out.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Oct 17, 2026
)

Review of microcosm#1306, item I.

The local compile-parity receipt, which the release-blocking preflight
reads, still carried the retired household-share factor on its 694 kWh
rows. Regenerated on abf0921 at this head, exactly those rows move (348
electricity, 346 gas, ledger_value only), each by its fuel's new factor
over the interim's: 0.99901 and 1.01582. The local membership report
moves on the same cells' holds and resolved values. Everything else in
both files, and the local references, is byte-identical to #1302's
full-feed regeneration.

Both generators hold the whole feed in memory, which this machine cannot.
They ran on a cut of the pinned artifact, digest checked against the pin:
the constituency and local-authority rows matching a local contract
selector. No local reference reads operands outside its own selector, so
the cut holds every candidate, as the unchanged rows confirm.

María signed off a rewording of the desnz_subnational_consumption_basis
adjudication and its census fence on 2026-10-17. One sentence on
Northern Ireland's periods and comparability is added, with dates
unchanged, so the caveat stays under an adjudication now the interim's
fence is gone. The census also changes:
- DESNZ's two Northern Ireland publications are family sources,
  verified 2026-10-17.
- The electricity note says Northern Ireland is published separately.
- The module docstring and the verified-on comment no longer describe
  the interim.

I's fragment says why the share stands over the plan's subtraction
(María's ruling of 2026-10-17): the sub-national statistics run about
9% (electricity) and 12% (gas) above Energy Trends, and the share keeps
both nations on that one basis. The re-pin history entry points here.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vahid-ahmadi

Copy link
Copy Markdown
Contributor

@juaristi22 thanks, this is a post-merge review at 6715824ad.

Verdict: the re-pin is sound and nothing here needs reverting. There are no musts. Two shoulds on the new factor bound and four nits can go into one follow-up.

I checked it against the list of re-pin needs from the chronicle#359 review, and every item is handled apart from the minor _normalize_period_value one (nit 4). The merge commit 1f1cfa615 has the same tree as 6715824ad.

Should (follow-ups)

  1. should: no test exercises the bound on the pass production actually checks. Every real surface has off-surface cells, so only the second reconcile() (uk_runtime/ledger_targets.py:2738) enforces max_factor_shift. The first pass is unchecked there (:2655).

    • I mutated :2738 to reconcile(check_factor_bounds=False). All 5,053 UK engine-free tests still pass (34 skipped), so the bound could be switched off in production and CI would stay green.
    • This test fails under that mutant and passes on head, so it would close the gap:
      def test_the_uc_bridge_bound_refuses_on_the_off_surface_pass():
          target = "dwp.uc.households_by_area"
          with pytest.raises(ValueError, match=r"the constituency rows of leg .* by 1\.5 "):
              _uc_surface([_off("local_authority", "S12000034", 40.0, target_id=target)])
    • Your re-review found the same gap. I'm repeating it because the PR merged without the test.
  2. should: the bound covers each grain's GB total, not each leg. The input-total read (cross_grain.py:555-580) does stop the exact-signature group from hiding a short authority tier in aggregate. But that group still rescales each leg's authorities onto its constituencies with no bound.

    • Probe on the shared test's surface (test_bridge_bounds_a_lower_grain_another_group_rescales_first): constituencies E 60 / S 40 under a control of 100, authorities E 99 / S 1. The bridge accepts it, both bridge legs read 1.0, and the Scottish authority is scaled by 40.
    • The declaration reason (uk/cross_grain_declarations.json:60), the class docstring (cross_grain.py:45-49) and the changelog say each grain is bounded. That's true only of the GB totals.
    • No value is wrong today, since the committed factors sit near one. The fix is to apply the bridge's bound to the exact group's per-leg factor on a bridged lower side, or to say "each grain's GB total" in those places.
    • Your re-review listed this as low. I'd rate it a should, because the declaration and docstring state a guarantee the code doesn't give, and a per-leg error is exactly what a broken authority tier would look like.

Nits

  1. nit: the kWh references don't name their period type. The desnz.subnational.{electricity,gas}_kwh contract selectors (uk/uk_population_targets.json:31630, :31669) pin only source_name and source_concept. So a future restamp of those facts would not fail by name. That is short of docs/uk-chronicle-feed-repin.md:214, which says "every sub-national lookup names its period type". The meter reads and the GB level factor do name it. Adding period_type from SUBNATIONAL_PERIOD_TYPES to the two selectors would make the doc true. Relatedly, domestic_energy_targets.py:25 still says "consumption is calendar 2024 kWh", and gas is now gas year 2024.
  2. nit: _normalize_period_value (ledger_targets.py:3785) still has no gas_year. Nothing selects on the declared type today, so values match as plain years and the membership resolves. UK: re-pin the Chronicle feed to chronicle#359 (paid-claims UC area cells, OBR table 4.9 geography, DESNZ gas years and Northern Ireland energy) #1300 listed it as minor, but the PR neither adds it nor records a decision. Adding it to annual_types, or one line in the procedure, would stop it being lost.
  3. nit: the shared validator accepts a bound of 1 or more (cross_grain.py:1027-1037). Only the UK loader (uk_runtime/cross_grain_declarations.py:136) requires (0, 1). A bound of 1 or more is never meaningful for "the two sides measure one quantity", so the shared check could match.
  4. nit: the region vendor selection excludes Northern Ireland only through geography_vintage: publication_2025_nspl. It works: the re-vendored 112 rows carry no Northern Ireland geography, so Northern Ireland's rows don't carry that vintage, and the count would catch a change. But pinning the GB geographies or domestic_scope would state the intent directly, as the local-authority selection does.

What looks right

  • Our list from chronicle#359:
    • UC. All four child bands and households_by_area add payment_indicator to dimensions and pin Yes. The committed local references carry payment_indicator: Yes on all 3,508 UC cells plus the 1,896 three-plus operands.
    • Region energy. The selection re-vendors at 112 rows with no Northern Ireland geography.
    • Table 4.9. All twelve lines are stamped in target_references.json: 8 at K03000001 and 4 at K04000001. The obr.state_pension diagnostic is at K03000001, and the generator's TARGET_ID_GEOGRAPHY_PINS carries the twelve.
    • Bind local-authority energy and regional bus journeys before the Budget (#1207) #1286's selection. It reads gas_year and calendar_year at domestic_scope: all_domestic and still gets 697 rows.
    • UK digest. It is d953306d… in both uk/chronicle_feed.json and the test.
    • NI caveats. The non-comparability caveat is in the code, the vendor purpose, the fence and the adjudication.
  • Gas years. I diffed both DESNZ resources against the pre-merge base.
    • 56 country and region gas rows (28 consumption, 28 meters) and 347 local-authority gas meter rows change only period.type (calendar_year to gas_year) and their fact keys.
    • No value, dimension, period_value or period_coverage changes, and no row is added or dropped.
    • So gas year Y pairs with calendar year Y exactly as before, and the shares and GB levels can't move.
  • The bound's logic.
    • It reads the lower rows as given (local_frame), not as an earlier group left them. Validation refuses bools, non-finite numbers and values of zero or less. Each bounded leg's receipt records the bound, the input factor and whether it was checked.
    • Mutations: reading off reconciled fails 2 tests, a one-sided bound fails 2, no check fails 7, checking on the first pass fails 1, and dropping validation fails 4. Only the second-pass mutant survives (item 1).
  • The 0.921 to about 1.001 move. The explanation is right. Before, every area's all-claims count was scaled by the GB paid share. Now each cell is its own area's paid count, and the remaining gap (6,197,311 against 6,190,821, a factor of about 1.00105) is the GB claims with no area. A 2% bound is about 20 times that gap, and it would refuse a return to all-claims cells (7.9% off). That seems a sensible width.
  • NZ. nz/chronicle_feed.json pins chronicle 5c4912a, which predates the digest change, so its cc9efb90… agrees with the NZ artifact it names. The only NZ check is test_real_nz_artifact_matches_committed_feed_pin, and it skips unless NZ_CHRONICLE_ARTIFACT is set. No test compares the UK and NZ digests, so nothing fails now. The NZ pin has to move only when NZ next re-pins, as the body says.
  • The NI interim ruling is recorded with approver, date, a fence and an expiry (2026-12-15) in local_binding_adjudications.json. It is mirrored in the census fence, the energy code and the selector notes.
  • Disclosure. The body, changelog, docs, commit messages, receipt and tests carry only published DWP, OBR, DESNZ and DfI aggregates, shares, or synthetic values. There are no microdata counts, minima, maxima or ESS.

What I checked

  • Setup: a fresh clone at merge commit 1f1cfa615 (tree identical to 6715824ad) with a new uv venv. I read the body, UK: re-pin the Chronicle feed to chronicle#359 (paid-claims UC area cells, OBR table 4.9 geography, DESNZ gas years and Northern Ireland energy) #1300, the two review comments and your response, and the full non-generated diff against 866fee4e7.
  • Tests:
    • the full engine-free CI job list: 18,928 passed, 163 skipped. The 2 failures are the two orrery-parser tests. They shell out to node scripts in tools/orrery-contract, which has no node_modules here because I didn't run the CI script's npm ci step. They aren't related to this PR.
    • all UK engine-free tests under the second-pass mutant;
    • the six bound mutations on the four bound-test files;
    • the proposed test in item 1, run on head and under the mutant.
  • Lint and checks: ruff check . is clean, and ruff format --check is clean on the 21 touched Python files. These report current: census_uk_local_targets.py --check, build_uk_release_input_coverage_manifest.py --check, ci_test_plan.py verify and graph_acceptance_burndown.py --verify.
  • Data: the vendored DESNZ resources diffed old against new, the UC and OBR selectors in the committed references, the NI vintages in chronicle#359's NI packages, and the per-leg probe in item 2.
  • Not checked: I didn't run the UK-engine or integration CI jobs locally; CI was green on them. I didn't have the 7.4 GB feed. So I couldn't rerun the vendoring, the reference and membership generators, the compile-parity receipt or the feed-gated tests. Those regenerations rest on your run and your re-review's byte-identical receipt rebuild. I did check the receipt diff: only ledger_value on UC rows, the two header counts and the one removed cell change. I didn't measure peak memory on the larger feed either.

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

UK: re-pin the Chronicle feed to chronicle#359 (paid-claims UC area cells, OBR table 4.9 geography, DESNZ gas years and Northern Ireland energy)

2 participants