Skip to content

Place UK private rent and BRMA at the assigned area; bind Scotland's rents at BRMA grain (#1262, #1090) - #1305

Draft
juaristi22 wants to merge 22 commits into
mainfrom
uk-private-rent-area-1262
Draft

juaristi22 wants to merge 22 commits into
mainfrom
uk-private-rent-area-1262

Conversation

@juaristi22

@juaristi22 juaristi22 commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator

Puts private rent and the Broad Rental Market Area (BRMA) at each placed household's assigned area on the local line (#1262, item 4 of #1123), and binds Scotland's PIPR rents at the grain ONS publishes them. Draft until the head rebuilds: one spine and one dense build, compared with the 15 October main build (María's ruling of 16 October). #1307, which defers the high-value-homes cells that stopped a dense build on main, has merged, so the head builds can run. The PR also waits for Max's view on the direction, asked on PolicyEngine/policyengine-uk-data#521.

Rebased onto main after #1307 and #1306 merged.

Closes #1262. Closes #1090. Folds in and supersedes #1092: its overlap table, decision note and rent dispersion receipt, rebuilt on the area-to-BRMA layer.

The Northern Ireland energy interim is not retired here. María ruled on 17 October that #1306 retires it with its share formula, and #1306 has merged; this PR's subtraction commit was dropped.

Summary

B1. BRMA from the assigned area

  • The layer. uk/brma_area_layer.json places every 2021 LSOA of England and Wales and every 2022 output area of Scotland in one BRMA. tools/build_uk_brma_area_layer.py rebuilds it from Chronicle's raw registrations, pinned by sha256:
  • The node. uk.full.geography.brma (--brma-localisation area, the default on the atomic assignment) reads each placed Great Britain clone's BRMA off its LSOA or output area, so its LHA rate describes the same place as its local authority and constituency.
    • Northern Ireland and the national line keep the spine's frs_brma draw.
    • A terminal, release-blocking gate, uk_local_brma_enum_domain, rechecks the column; an off-domain value stops even a development build. The full and dense battery digests move.

B2. Private rents from the lists of rents

  • The quantile table. uk/brma_rent_quantiles.json holds 25 knots per GB BRMA and bedroom category B to E. Its sources:
    • England: VOA's April 2026 list;
    • Wales: Rent Officers Wales, October 2023 to September 2024 (ATISN 25142 Table 2);
    • Scotland: the Scottish Government's 2025 quartiles with the shape of Rent Service Scotland's 2016–21 lettings.
  • The node. uk.full.private_rent (--private-rent-localisation brma_list) redraws each GB private renter's positive rent, in both channels.
    • Each renter keeps its rank among its region and bedroom category's FRS-channel renters. SPI rows are ranked against that reference, and all clones of a household share a rank.
    • It reads that quantile off its assigned BRMA's list.
    • One scale per region holds the region's FRS mean rent on the census private-renter mix of its BRMAs (María's ruling, kept on 17 October). Placement follows all households, not renters, so at design weights a region's level moves by the gap between the two mixes until calibration's tenure rows restore it; the receipts report that level as a named check.
    • A sampled development rung ranks and scales a region whose sample holds fewer than ten FRS renters on Great Britain's renters, counted in the receipt. A full-scale build uses each region's own renters and refuses a region with none.
    • Category A is never used. Owners, social renters, zero rents and Northern Ireland are untouched, and the boarder, lodger, subletting, service-charge and reported Housing Benefit columns stay as reported. The receipt carries rounded record counts only.

B3. Scotland's rents at BRMA grain (#1090)

  • The rows. ons.rent.private_rent_brma binds PIPR's 18 Scottish BRMA rows. They sit at the crosswalk's new statistical_scope level and on a partial brma area type: households outside Scotland belong to no BRMA area.

  • The values. Each row is 12 × the BRMA's calendar-2025 mean × its private renters.

    • The renters are derived after reconciliation from the councils' tenure/private_rent cells, through Census 2022's council shares of each BRMA's renters (UV404 by output area).
    • So the BRMA counts add up to Scotland's reconciled renters.
  • The 32 council cells stay signed deferred; councils have no published rent. The parity concern cross_grain_private_rent_scotland_brma resolves to the BRMA rows.

  • Selection. --target-geographies can name brma. Without area BRMAs the rows would bind on the spine's regional draw, so they are left out and counted; naming brma without area BRMAs is refused.

  • Weighting. The rows weigh inside the local-authority grain under every rule, grain_equal included.

  • Diagnostics. A BRMA row's model figures leave a build only when at least 10 distinct source households of positive weight carry them (María's ruling of 17 October). A thinner row keeps its place and its failure. Its estimates and errors are blank in:

    • the calibration diagnostics and the targets table;
    • the gate report, whose fit gates name it by key only;
    • the manifest and the scorer;
    • the sign-off drafter and Orrery's summaries.

    It stays out of every rollup, the family fit gate included. While any row is withheld, the diagnostics' totals cover the published rows, labelled in uk_diagnostics.model_figures_withheld (María's ruling of 18 October). The release contract accepts such a row only if all its model fields are null and the block counts it.

  • Active local references move from 22,986 to 23,004.

Texts and receipts

For review

  • The rent rows bind PIPR's level; the node holds the FRS level, which sits below it. With tenure rows bound, calibration can close the gap only by moving weight toward higher-rent renters within each area. That is the level trade-off of UK private rent at the assigned area (#1123 item 4, "PR B"): what is in place and the pieces still missing #1262's A3/A4 rulings, declared on both cross-grain routes rather than modelled. The 18 BRMA rows' fit in the head build shows its size.
  • A withheld row stays out of the family fit gate's decision. uk_local_per_family_fit decides on the published rows, so a family share never covers a row whose figures are withheld; uk_local_target_fit still decides on every row. For a problem that binds BRMA rows, Orrery's size summary publishes no total over every row, since it cannot tell which rows a build withholds.
  • The BRMA rows weigh inside the local-authority grain's private-rent family. A grain of their own would take a fifth of the loss for 18 rows (a quarter under grain_equal) and push the energy and bus families below their loss-share floors.
  • Cross-border BRMAs. Wales's list is a year older than England's, and 16 LSOAs read the other nation's list under their own region's scale: West Cheshire's 11 Welsh LSOAs, and Monmouthshire's 4 and Brecon and Radnor's 1 English LSOAs.
  • Calibration diagnostics gain a brma geography level without a schema bump, as additive levels have before. Reports written before it may leave it out, and no out-of-repo reader validates the level set.
  • The local candidate scorer counts the 18 BRMA rows as candidate-only rows and labels them in its output.
  • Certification. The full-manifest pins, the declared gate count (73) and the exclusions move, so a national or dense release certified before this merges fails validate_release after it. Sequence the merge against the staged national bundle and the frozen build.
  • The planned uk_local_brma_area_layer gate was dropped. The BRMA node fails closed on any area it cannot place, or places in another FRS region.

Not yet run

  • The head spine and dense builds, and the receipts they feed:
    • rent fit by nation, region and category, and the design-weight regional level check;
    • the 316 + 18 rent rows;
    • PIPR by district and bedrooms;
    • the moves in the UC housing element (all, and by tenure), Housing Benefit (obr.housing_benefit), and AHC poverty on the national and local-authority rows;
    • the mean LHA rate, and UC renters above their LHA rate against DWP (indicative: a hand-copied CIH table).
  • These follow before the PR leaves draft. The receipts will carry aggregates only. The receipt tool counts carriers by distinct source household of positive weight, FRS-channel only for spine values. It withholds any rent row, cell, group or share resting on fewer than ten, with complements until the withheld cells rest on ten together. It refuses to write inside any git work tree.

Tests

  • The tools' --check modes reproduce their committed resources: the area-to-BRMA layer, the Scottish overlap and the rent quantiles, run locally on 17 October. The quantiles were rebuilt for a text change after their check, and all three resources are byte-identical at this head.
  • The touched tests pass locally, and the full suite runs in CI.
  • Since the review, the tests also cover:
    • the summary writers' disclosure floors;
    • a Monte-Carlo check that redrawn rents placed on the census mix keep the FRS level;
    • the BRMA enum gate's structural stop;
    • grain_equal shares with the BRMA rows;
    • both measurement modes' release refusals, and the override flags in none mode;
    • the size identity moving with a rent pin;
    • the sampled-rung fallback;
    • the BRMA codes and metric;
    • a pinned digest of the supports' area-to-region pairs, which CI checks against the layer;
    • the crosswalk's layer pin without the ladder.
  • Since the re-review, they also cover:
    • the production fit bindings and the terminal gate kernel on a withheld BRMA row, and the kernel failing closed when a binding writes its error;
    • an attempt to recover a withheld row from the diagnostics' totals;
    • a dense CLI run whose targets table carries a withheld row;
    • the scorer, the release contract, the sign-off drafter and Orrery's summaries with a withheld row;
    • the receipt tool's main end to end, its per-row carrier rule and every writer's work-tree guard;
    • a thin sampled region ranked on Great Britain's renters, strict_cells at both ends, and the Monte-Carlo level check at 1%;
    • build_uk_full_local_problem with a Scottish BRMA row.

🤖 Generated with Claude Code

@juaristi22

Copy link
Copy Markdown
Collaborator Author

Independent adversarial review of this PR at 29e5c5f

Run with Claude Code on 16–17 October 2026 at 29e5c5f: seven independent review lenses (correctness, method, generated artifacts, tests, claims against the diff, disclosure, a second fresh-eyes pass), then one to three skeptics per finding who tried to refute it; (k/n) is how many agreed. No builds and no feed-gated tests were run. The head has not moved since.

Leave it draft. It is not ready to merge. The BRMA layer, the rent quantile table, the Scottish overlap, the B3 composer and the rent node all check out against their sources, and every touched test passes at head 29e5c5f. Three things block it:

Before the receipts run, close the disclosure gaps in the receipt tool and fix the region-scale claim. Otherwise the first numbers will either be misread or be unsafe to quote.

Blocking

Should fix

  • The region scale assumes the census renter mix, but placement follows the all-household mix, so the "FRS level" claim fails at design weights.

    • Verdict (1305-01): CONFIRMED by 2/2 skeptics, medium.

    • Where: private_rent_localisation.py:297-340, where scale = mean_rent / level and the level comes from census private-renter BRMA counts. Placement is in atomic_area_support.py:420-427, which draws areas by all-household count within the FRS region and has no tenure term.

    • What goes wrong: from committed public resources only, Scotland's realised over assumed level is:

      • by band: B 0.909, C 0.940, D 1.004, E 0.966;
      • overall about 0.953, with a total variation distance of 0.122 between the two mixes;
      • Lothian holds 19.3% of renters but 13.1% of households, and Greater Glasgow 15.9% against 13.0%.

      The LSOA-count proxy elsewhere gives London −3.7%, North West −1.8%, Wales −1.4% and East of England +1.8%. A toy run through localise_rents, with 50/50 placement against an 80/20 renter split, gives 0.833 of the FRS mean; placing rows on the renter mix gives 1.000.

    • Texts that state the false consequence: graph_population.py:707, the module docstring at private_rent_localisation.py:9-12, docs/uk-full-build-graph.md:31, changelog.d/1262-uk-private-rent-from-area.added.md and the body. The receipt's region_mean_rent_after_over_before (tools/uk_private_rent_receipt.py:730-741) will show about 0.95 for Scotland, and readers will take that as the effect of the lists.

    • What limits it: weights are untouched, and the local-authority tenure/private_rent rows pull the mix back during calibration. The toy tests (test_uk_private_rent_localisation.py:214, :357-389) already place renters on the census mix, so they cannot show the gap.

    • Fix, either one:

      • Compute s_r on the pool at the node: m_r divided by the weighted mean of Q over FRS-channel rows at their assigned BRMAs. This is exact at design weights but K-invariant only in expectation, so it gives up what test_localise_rents_does_not_depend_on_k pins.
      • Or keep the census mix on purpose and reword the five texts to say the level holds once calibration restores the renter mix. Then add the design-weight regional level to the receipts as a named check.
  • The 18 Scottish BRMA rows bind roughly the list (PIPR) level that the rent node removes.

    • Verdict (1305-03): PLAUSIBLE, 1/2 skeptics.
    • Where: ledger_targets.py:2272-2408, where total = months * mean * renter_households (:2359) with no levelling, against private_rent_localisation.py:332.
    • What goes wrong:
      • Against the SG 2025 published means, PIPR 2025 S33 means range from −16.5% to +16.4% per BRMA. The mean gap is about 0, so the list level is about 0.99 of PIPR.
      • The node holds Scotland at the FRS level. With tenure counts bound, calibration can close the gap only by shifting Scottish weight toward high-rent renters within each BRMA.
    • Why only plausible: the intent skeptic found the trade-off on record, in UK private rent at the assigned area (#1123 item 4, "PR B"): what is in place and the pieces still missing #1262 A3/A4, María's uk-data#521 comments of 13 October (Q4) and 16 October, and commit 609ebfd. The 316 England and Wales LA rows carry the same tension. What remains is that nothing in Place UK private rent and BRMA at the assigned area; bind Scotland's rents at BRMA grain (#1262, #1090) #1305 says these rows sit above the node's level.
    • What would settle it: the fit of the 18 rows in the head dense build, plus the shift in Scottish renter weights and in UC/HB exposure.
    • Fix: add one line to "For review" and to the cross-grain routes (cross_grain_declarations.json:612, :648) declaring the expected level miss. Or bind the rows as a pattern, rescaling the 18 targets to Scotland's FRS-level total, and decide the same for the LA rows.
  • Under --target-weight-rule grain_equal, "brma" becomes a grain of its own and takes 25% of the loss.

    • Verdict (1305-02): CONFIRMED by 2/2 skeptics, medium.
    • Where: local_rowwise.py:1273-1276, which takes raw area_type labels, and :1298-1302, which passes them unmapped and skips the floor check. The same unmapped path appears at :2074-2077 (holdout) and at target_weights.py:507-510 (uk_rule_loss_weights). UK_LOCAL_WEIGHT_GRAIN is applied only in _rule_grains (target_weights.py:381-392).
    • What goes wrong: with realistic counts (650 constituency, 361 la, 18 brma and 364 national labels), each grain takes 0.25 of the loss, and a BRMA row weighs about 20 times an LA row. The national share also drops from 1/3 to 1/4, so a grain_equal arm on the head is not comparable with main's. The dense posture allows grain_equal (rowwise_posture.py:270-274). Release candidates refuse non-default rules (rowwise_cli.py:676-678), so no release is affected. The doc at docs/uk-scotland-private-rent-brma-1090.md:159-162 and the body state the mapping without that condition.
    • Fix: map the labels through UK_LOCAL_WEIGHT_GRAIN in all three places, and keep the raw labels for the target_rows.grain alignment check. Add a test that grain_equal weights do not change when the 18 brma rows are added.
  • The structural stop on the new gate uk_local_brma_enum_domain is untested.

    • Verdict (1305-04): CONFIRMED by 2/2 skeptics, medium; one skeptic rated it low.
    • Where: full_gates.py:336-340 (structural_local).
    • What goes wrong: with the gate patched out of structural_local in memory, the result was unenforced_release_failures at f0.1 and exportable_blocking at f1.0, with artifact_permitted=True in both, and all 7 existing classification cases still passed.
      • The parametrize list at test_uk_full_gates.py:333-352 leaves the gate out.
      • test_uk_battery_bindings.py:209-214 stubs enum_domain to pass, so the assertion at :978-981 proves nothing.
    • Narrowed:
      • Broken terminal evidence fails closed (gate_battery.py:645-722).
      • A release would still be blocked by the dense contract's _UK_DENSE_RELEASE_BLOCKING_IDS (contract.py:753-764, 4697-4702).
      • So the worst case is a diagnostic candidate exported with an off-domain BRMA.
    • Fix: add the gate to the parametrize list, and add a terminal-phase test that runs the unstubbed binding on a frame with an off-domain BRMA and asserts FAILED and a structural stop.
  • The functions that write the quotable private_rent_summary.json have no tests.

    • Verdict (1305-07): CONFIRMED by 2/2 skeptics, medium.
    • Where: tools/uk_private_rent_receipt.py:655-679 (measure_households), which calls rents_by_cell (:358-385), pipr_comparison (:475-534), weight_concentration (:582-596) and coherence (:599-618).
    • What goes wrong: the docstring (:31-41) says this file is the only one that may be quoted in a PR or a committed receipt, and that every cell rests on at least 10 distinct source households. But test_uk_private_rent_receipt_tool.py tests only the building blocks, and nothing sanitises the output before the write at :800-801. Synthetic probes reproduced three defects, listed in the next four bullets. rents_by_cell's suppression and pipr_comparison's cell exclusion behaved correctly in the probes, but no test pins them.
    • Fix: write synthetic-frame tests on measure_households' summary covering:
      • a region × category cell under 10 source households reads "<10" and withholds a complement;
      • a district cell under 10 is excluded;
      • weight_concentration and coherence withhold figures below 10 source households;
      • a group with fewer than 3 cells is withheld;
      • a mixed-channel spine-values cell.
  • top_1pct_weight_share has no record or household floor.

    • Verdict (1305-08): CONFIRMED by 1/1, low.

    • Where: tools/uk_private_rent_receipt.py:582-596, where top = max(1, ceil(0.01 * n)). :670 puts it in the summary.

    • What goes wrong:

      • With 41 rows it prints a single row's weight share.
      • With 900 rows it rests on 9 rows.
      • With 5,000 rows whose top 50 are clones of 2 households, it still prints.

      It bites only on small or subsampled runs, but nothing enforces that limit.

    • Fix: print "<10" unless the top slice has at least 10 rows spanning at least MIN_SOURCE_HOUSEHOLDS distinct source_key values. Or move the figure to the audit file.

  • The coherence unweighted share floors on clone rows, not source households.

    • Verdict (1305-23): CONFIRMED by 1/1, low.
    • Where: tools/uk_private_rent_receipt.py:614, floored_share(mask.sum(), mask.size). weighted_share at :144-160 counts distinct keys instead.
    • What goes wrong:
      • 8 of 800 households × 25 clones gives weighted "<10" but unweighted 0.01.
      • A complement case (all but 3 of 100 households) gives 0.97.
    • Fix: pass the carriers, and print "<10" when either side has 1-9 distinct source keys.
  • pipr_comparison summarises groups of one or two district cells.

    • Verdict (1305-24): CONFIRMED by 1/1, low.
    • Where: tools/uk_private_rent_receipt.py:517-529 sets no minimum cell count. Only _spearman (:467-472) refuses below 3.
    • What goes wrong: in a one-cell group, median_ratio is that district's model/PIPR ratio. Multiplying it by the public PIPR mean gives back the model district mean; in a toy case, 1.25 × 800 = 1000. That breaks the docstring's "no figure describes a single district or BRMA" and the sub-GOR rule.
    • Fix: withhold the group's statistics unless at least 3 cells pass, and preferably 10.
  • Spine-values cells count SPI copies under their FRS parent's key.

    • Verdict (1305-25): CONFIRMED by 1/1, low.
    • Where: tools/uk_private_rent_receipt.py:345-355 (_mean_cell counts source_key.nunique() across channels) and :711-714 (the spine block). Related code: spi_support.py:260-266, :726-746, and spi_housing_shell.py:505-552, which redraws SPI tenure, bedrooms and rent.
    • What goes wrong: a toy cell with 4 FRS-channel renters and 6 SPI copies reports 10 carriers and prints a mean. Narrowed: the QRF interpolates across many donors rather than copying a few. Still, fewer than 10 real reported rents can pass the floor.
    • Fix: for the spine block, count carriers over FRS-channel rows only, or drop spine-values rents_by_cell from the summary.
  • Licensed outputs have no guard against landing in the repo.

    • Verdict (1305-26): CONFIRMED by 1/1, low (defensive).
    • Where: tools/uk_private_rent_pool.py:117-119 writes pool.npz by default to <run>/audit-1262/. It holds per-household rent, subrent, reported HB, weights and source keys. Also tools/uk_private_rent_receipt.py:745-747, :774, :800-801, and engine_rent_diagnostics.py:208, :268-269.
    • What goes wrong: .gitignore covers *.h5, out/ and build/, but not *.npz, audit-*/ or runs/. A run directory inside the checkout plus a later git add -A would stage unit-record files. The guard pattern already exists at tools/build_uk_efrs_parity_reference.py:565, :623.
    • Fix: refuse output paths under the repo root, and add *.npz and audit-*/ to .gitignore.
  • "Not yet run" omits UK private rent at the assigned area (#1123 item 4, "PR B"): what is in place and the pieces still missing #1262's acceptance measures.

Minor

  • The toy-graph test comment contradicts its own data, and the scale test re-derives the formula.

    • Verdict (1305-05): CONFIRMED by 2/2 skeptics, low.
    • Where: test_uk_private_rent_localisation.py:372-375 says the region mean is unchanged, but the toy renter goes from 9,000 to 5,271.02, which is 9,000 × (0.5 + u).
    • What goes wrong: a kernel that writes the rent back unchanged passes this test, though the London test catches it. A category-blind π passes the scale and redraw tests, because the toy gives every category the same mix (uk_atomic_support_fixtures.py:319).
    • Fix: correct the comment, assert that the rent changes, give the toy category-specific mixes, and add a property test that the weighted redrawn mean holds within Monte-Carlo tolerance.
  • The London toy test's ratio check is conditional.

    • Verdict (1305-19): CONFIRMED by 1/1, low.
    • Where: test_uk_private_rent_localisation.py:411-424. The if len(by_brma) == 2 branch and the subset assertions all hold on an empty frame.
    • What goes wrong: with all London clones in one BRMA, a write-back kernel passes everything, and so does the Lothian test. Seed 7 splits them 2/4 today.
    • Fix: assert len(london) == 6 and len(by_brma) == 2 unconditionally, and that the rents differ from 20,000.
  • A stale signed UC rationale is carried through the regenerated receipt.

    • Verdict (1305-09): CONFIRMED by 1/1, low.
    • Where: tools/build_uk_ledger_compile_parity_signed_differences.py:428-447.
    • What goes wrong:
      • The text claims a uniform fixture/ours of 0.8925-0.9003 across 982 rows. At the head the 980 rows span 0.793-1.048, with p5-p95 0.937-0.973 and 3 rows above 1.
      • For children rows the text claims p5-p95 of 0.77-1.19; the actual is 0.815-1.237.
      • The text was already wrong at the base; this PR re-signs it.
    • Fix: restate the rationale from the regenerated rows, or compute the band in the tool, and say what residual gap the paid-claims basis leaves.
  • "Certification part digests do not move" understates the effect.

    • Verdict (1305-11): CONFIRMED by 1/1, low.
    • Where: the body's line 15 and the changelog, against contract.py:3191-3200.
    • What goes wrong: the check compares spec.{gates_manifest_sha256, policy_sha256, spec_fingerprint} with the full-manifest pins this PR moves (:450-458). It also checks declared_entry_count, now 73, and the exclusions, which gain uk_local_brma_enum_domain (:811-814). Any national or dense release certified before the merge therefore fails validate_release afterwards (:6115-6130).
    • Fix: say so in the body, and sequence the merge against the staged national bundle and the frozen build.
  • --target-geographies cannot name brma.

    • Verdict (1305-12): CONFIRMED by 1/1, low.
    • Where: full_build_cli.py:399-411, graph_build.py:181-190, and graph_targets.py:124, :194, :405-407. target_selection.py:83-110 drops the rows silently.
    • What goes wrong: "country,region,constituency,la" used to equal "all" and now drops the 18 rows. The release check (rowwise_cli.py:701-708) still passes. The default is unaffected.
    • Fix: accept brma in the parser, config and help text, or imply it from la. At minimum, receipt the dropped rows.
  • Cross-border BRMAs mix list vintages under one region scale.

    • Verdict (1305-13): CONFIRMED by 1/1, low.
    • Where: tools/build_uk_brma_rent_quantiles.py:22-24, :556-559, and private_rent_localisation.py:297-339, :388-405.
    • What goes wrong:
      • 11 Welsh LSOAs in WEST_CHESHIRE read the newer English list under the Wales scale.
      • 4 South West LSOAs in MONMOUTHSHIRE and 1 West Midlands LSOA in BRECON_AND_RADNOR read the older Welsh list under English scales. These 5 are the VOA overrides at tools/build_uk_brma_area_layer.py:120-124.
      • A synthetic 5% vintage gap passes straight through. About 16 LSOAs are affected.
    • Fix: uprate the Welsh list to England's window, or qualify the "a nation's factor cancels" text and name these BRMAs.
  • --brma-localisation none still binds the BRMA rows, on spine labels unrelated to placement.

    • Verdict (1305-14): CONFIRMED by 1/1, low.
    • Where: full_problem.py:89-99. The spine BRMA comes from frs_brma.py:79-91 (keyed on benunit id, region × category only), there is no node in none mode (graph_population.py:1236-1239), and rowwise_cli.py:695-697 refuses none only for release candidates.
    • What goes wrong: the docstring at local_targets.py:134-147 is wrong in this mode. The CLI default is "area", and only the config falls back to none on the legacy assignment.
    • Fix: drop the brma grain, or fail closed, when the mode is none.
  • The private-rent node can refuse small dev rungs (f001).

    • Verdict (1305-15): CONFIRMED by 1/1, low; the skeptic had medium confidence.
    • Where: private_rent_localisation.py:269-280, :333-337, :385-387. Sampling stratifies by region only (national_sampling.py:236-245), and SPI renters are redrawn independently.
    • What goes wrong: a synthetic demo shows both refusals. How often this happens at real f001 needs licensed counts.
    • Fix: below f100, fall back to a GB-wide pool with a receipt, or resolve f001/f010 to none with a receipt. Keep f100 strict.
  • The layer-vs-supports region test always skips in CI.

    • Verdict (1305-17): CONFIRMED by 1/1, low.
    • Where: test_uk_brma_area_layer.py:314-337. build/ is gitignored and no CI job mounts the supports.
    • Mitigation:
      • The provenance sha pins run in CI (:94-108).
      • Release candidates check support digests (full_build_cli.py:1013-1020).
      • The invariant passes today on the pinned supports.
      • A drift fails closed at brma_area_layer.py:276-281 during the head build.
    • Fix: commit a compact region-by-area digest that an engine-free test can compare, or run the test in a job that has the supports.
  • The body claims the tools check their committed resources against a rebuild.

    • Verdict (1305-18): CONFIRMED by 1/1, low.
    • Where: the body's line 56.
    • What goes wrong: only test_uk_scotland_brma_la_overlap.py:274 runs a rebuild, and it skips without the feed. No test runs the layer or quantile --check, and no record shows one was run.
    • Fix: reword to say the tools have a --check mode run locally on a given date against a given commit, and record that run.
  • New CLI, size-identity and weighting paths are untested.

    • Verdict (1305-20): CONFIRMED by 1/1, low.

    • What goes wrong: a line trace over about 30 test files shows these lines never run:

      • the refusal raises at full_build_cli.py:909, :968;
      • the rowwise_cli.py:696, :699 refusals;
      • size_checkpoint.py:149-155, :158-165;
      • the brma branch at full_problem.py:93;
      • local_targets.py:144-147 and :353-360.

      _rule_grains never saw a brma label.

    • Fix: tests for the release-candidate override refusal, both "none" refusals, the size identity changing when a rent pin changes, brma grains in the target weights, and a local problem with a brma grain.

  • The Scotland doc promises per-area correction records the tool does not write.

    • Verdict (1305-22): CONFIRMED by 1/1, low.
    • Where: docs/uk-scotland-private-rent-brma-1090.md:21-27.
    • What goes wrong: the tool writes only counts (tools/build_uk_brma_area_layer.py:533-539): 9 filled, 59 corrected, 2 overridden. Measure Scotland's BRMA × council overlap; keep BRMA rents out of council targets #1092's 61 is probably 59 + 2.
    • Fix: state the counts and where they live, or write the list of areas into the provenance. Area codes are geography, so listing them raises no disclosure problem.
  • The 18 new BRMA rows need a carrier floor before any BRMA-level diagnostics leave the build. This one is María's call.

    • Verdict (1305-27): CONFIRMED by 1/1, low; the skeptic had medium confidence.
    • Nobody has measured carriers per BRMA yet. See the Disclosure check below.
  • Nits, each CONFIRMED by 1/1:

    • No schema bump for the new diagnostics level (1305-16). schema.py:22, :236-258: older readers reject brma rows. All in-repo readers are updated, and there is precedent for additive changes without a bump (21a4d3a). Say in the body that no out-of-repo reader validates the level set.
    • The scorer counts the 18 BRMA rows as candidate-only without saying so (1305-21). tools/score_uk_local_candidate.py:41-47, :331-333. The count move is disclosed and the scorer's output labels the rows. Add one line to "For review".
    • A cross-grain reason names the wrong composer (1305-28). cross_grain_declarations.json:648 cites uk_private_rent_mean_to_total; it should cite uk_private_rent_brma_mean_to_total (ledger_targets.py:2272).
    • Override and sha flags are silently ignored when the mode is none (1305-29). full_build_cli.py:898-900, :939-951. The property path already behaves this way. Refuse the flags in none mode.
    • The crosswalk's area_layer_sha256 is checked only by a test that skips in CI (1305-30). local_area_crosswalk.json:2136-2138; the test is at test_uk_local_area_crosswalk.py:34-43 and needs the ladder. The pin matches today. Add a ladder-free assert, or drop the field.
    • Two English UC fixture_only cells carry the Northern Ireland rationale (1305-31). tools/build_uk_ledger_compile_parity_signed_differences.py:548-549, :842-843. They belong to the support-floor deferrals, not the Northern Ireland class. This predates the PR. Route support-floor deferrals before the per-metric maps.
    • The parity text names the PIPR September edition, but the selector resolves the July record set (1305-32). data_target_parity.py:369, uk_data_target_parity.json:947. All 2025 values are identical across the two editions, and the error predates this PR (04f7202). Correct the wording.
    • The DWP above-LHA shares are hand-copied from a quote on uk-data#521 (1305-33). engine_rent_diagnostics.py:55-61; no Chronicle fact exists. Register the table in Chronicle, or cite the primary table and mark the comparison indicative.
    • "Rounded counts only" is not quite true (1305-34). At graph_population.py:709-710, two counts in the receipt are exact (:778-785), but they count cells and regions, not records. Reword to "rounded record counts only".

Cross-PR

Disclosure check

What the disclosure lens checked:

  • the body;
  • all 8 commit messages;
  • the 5 changelog fragments;
  • docs/uk-scotland-private-rent-brma-1090.md and the experiments/1090 script and JSON;
  • every committed resource: the layer and its provenance, the quantiles, the overlap, the compile-parity receipt, references, membership, population targets, cross-grain, parity, census and the DESNZ facts;
  • kernel receipts and error messages;
  • all new test fixtures, which are synthetic.

Every figure comes from Census/NRS, DESNZ/ONS, VOA, Rent Officers Wales, SG, RSS, PIPR, DWP admin counts or target-registry counts. The "30.9%" in 6421fe4 comes from Rent Service Scotland FOI lettings: admin data, outside UKDS scope. No committed content carries a FRS/LCFS/WAS/SPI/NTS figure, a record id, a per-record value or an extreme. The FRS level is applied only inside the kernel, the region scales and ranks never leave it, and the receipts round record counts.

Gaps, all in tooling that has not run yet, so nothing has been published:

  • 1305-07: no tests on the summary writers.
  • 1305-08: top_1pct_weight_share has no floor.
  • 1305-23: the unweighted share counts clone rows.
  • 1305-24: one-cell PIPR groups are summarised.
  • 1305-25: SPI copies are counted as carriers.
  • 1305-26: no repo-path guard on licensed outputs.

Close these before the receipts run. The PR plans to quote rent fit by region and category, and PIPR by district and bedrooms, from exactly this file.

For María to decide (1305-27): the 18 new BRMA rows need a carrier floor before any BRMA-level diagnostics leave the build.

The figures in this review are synthetic test values, counts of targets, areas or tests, or come from public census, rent-officer list, PIPR, SG, DESNZ and DWP sources.

Tests and tools run

All runs were at head 29e5c5f.

  • Correctness lens:
    • Rent localisation, BRMA layer, BRMA graph, quantile and Scottish overlap tests (test_uk_private_rent_localisation, test_uk_brma_area_layer, test_uk_brma_area_graph, test_uk_brma_rent_quantiles, test_uk_scotland_brma_la_overlap): 95 tests, 0 failures, 3 skipped (supports not mounted, feed absent).
    • The other touched test files (test_uk_domestic_energy_targets, test_uk_local_rowwise, test_uk_diagnostics, test_uk_local_target_census, test_uk_population_targets, test_uk_pipr_deferrals, test_uk_local_gate_battery, test_uk_private_rent_receipt_tool, test_uk_brma_area_layer_tool): 167 tests, 0 failures.
    • Build-preparation tests (test_uk_full_build_preparation, test_uk_size_l2, test_uk_rowwise_candidate, test_uk_local_candidate_scorer): 89 tests, 0 failures.
    • A check of the layer against the pinned supports: 0 missing, 0 extra, 0 region mismatches.
    • A check of the region levels: every GB region × category cell resolves.
    • A synthetic grain_equal weighting probe.
  • Method lens:
    • Rent localisation, quantile, layer, domestic energy and Scottish overlap tests: 100 tests, 0 failures, 3 skipped.
    • The engine test tests/engine/uk/test_uk_brma_area_layer.py under policyengine-uk 2.132.6: 3 tests, 0 failures.
    • test_uk_ledger_targets, test_uk_local_rowwise and test_uk_pipr_deferrals, filtered with -k 'brma or partial or pipr or private_rent': 14 tests, 0 failures.
    • Read-only checks on committed public resources: the NI energy level factors, the Scottish and England and Wales tenure-mix level ratios, and PIPR against SG means by BRMA.
  • Artifacts lens:
    • Resource tests (the BRMA layer and layer tool, quantiles, Scottish overlap, rent localisation, domestic energy, local target census, PIPR deferrals and population targets, plus shared/test_country_spec): 291 tests, 0 failures, 3 skipped.
    • Pin tests (shared/test_gate_battery_contract_pins, test_uk_local_gate_battery, test_uk_local_small_cell_rule): 37 tests, 0 failures.
    • The engine layer test under policyengine-uk 2.132.6: 3 tests, 0 failures.
    • tools/census_uk_local_targets.py --check and tools/census_uk_data_target_parity.py --check: both current.
    • tools/ci_test_plan.py verify: ok (735 files).
    • A consistency check of the quantile table and the layer (Scottish facts against the feed; monotone knots, means and bases; the layer's BRMA set against the quantile table's): 288 checked, 0 mismatches.
    • A check of the layer's BRMA names against BRMAName and LHA rates under policyengine-uk 2.134.5: passes.
  • Tests lens:
    • New BRMA and rent test files (test_uk_brma_area_graph, test_uk_brma_area_layer, test_uk_brma_area_layer_tool, test_uk_brma_rent_quantiles, test_uk_private_rent_localisation, test_uk_private_rent_receipt_tool, test_uk_scotland_brma_la_overlap): 112 tests, 0 failures, 3 skipped.
    • Modified test files, first set (test_uk_diagnostics, test_uk_domestic_energy_targets, test_uk_ledger_targets, test_uk_local_candidate_scorer, test_uk_local_gate_battery, test_uk_local_rowwise, test_uk_local_target_census, test_uk_pipr_deferrals, test_uk_population_targets, test_uk_rowwise_candidate, test_uk_full_gates): 312 tests, 0 failures.
    • Modified test files, second set (test_uk_battery_bindings, shared/test_country_spec, test_uk_orrery_contract, test_uk_size_l2, test_uk_full_target_graph, test_uk_take_up_gate): 239 tests, 0 failures.
    • Tests that cover the changed code (test_uk_target_weights, test_uk_target_weight_wiring, test_uk_full_measure, test_uk_local_targets, test_uk_cross_grain_declarations, test_uk_calibration_run, test_uk_release_certification, test_uk_local_small_cell_rule, test_uk_full_population_graph, test_uk_graph_terminal, test_uk_property_localisation): 157 tests, 0 failures.
    • microcosm-data shared/test_contract.py and microcosm-diagnostics shared/test_schema.py: 343 tests, 0 failures.
    • Engine tests (test_uk_brma_area_layer.py, test_uk_full_graph_admission.py): 4 tests, 0 failures.
    • A synthetic toy-node probe: 2 passed.
    • ruff check and ruff format --check on the 63 changed .py files: clean.
  • Claims lens: test_uk_domestic_energy_targets.py, test_uk_scotland_brma_la_overlap.py, test_uk_brma_rent_quantiles.py, test_uk_pipr_deferrals.py and shared/test_gate_battery_contract_pins.py: 77 tests, 0 failures, 1 skipped (the feed-gated overlap rebuild).
  • Disclosure lens: test_uk_private_rent_receipt_tool.py and test_uk_private_rent_localisation.py: 29 tests, 0 failures, plus synthetic disclosure probes.
  • Second fresh-eyes pass: the engine layer test (3 tests, 0 failures) and the engine-free rent localisation, BRMA graph, layer, quantile and domestic energy tests (80 tests, 0 failures, 2 skipped).
  • Skeptics:
    • 1305-04: 27 tests, 0 failures, plus a mutation check (all 7 classification cases pass against the mutant).
    • 1305-05: 7 passed.
    • 1305-17: 26 tests, 2 skipped.
    • 1305-18: 9 tests, 1 skipped.
    • 1305-21 (the scorer): 5 tests, 0 failures.
    • 1305-30: 1 test, skipped (it needs the ladder).
    • 1305-19: 5 tests, 0 failures, plus 1 further test, 0 failures.
    • A line trace over about 30 test files: 4 batches, all exit 0.
    • git merge-tree simulations of the cross-PR merges, and fixed-string searches of the feed.

Checked and found sound

  • Layer: it covers the pinned supports exactly (35,672 England and Wales LSOAs, 46,363 Scottish OAs), with 0 region mismatches. All 192 GB BRMA names are in BRMAName under policyengine-uk 2.132.6 and 2.134.5, and all have LHA rates.
  • BRMA roster: the layer, the quantile table and the count table agree. The 232 region × BRMA pairs match exactly.
  • Quantiles: knots are monotone. Each stored mean equals knot_mean. Scottish knots equal the published quartiles. The 288 Scottish published values match the abf0921 facts. Medians rise B ≤ C ≤ D ≤ E in every BRMA. No cell is modelled with fewer than 20 lets.
  • Rent node mechanics:
    • Units run weekly to annual through WEEKS_IN_YEAR on both sides of the scale.
    • The rank formula and the tie-break are K-invariant and do not depend on row order.
    • Northern Ireland, owners, social renters and zero rents are untouched.
    • Clones share a rank.
  • B3 composer: BRMA renters add up to the councils' reconciled total. All 32 council tenure cells are active. It runs after reconciliation.
  • No double counting: BRMA rows are Scotland-only, LA rent rows are England and Wales only, and Scottish council rent cells stay deferred.
  • Partial grain: the −1 area index is only ever compared with ==, never used to index, so nothing wraps onto the last BRMA.
  • Default weight rule: BRMA rows weigh inside the LA private-rent family, and the energy and bus floors are unaffected.
  • NI energy change: the units are consistent, and the level moves (−0.354% electricity, +1.448% gas) match the body.
  • Membership, references and receipt counts are internally consistent.
  • Resources: country_package.json lists all four new resources, and the retired items leave no dangling references.
  • Test changes: no assertions were deleted or loosened, no new test reads the clock, and the CI test plan verifies.

Refuted during verification

No finding was refuted outright. These parts were dropped or narrowed:

Not covered

  • The claim that each commit's touched tests pass. Every lens checked the head only.
  • build_uk_brma_area_layer.py --check and build_uk_brma_rent_quantiles.py --check were not run (they need geopandas, raw registrations and a full feed stream).
  • The logic of experiments/1090-scotland-brma-la-rent-dispersion.py. Only its JSON receipt was checked.
  • Whether the rent-officer lists exclude tenancies paid by HB or UC. If they do, the within-region pattern the node imposes leans toward lets without a claimant.
  • How the redrawn rents interact with reported HB/UC amounts and take-up inside policyengine-uk, beyond confirming that benunit_rent derives from household rent.
  • The head-build receipts. That includes the real Scottish weight shift behind 1305-03, f001 refusal rates (1305-15) and carriers per BRMA (1305-27).
  • A wider claims-lens test run, which never ran.
  • An untraced lead from one skeptic, not a finding: on Place UK private rent and BRMA at the assigned area; bind Scotland's rents at BRMA grain (#1262, #1090) #1305, the regenerated receipt's 694 DESNZ kWh ledger_values differ from the membership's resolved_values, by about ×0.986 for gas and ×1.004 for electricity. On main the two agree. It is worth checking whether the membership carries levelled energy values and needed regenerating after 6928d08. The feed-gated currency test that would catch this skips in CI.
  • Whether all 59 Balloch corrections really are around Balloch (1305-22).

🤖 Generated with Claude Code

@juaristi22
juaristi22 force-pushed the uk-private-rent-area-1262 branch from 29e5c5f to c5ec9f6 Compare October 17, 2026 15:19
juaristi22 added a commit that referenced this pull request Oct 17, 2026
From the #1305 review, before any receipt runs:
- carriers come from one column. A cell of spine values counts its
  FRS-channel households only, since an SPI-channel row's spine rent is
  drawn from FRS donors;
- the top 1%'s weight share needs ten rows over ten households;
- an unweighted coherence share needs both sides on ten households, not
  ten clone rows;
- a district or BRMA group is summarised only over ten cells;
- the summary, the audit, the pool table and the engine report are refused
  inside the repository, and .gitignore covers *.npz and audit-*/.

The design-weight regional level is now a named check,
region_level_at_design_weights. The DWP rent-above-LHA comparison is marked
indicative: it was copied by hand from a quotation, and no Chronicle fact
carries the table. Tests cover the summary writers: a thin cell and its
complement, thin district groups, the top 1%, clone rows, FRS-only carriers
and the repository guard.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Oct 17, 2026
María kept the census renter mix for the region scale on 2026-10-17 (the
#1305 review, item 01). The scale holds each region's FRS mean rent on the
census private-renter mix of its BRMAs. Placement follows all households, so
at design weights a region's level moves by the gap between the two mixes
until calibration's tenure rows restore it. The kernel and module docstrings,
the build-graph doc and the changelog now say so, and the receipt reports
that level as a check.

The quantile tool and its resource qualify "a factor common to a nation's
list cancels". It does not cancel for the 16 LSOAs whose BRMA crosses the
border: West Cheshire's 11 Welsh LSOAs read England's list under the Wales
scale, and Monmouthshire's 4 and Brecon and Radnor's 1 English LSOAs read
Wales's older list under English scales. The resource is rebuilt for the
text (--check reproduced the committed one first). The kernel's receipt says
rounded record counts only.

Tests:
- the toy counts give London's categories different BRMA mixes, so a scale
  that ignored the category fails;
- a Monte-Carlo check holds the weighted redrawn mean at the FRS level for
  renters placed on the census mix;
- the toy-graph test asserts that the rent changes;
- the London clone test asserts its six clones and both BRMAs
  unconditionally.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Oct 17, 2026
Only the row-metadata rules mapped the 18 Scottish BRMA rows into the
local-authority grain. Under grain_equal the raw labels made "brma" a grain
of its own, with a quarter of the loss for 18 rows: about 20 times an
authority row's weight, and the national share fell from a third to a
quarter (#1305 review). UK_LOCAL_WEIGHT_GRAIN now sits in local_doctrine,
and uk_local_target_loss_weights maps through it. That covers the solve,
the holdout and uk_rule_loss_weights at once, while the alignment check keeps
the raw labels. A test holds each grain's share when the rows are added.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22
juaristi22 changed the base branch from uk-chronicle-repin-1300 to main October 17, 2026 15:19
juaristi22 added a commit that referenced this pull request Oct 17, 2026
uk_local_brma_enum_domain's structural stop had no test. The classification
test gains it, and a new test runs the unstubbed enum binding on a frame
with an off-domain BRMA. The gate fails, and at both rungs the build stops
with no candidate exported (#1305 review).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Oct 17, 2026
From the #1305 review:
- --brma-localisation none no longer binds the BRMA rows on the spine's
  regional draw. The selection leaves them out and its payload counts them;
  an explicit brma level without area BRMAs is refused.
- --target-geographies can name brma, so "country,region,constituency,la"
  dropping the 18 rows is now a choice the selection receipt records.
- The layer and quantile override and sha flags are refused in none mode,
  where nothing reads them.
- On a sampled development rung, a region whose sample holds SPI-channel
  renters but no FRS renter ranks and scales on Great Britain's FRS renters,
  on its own census mix, and the receipt counts the cells and regions. A
  full-scale build still refuses.

Tests cover both measurement modes' release refusals, brma in
--target-geographies, the none-mode flag refusals and the release-candidate
override refusals, the size identity moving with a rent pin, the sampled-rung
fallback, the config rule, and the BRMA codes and metric.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Oct 17, 2026
María's ruling of 2026-10-17 on the #1305 review, item 27: a BRMA row's
model figures leave a build only when at least ten distinct source
households carry them. The support gate covers constituencies and
authorities, not the partial BRMA grain. A row's carriers are the distinct
source_household_ids of the positive-weight households in its problem row,
as the area support counts them.

A withheld row keeps its place and its failure everywhere the build writes
it, without its estimates or errors:
- the calibration diagnostics: model fields null, with a note in metadata.
  It counts as scored but never as passing, and stays out of the family
  rollup, so no total can be differenced against the published rows;
- the targets CSV and the gate report's target_diagnostics;
- the gate report: the fit gates' failure lines quote target and estimate
  and their details list every row's error and the worst offenders. A
  withheld row's line keeps its name only, and it leaves the error lists.

The gates still evaluate on the full figures.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Oct 17, 2026
)

From the #1305 review:
- Both private-rent routes in uk/cross_grain_declarations.json declare
  their level miss. The rows bind PIPR's level while the node holds each
  region at its lower FRS level, so with tenure bound, calibration closes
  the gap only by moving weight toward higher-rent renters (item 03). The
  BRMA route names its own composer, uk_private_rent_brma_mean_to_total
  (item 28).
- The compile-parity tool reads the UC drift rationales' fixture/ours bands
  off the regenerated rows instead of quoting stale ones, and says what the
  paid-claims basis leaves (item 09). It routes the support floor's signed
  deferrals before the per-metric maps, so E06000053 and E09000001 no longer
  read as Northern Ireland's (item 31).
- The parity census names PIPR's July 2026 edition, the record set the
  selector resolves (item 32).
- The Scottish overlap note states the layer's correction counts and where
  they live: 9 filled, 59 corrected by the postcode lookup, 2 Balloch
  overrides (item 22).
- A pinned digest of the atomic supports' (area, FRS region) pairs lets CI
  check the committed layer against supports it does not mount; the mounted
  test re-derives it (item 17). A ladder-free test checks the crosswalk's
  BRMA level against the committed layer (item 30).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 added a commit that referenced this pull request Oct 17, 2026
On main after #1302: 25,327 compiled cells. The 18 Scottish BRMA rows are
ledger-only by construction, and no signed value moves. The UC drift
rationales now read their fixture/ours bands off the rows, and the support
floor's deferred cells carry their own signed rationale. The release-blocking
preflight gate uk_ledger_compile_parity_local_incumbent_2025 reads this
receipt, so whichever of #1305, #1306, #1307 and #1308 lands last
regenerates it on the combined tree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22

Copy link
Copy Markdown
Collaborator Author

Addressed in c5ec9f6, rebased onto main after #1302 merged. Item by item:

Blocking

Should fix

  • 1305-01. María kept the census-mix scale. The five texts now say the FRS level holds on the census renter mix, and that at design weights a region moves by the mix gap until calibration's tenure rows restore it. The receipt reports region_level_at_design_weights as a named check.
  • 1305-03. Both private-rent routes, and the body, declare the level miss.
  • 1305-02. UK_LOCAL_WEIGHT_GRAIN moves into local_doctrine, and uk_local_target_loss_weights maps through it. That covers the solve, the holdout and uk_rule_loss_weights; the alignment check keeps the raw labels. A test holds each grain's share when the 18 rows are added.
  • 1305-04. The gate is in the classification parametrize. A new test runs the unstubbed binding on an off-domain BRMA, which fails and stops at both rungs.
  • 1305-07, -08, -23, -24, -25, -26.
    • Carriers come from one column, FRS-channel only for spine values.
    • The top 1% needs ten rows over ten households, and the unweighted coherence share counts households on both sides.
    • District and BRMA groups need ten cells.
    • The summary, the audit, the pool table and the engine report refuse paths inside the repository, and .gitignore covers *.npz and audit-*/.
    • Synthetic tests cover every summary writer.
  • 1305-06. "Not yet run" lists the UC housing element, obr.housing_benefit, and national and local-authority AHC. The body repeats the stay-as-reported columns.

Minor

  • 1305-05 and 1305-19.
    • The toy-graph comment is corrected, and the test asserts the rent changes.
    • London's toy categories have distinct BRMA mixes, and a Monte-Carlo test holds the weighted redrawn mean at the FRS level for renters placed on the census mix.
    • The London test asserts its six clones and both BRMAs unconditionally.
  • 1305-09 and 1305-31. The parity tool reads the UC bands off the regenerated rows and says what the paid-claims basis leaves. It routes the support floor's signed deferrals first, so E06000053 and E09000001 carry their own rationale.
  • 1305-11, -16, -21. Added to "For review".
  • 1305-12. --target-geographies accepts brma, and the selection receipt lists what a selector drops.
  • 1305-13. The quantile tool, its resource and the build-graph doc name the 16 cross-border LSOAs. The resource was rebuilt for the text after its --check reproduced it.
  • 1305-14 and 1305-29. Without area BRMAs the selection leaves the BRMA rows out and counts them; naming brma is refused. The layer and quantile override and sha flags are refused in none mode.
  • 1305-15. Below full scale, a region whose sample holds no FRS renter ranks and scales on Great Britain's, and the receipt counts the cells and regions. Full scale stays strict.
  • 1305-17 and 1305-30.
    • A pinned digest of the supports' area-to-region pairs (35,672 LSOAs, 46,363 output areas) lets CI check the layer; the mounted test re-derives it.
    • A ladder-free test checks the crosswalk's layer pin.
  • 1305-18. The layer, overlap and quantile --check runs are recorded in the body: 17 October at 3a45efd1b.
  • 1305-20. New tests cover:
    • both none release refusals;
    • the release-candidate override refusals and the none-mode flag refusals;
    • the size identity moving with a rent pin;
    • brma in the target weights;
    • the BRMA codes and metric.
  • 1305-22. The note states 9 filled, 59 corrected and 2 Balloch overrides, and where those counts live.
  • 1305-27. María's ruling: 10 distinct source households. A thinner BRMA row keeps its place and its failure in the calibration diagnostics, the targets CSV and the gate report, without estimates or errors, and stays out of every rollup. The gates still evaluate on the full figures.
  • 1305-28, -32, -33, -34.
    • The composer is named correctly.
    • The parity census names the July 2026 edition the selector resolves.
    • The DWP comparison is marked indicative.
    • The receipt says rounded record counts.

Cross-PR.

🤖 Generated with Claude Code

juaristi22 added a commit that referenced this pull request Oct 17, 2026
The energy module's docstring said consumption is calendar 2024 kWh. Since
the abf0921 re-pin the gas rows are DESNZ's gas year 2024, which the
sub-national lookups pair with calendar 2024 (SUBNATIONAL_PERIOD_TYPES);
electricity stays on calendar years. Docstring only.

The other wording the re-review names in this module (the module docstring's
account of the Northern Ireland interim, UK_ENERGY_LEVEL_BASIS and the
interim text in the household-share receipt) is left as it is: the two
stacked changes that retire the interim, #1305 and #1306, rewrite those same
lines, and UK_ENERGY_LEVEL_BASIS is stamped into each kWh cell's metadata, so
it moves with their regeneration.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22

Copy link
Copy Markdown
Collaborator Author

Re-review of this PR at c5ec9f6

Run with Claude Code on 17 October 2026, after the rebase onto main (post #1302) and the nine new commits. Three independent reviewers re-checked every first-round finding, attacked the new code with synthetic probes, checked the new public text against the UK Data Service rules, and simulated the merges with #1306, #1307 and #1308. No builds and no feed-gated tests were run.

Verdict

The rebase is clean:

But 48529f1, which withholds the model figures of BRMA rows resting on fewer than 10 distinct source households, has four high-severity problems. Two independent reviewers found them. Fix them before the head build:

  • one withheld row breaks the dense build;
  • three routes still give a withheld row's figures back.

The response's 1305-27 line ("without estimates or errors … and stays out of every rollup") is not yet true for the reasons below.

Blocking (before the head build)

  • One withheld BRMA row breaks the dense build after export (high; fail-closed).
    • The manifest projection reads the withheld targets CSV back (graph_terminal.py:1522), and the blank cells become NaN.
    • max_abs_relative_error, median_abs_relative_error (:1614, :1791-1796) and fit.local_by_family (:1865, whose np.argmax picks the NaN row as the worst) all go NaN.
    • json_text writes with allow_nan=False (rowwise_cli.py:153-164), so full_build_cli.py:2446 raises "Out of range float values are not JSON compliant: nan". That happens after the solve and the export, leaving no manifest and no build.json.
    • The scorer (tools/score_uk_local_candidate.py:78-103) also refuses any null final estimate, and the dense release requires score_vs_incumbent.json.
    • Plausible at release size: on public census counts, proportional selection at 100,000 households would put only about 10–18 private-renter rows in the smallest Scottish BRMAs (an expectation from the census, not a count of frame records), and distinct source households can only be fewer. It is near-certain on sampled rungs.
    • Fix:
      • Compute those manifest figures over rows with finite errors, and count the withheld rows.
      • Have the scorer skip withheld rows, counting them.
      • Add a projection test and a scorer test with a withheld row.
  • The real local fit gate's output keeps a withheld row's error (high; disclosure).
    • uk_withhold_gate_report_rows (uk_runtime/diagnostics.py:1017-1049) matches lines starting with ":" and details.relative_errors/worst_offenders. That is the generic target_fit_gate's shape.
    • uk_local_target_fit writes something else (terminal_gates.py:1019-1022, :1101):
      • lines like microcosm_uk_2024_25: private_rent/<BRMA>/rent/private_rent_brma relative error +x% exceeds 25%;
      • details.failing_targets;
      • rows keyed family/area/metric (battery_bindings.py:1391).
    • A synthetic probe on the production binding kept the failure line and the error after withholding.
    • uk_local_per_family_fit still counts withheld rows in its family shares and sizes. Set against weakest_families, which drops them, that gives each row's within-10% and within-25% status.
    • Where it spreads: the full gate report, the signed local report, the dense manifest's blocking_failures/diagnostic_failures, and staging.
    • The commit's test builds the generic shape by hand.
    • Fix:
      • Map each withheld row to its gate key, and remove it from every details mapping, list and failure line.
      • Publish family shares and sizes without withheld rows, with a withheld count.
      • Test against the real bindings, through UKFullGateKernel.
  • The calibration diagnostics' totals give a withheld row back by difference (high; disclosure).
    • Withholding blanks per-row fields only (diagnostics.py:74-81, :1143-1203).
    • initial_loss, final_loss, loss_trajectory, fraction_within_10pct and past_cap_census.pushed_out_rows are still computed over every row (microcosm-calibrate diagnostics.py:270-350, :436-438).
    • final_loss equals the sum of the published contributions to within 1e-12, and the withheld row keeps its target, target_loss_weight_share and target_loss_scale. So |estimate − target| = (final_loss − other contributions) / share × scale.
    • On the commit's own fixture this recovered the withheld estimate exactly, and pushed_out_rows lists the withheld row with its relative errors.
    • These files ship in the dense release.
    • Fix (María's call on the form):
      • while any row is withheld, publish the losses, trajectory and fractions over the published rows only, labelled as such, or null them;
      • take withheld rows out of pushed_out_rows;
      • add a test that tries to recover a withheld row.
  • The quotable rent receipt summarises the 18 BRMA rows with no carrier rule (high; disclosure).
    • tools/uk_private_rent_receipt.py:338-376 (rent_row_fit), written to summary["rent_rows"] at :798.
    • It gives the BRMA group's share_within_10pct, rows_over_25pct and median_abs_error.
    • With one of 18 synthetic rows withheld, the median's arithmetic returned its |error| (0.089 against a true 0.0891), and share × 18 minus the published count returned its within-10% status.
    • The PR plans to quote exactly these "316 + 18 rent rows".
    • Fix: count carriers per rent row as uk_withheld_brma_rows does, and leave withheld rows out of every group, with a count.

Should fix

  • A cell with zero FRS carriers passes every floor (medium; receipt tool). The floors test 0 < n < 10 (:213, :230, :250), and frs_only blanks the SPI keys. So a spine-values cell made only of SPI rows prints a mean with source_households: 0. Treat any non-empty side with fewer than 10 counted carriers, zero included, as "<10".
  • SPI copies count as their FRS parent for housing figures (medium; from code reading; María's call).
    • SPI copies carry their parent's lineage, but their tenure, bedrooms, rent, subrent and reported Housing Benefit are drawn from FRS donors (spi_housing_shell.py).
    • So the receipt's release and design blocks, and the BRMA carrier rule, overcount carriers in the way c4afed1 fixed for spine values only.
    • Fix: count FRS-channel rows only for housing figures.
  • The Monte-Carlo test's tolerance (rel=0.03) is loose.
    • The unmutated ratios are 0.995–1.000.
    • A mutant using one category's BRMA mix for every category gives 0.970–0.974, and all 25 tests still pass.
    • About 1% would catch it.
  • 1305-15 is partly fixed. A sampled region with one or two FRS renters still computes its own scale, and refuses when the scale leaves SCALE_BAND. cells_ranked_against_their_region double-counts fallback cells.

Low and nits

  • The repository guard knows only this checkout (uk_private_rent_receipt.py:97, :164-174, the pool tool, the engine script). It accepts a path inside another git work tree, and on a case-insensitive volume this checkout's own path written in different case. Refuse when any ancestor of the resolved path holds a .git entry.
  • Two thin cells get no complementary withholding. suppress_cells withholds a complement only when exactly one cell is thin (:254).
  • Zero-weight rows count as carriers in the receipt; the diagnostics rule uses positive weights only. Defensive today.
  • Tests that pass with the fix reverted.
    • Nothing runs the kernel's withholding.
    • Nothing calls measure_run, main or the pool tool's guard, so reverting frs_carriers=True or the guard call passes everything.
    • Nothing pins strict_cells.
  • 1305-11: changelog.d/1262-uk-brma-from-area.added.md still ends "while the certification part digests do not".
  • 1305-20 is partly fixed. There is no test of the scorer's statistical_scope/brma grain, or of build_uk_full_local_problem with BRMA rows.
  • Nits:
    • the floors' #: comment in local_doctrine.py:98-115 now documents UK_LOCAL_WEIGHT_GRAIN;
    • stale six-gate counts in three places (graph_national.py:1347, contract.py:4608, calibration_run.py:269);
    • local_validation_levels.json:27/47 still name the September PIPR record sets;
    • the cross-grain rent reasons lack 3b65352's design-weight qualifier;
    • the pool tool and the engine report check --out only after the expensive work;
    • max_target_scaled_change in the size receipt is a maximum over all rows, withheld ones included;
    • no changelog fragment mentions the null BRMA model fields.

First-round dispositions

  • Fixed: 1305-01 to -04, -06 to -09, -12 to -14, -16 to -19, -21 to -26 and -28 to -34. The parity UC bands are recomputed from the rows: 0.937–0.973 over 980, and 0.815–1.237 over 2,527.
  • Partly fixed: 1305-05 (tolerance), -11 (changelog), -15 (thin sampled regions) and -20 (two test gaps).
  • The fix introduces new problems: 1305-27 (the four Blocking items).
  • No longer applies: 1305-10, c01, X-m01, X-m02, X-s01, X-s02 and X-s06.
  • Future sequencing: c02, X-m04 and X-m05.

Cross-PR

Disclosure check

The new commit messages, added lines and body carry no survey-derived figures. The three leak routes in the new code are under Blocking.

Tests and tools run

🤖 Generated with Claude Code

juaristi22 and others added 10 commits October 18, 2026 02:42
…1262)

uk/brma_area_layer.json places each 2021 LSOA of England and Wales and
each 2022 output area of Scotland in one Broad Rental Market Area, so a
placed household's BRMA can be read off its assigned area (microcosm#1262,
B1). tools/build_uk_brma_area_layer.py rebuilds it from Chronicle's raw
registrations, pinned by sha256:

- England and Wales: LSOA 2021 population-weighted centroids in the VOA
  and Rent Officers Wales polygons, with policyengine-uk-data#516's
  overrides; its region x BRMA support table agrees exactly.
- Scotland: output-area centroids in the Scottish Government polygons,
  corrected by Rent Service Scotland's postcode lookup, with the two
  Balloch overrides of microcosm#1092; its 63 BRMA x council pairs agree
  exactly.

uk_runtime/brma_area_layer.py loads it and reads a BRMA per household.

microcosm#1092 folds in on the same layer. The Scotland BRMA x council
overlap now sums Census 2022 UV404 by output area from the pinned feed
(PolicyEngine/chronicle#326), so the ward split and its unregistered ward
export go: all-household cells are unchanged, private renters total
322,903 against the ward split's 322,995, and the decision note and the
rent dispersion receipt are rebuilt on it (BRMA rents still cannot become
council targets: up to 30.9% from a council's own mean).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
uk/brma_rent_quantiles.json holds 25 weekly-rent knots (0.005 to 0.995)
and the implied mean for each Great Britain BRMA and bedroom category B
to E, the shape the private-rent redraw reads (microcosm#1262, B2; the
level comes from the FRS, one scale per region).
tools/build_uk_brma_rent_quantiles.py rebuilds it from Chronicle's raw
registrations and the pinned feed:

- England: the VOA list of rents, April 2026.
- Wales: Rent Officers Wales, ATISN 25142 Table 2 (October 2023 to
  September 2024).
- Scotland: the Scottish Government's 2025 lower quartile, median and
  upper quartile per BRMA and size, with the shape of Rent Service
  Scotland's 2016-21 market evidence; each implied mean stays within 15%
  of the published one.

A cell with fewer than 50 lets borrows its nation's shape at its own
median (20 to 49 lets) or a log-additive BRMA plus category median.
Category A is never used. uk_runtime/private_rent_localisation.py holds
the table's loader and the redraw's pure functions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…1262)

Two post-geography nodes on the local line (microcosm#1262, B1 and B2):

- uk.full.geography.brma (--brma-localisation area, the default on the
  atomic assignment) reads each placed Great Britain clone's BRMA off its
  LSOA or output area from the committed layer, so its LHA rate describes
  the same place as its local authority and constituency. Northern
  Ireland and the national line keep the spine's frs_brma draw. A
  terminal, release-blocking uk_local_brma_enum_domain gate rechecks the
  column; the full and dense battery digests, the declared gate count
  and the certification exclusions move with it.
- uk.full.private_rent (--private-rent-localisation brma_list, the default
  with area BRMAs) redraws each GB private renter's rent with a positive
  rent, in both channels. Each keeps its rank among its region and
  bedroom category's FRS-channel renters (SPI rows ranked against that
  reference) and reads that quantile off its assigned BRMA's list. One
  scale per region holds the region's FRS mean rent on the census
  private-renter mix, so the lists set where rents sit within a region,
  not the level. Category A is never used; everything else stays bit for
  bit, and receipts carry rounded counts only.

Both modes ride the size checkpoint and the orrery geography group, and a
release candidate refuses none. Rulings applied: María's of 2026-10-15/16
(A1, A3 per region, A6 both channels, A7 FRS-channel ranks, A8).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The live tree names the retired data package by its short name only;
uk-data#516 replaces the full name in the layer tool, its provenance, its
test, the runtime module and the changelog fragment.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ONS's PIPR publishes Scotland's rents for its 18 Broad Rental Market
Areas only, and they now bind there: ons.rent.private_rent_brma, at the
crosswalk's statistical_scope level (the layer's 18 codes and each one's
BRMA) and a partial `brma` area type on the local surface.

- Each row is 12 x the BRMA's calendar-2025 mean x its private renters,
  which uk_private_rent_brma_mean_to_total derives after reconciliation:
  the councils' tenure/private_rent cells in Census 2022's council shares
  of each BRMA's renters (scotland_brma_la_overlap.json). The counts add
  up to Scotland's reconciled renters; a missing council refuses.
- A placed Scottish household joins its BRMA's cell through the BRMA of
  its assigned output area; households outside Scotland belong to no
  BRMA area, which the matrix builder now allows for a partial grain.
- The rows weigh inside the local-authority grain's private-rent family
  (a grain of their own would dilute the energy and bus families below
  their floors), are signature-incomplete beside the authority rent rows
  and route as no_higher_control.
- The 32 council cells stay signed deferred with the reason restated,
  the Northern Ireland reason says PIPR has a country mean only, the
  parity concern resolves to the BRMA rows, and diagnostics gain a
  `brma` level, optional in reports written before it.
- Rent texts say placed renters' rents now come from the BRMA lists at
  the region's FRS level, so PIPR's pattern is not independent
  validation; "A17-uprated" becomes SPREE-drifted and reconciled.

Local references regenerate once: 23,004 active, the 18 BRMA rows added.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The local compile-parity tool explains the 18 Scottish BRMA rows, which the
incumbent has no rows for, as ledger-only, and restates the fixture-only
rationale of the private-rent rows. The receipt is regenerated on the head
that is built.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`tools/uk_private_rent_receipt.py` measures a local build, and a control
build: the rent rows at design, dense and release weights; rents by
region and bedrooms; PIPR by district and bedrooms; each region's BRMA
mix against the census; and renters' weights and coherence. Its summary
keeps only aggregates that pass UKDS disclosure control (ten distinct
source households per cell, complementary suppression, rounded counts,
nothing for a single district or BRMA); its audit stays with the run.

`tools/uk_private_rent_pool.py` writes the clone pool from the graph
store with each row's spine rent and BRMA beside the rewritten ones,
for the design-weight measures and a matched before and after.

`docs/evidence/uk-private-rent-1262/scripts/engine_rent_diagnostics.py`
runs the engine for UC, the UC housing element, Housing Benefit, AHC
poverty, the LHA rate and rent above LHA against DWP, split into spine,
area-BRMA and built variants at the release weights.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From the #1305 review, before any receipt runs:
- carriers come from one column. A cell of spine values counts its
  FRS-channel households only, since an SPI-channel row's spine rent is
  drawn from FRS donors;
- the top 1%'s weight share needs ten rows over ten households;
- an unweighted coherence share needs both sides on ten households, not
  ten clone rows;
- a district or BRMA group is summarised only over ten cells;
- the summary, the audit, the pool table and the engine report are refused
  inside the repository, and .gitignore covers *.npz and audit-*/.

The design-weight regional level is now a named check,
region_level_at_design_weights. The DWP rent-above-LHA comparison is marked
indicative: it was copied by hand from a quotation, and no Chronicle fact
carries the table. Tests cover the summary writers: a thin cell and its
complement, thin district groups, the top 1%, clone rows, FRS-only carriers
and the repository guard.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
María kept the census renter mix for the region scale on 2026-10-17 (the
#1305 review, item 01). The scale holds each region's FRS mean rent on the
census private-renter mix of its BRMAs. Placement follows all households, so
at design weights a region's level moves by the gap between the two mixes
until calibration's tenure rows restore it. The kernel and module docstrings,
the build-graph doc and the changelog now say so, and the receipt reports
that level as a check.

The quantile tool and its resource qualify "a factor common to a nation's
list cancels". It does not cancel for the 16 LSOAs whose BRMA crosses the
border: West Cheshire's 11 Welsh LSOAs read England's list under the Wales
scale, and Monmouthshire's 4 and Brecon and Radnor's 1 English LSOAs read
Wales's older list under English scales. The resource is rebuilt for the
text (--check reproduced the committed one first). The kernel's receipt says
rounded record counts only.

Tests:
- the toy counts give London's categories different BRMA mixes, so a scale
  that ignored the category fails;
- a Monte-Carlo check holds the weighted redrawn mean at the FRS level for
  renters placed on the census mix;
- the toy-graph test asserts that the rent changes;
- the London clone test asserts its six clones and both BRMAs
  unconditionally.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Only the row-metadata rules mapped the 18 Scottish BRMA rows into the
local-authority grain. Under grain_equal the raw labels made "brma" a grain
of its own, with a quarter of the loss for 18 rows: about 20 times an
authority row's weight, and the national share fell from a third to a
quarter (#1305 review). UK_LOCAL_WEIGHT_GRAIN now sits in local_doctrine,
and uk_local_target_loss_weights maps through it. That covers the solve,
the holdout and uk_rule_loss_weights at once, while the alignment check keeps
the raw labels. A test holds each grain's share when the rows are added.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 and others added 12 commits October 18, 2026 02:51
uk_local_brma_enum_domain's structural stop had no test. The classification
test gains it, and a new test runs the unstubbed enum binding on a frame
with an off-domain BRMA. The gate fails, and at both rungs the build stops
with no candidate exported (#1305 review).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From the #1305 review:
- --brma-localisation none no longer binds the BRMA rows on the spine's
  regional draw. The selection leaves them out and its payload counts them;
  an explicit brma level without area BRMAs is refused.
- --target-geographies can name brma, so "country,region,constituency,la"
  dropping the 18 rows is now a choice the selection receipt records.
- The layer and quantile override and sha flags are refused in none mode,
  where nothing reads them.
- On a sampled development rung, a region whose sample holds SPI-channel
  renters but no FRS renter ranks and scales on Great Britain's FRS renters,
  on its own census mix, and the receipt counts the cells and regions. A
  full-scale build still refuses.

Tests cover both measurement modes' release refusals, brma in
--target-geographies, the none-mode flag refusals and the release-candidate
override refusals, the size identity moving with a rent pin, the sampled-rung
fallback, the config rule, and the BRMA codes and metric.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
María's ruling of 2026-10-17 on the #1305 review, item 27: a BRMA row's
model figures leave a build only when at least ten distinct source
households carry them. The support gate covers constituencies and
authorities, not the partial BRMA grain. A row's carriers are the distinct
source_household_ids of the positive-weight households in its problem row,
as the area support counts them.

A withheld row keeps its place and its failure everywhere the build writes
it, without its estimates or errors:
- the calibration diagnostics: model fields null, with a note in metadata.
  It counts as scored but never as passing, and stays out of the family
  rollup, so no total can be differenced against the published rows;
- the targets CSV and the gate report's target_diagnostics;
- the gate report: the fit gates' failure lines quote target and estimate
  and their details list every row's error and the worst offenders. A
  withheld row's line keeps its name only, and it leaves the error lists.

The gates still evaluate on the full figures.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
)

From the #1305 review:
- Both private-rent routes in uk/cross_grain_declarations.json declare
  their level miss. The rows bind PIPR's level while the node holds each
  region at its lower FRS level, so with tenure bound, calibration closes
  the gap only by moving weight toward higher-rent renters (item 03). The
  BRMA route names its own composer, uk_private_rent_brma_mean_to_total
  (item 28).
- The compile-parity tool routes the support floor's signed deferrals
  before the per-metric maps, so E06000053 and E09000001 no longer read as
  Northern Ireland's (item 31). Its UC drift rationales read their
  fixture/ours bands off the rows, as #1307's code does (item 09).
- The parity census names PIPR's July 2026 edition, the record set the
  selector resolves (item 32).
- The Scottish overlap note states the layer's correction counts and where
  they live: 9 filled, 59 corrected by the postcode lookup, 2 Balloch
  overrides (item 22).
- A pinned digest of the atomic supports' (area, FRS region) pairs lets CI
  check the committed layer against supports it does not mount; the mounted
  test re-derives it (item 17). A ladder-free test checks the crosswalk's
  BRMA level against the committed layer (item 30).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On main after #1307 and #1306: 27,267 rows, 23,004 of them compiled. The 18
Scottish BRMA rows are ledger-only by construction, and no signed value
moves. The 40 cells of the support floor's two authorities, E06000053 and
E09000001, carry the floor's own signed rationale; nothing else changes.
The release-blocking preflight gate
uk_ledger_compile_parity_local_incumbent_2025 reads this receipt, so
whichever of #1305 and #1308 lands second regenerates it on the combined
tree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…1262)

From the #1305 re-review (Blocking 2): the real local fit gates did not see
the first round's withholding. uk_local_target_fit writes its own shape
(failure lines quoting each row's error, details.failing_targets keyed
family/area/metric), and uk_local_per_family_fit counted withheld rows in the
family shares and sizes, which set beside weakest_families gave each row's
within-10% and within-25% status back.

- build_full_gate_context counts each BRMA row's carriers on the installed
  weights and flags the rows resting on fewer than ten source households.
  The evidence lists them, and the kernel reads that one set.
- uk_target_fit_gate decides on every row but writes a withheld one by gate
  key only. Its failure line quotes no error and follows every other line in
  name order, so its rank among the errors gives nothing away. It moves from
  failing_targets to withheld_failing_targets, and an exclusion it stands on
  carries no error.
- uk_local_per_family_fit rests its shares, sizes and decision on the
  published rows and counts the withheld ones by family, so no family rollup
  includes them (María's ruling of 2026-10-17).
- The kernel refuses a gate report that names a withheld row beside a
  figure, so a gate that writes per-row errors fails the build instead of
  leaking. The first round's rewrite of the generic gate shape, which
  matched no UK gate, goes.
- The targets table and the embedded rows flag every row while any is
  withheld. A stored zero in a BRMA row no longer counts as a carrier.

The tests run the production bindings on a withheld row, the gate directly,
and the terminal kernel on a two-household fixture whose BRMA row is
withheld. A binding that drops the flag makes the kernel fail closed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…1262)

From the #1305 re-review (Blocking 3): the first round blanked a withheld
row's own fields, but the payload's totals still covered every row. final_loss
less the published rows' contributions gave the withheld row's contribution,
and with its loss weight share and scale, which stay published, its error.
The fraction within 10% and the past-cap census, which listed a pushed-out
withheld row with its misses, did the same.

María's ruling of 2026-10-18: while any row is withheld, the totals cover
the published rows only, labelled.
- final_loss is the published rows' summed contributions, on the solve's own
  loss weights, scales and cap. initial_loss is the same at the input
  weights, from the rows' initial estimates.
- fraction_within_10pct and past_cap_census count the published rows.
- loss_trajectory is empty: it is the solve's path over every row, and no
  part of it can be restated.
- uk_diagnostics.model_figures_withheld counts the withheld rows and names
  the basis. The schema gains the block as an optional field, so payloads
  without withheld rows are unchanged.

A test tries to recover a withheld row from the published totals and finds
nothing left. The restatement over every row reproduces the solve's own
totals, and a withheld row pushed past the cap leaves the census.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ract (#1262)

From the #1305 re-review (Blocking 1): one withheld row broke a dense build
after its export. The manifest projection read the withheld targets table
back, its blank cells became NaN, and max_abs_relative_error,
median_abs_relative_error and fit.local_by_family (np.argmax picked the NaN
row) went NaN. The manifest's strict JSON writer then raised, leaving no
manifest and no build.json. The scorer refused any null final estimate, and
the release contract required finite estimates on every row.

- The projection reads the table's model_figures_withheld flag. Every error
  summary and family fit covers the published rows, and the solve block
  counts the withheld ones and names the totals' basis. While any row is
  withheld, the size receipt's totals over every row (dense_loss,
  compact_loss and max_target_scaled_change) read null in the manifest, with
  a note: compact_loss less the published contributions would give the rows
  back. The stored receipt keeps them.
- The scorer skips a withheld row on both sides and counts it: its error is
  in no drift row and in no loss. A withheld row that carries an estimate is
  refused.
- The release contract lets a row whose metadata carries the note leave its
  estimates null. Such a row may carry no model figure, and the payload's
  withheld block must count exactly these rows.

Tests: a dense CLI run whose terminal table carries a withheld row, the
size receipt as the manifest publishes it, the scorer with a withheld and a
published row on the statistical_scope grain, and the contract's
withheld-row rules.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two more readers of a dense build's figures, found while fixing the
re-review's Blocking items.

- The sign-off drafter (microcosm#1301) refused a targets.csv with a blank
  relative error, so a build with a withheld BRMA row could not be signed.
  It now reads the flag, checks the gate's details against the published
  rows plus the withheld ones, and signs a withheld cell beyond the bound by
  key, as the gate names it. Its reason never carried a figure, and the
  private review file has none to give. The prediction of build 2 stands a
  withheld cell in at twice the bound or zero, as the gate found it, which
  is all the gate's decision reads; the stand-ins are never written. A
  withheld row that gives its error is refused.
- Orrery's result summary, which a staged build publishes to the public
  visualization service by default, listed every target row's initial and
  achieved estimates and errors. It now counts each BRMA row's carriers on
  the result's own weights, blanks a thin row's figures, and restates the
  final loss over the published rows, as the diagnostics do. The size
  summary cannot tell which rows a build withholds, so over a problem that
  binds BRMA rows it publishes no total over every row. Calibrate's
  result_summary splits its decoding into summary_result for the UK
  provider.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…1262)

From the #1305 re-review (Blocking 4 and the receipt items):
- The quotable summary gave the 18 BRMA rows' fit with no carrier rule.
  With one of 18 rows withheld, the group median gave back its |error| and
  the within-10% share times 18, less the published count, its status. Each
  rent row now rests on the distinct source households of positive weight
  with an entry in it, as the build's BRMA rule counts them. A row resting
  on fewer than ten stays out of every group, which counts it; the audit
  keeps every row with its count. The keys come from the pool table for the
  design and dense weightings and from the H5 for the release one. A
  weighting whose households have no keys is not summarised, and
  --control-pool gives the control's.
- A cell with rows but no counted carrier, such as spine values resting on
  SPI rows only, printed its mean. Any non-empty side with fewer than ten
  counted carriers, none included, now reads "<10".
- Two thin cells resting on fewer than ten households together took no
  complement. Complements are withheld until the withheld cells together
  rest on ten.
- A zero-weight row counted as a carrier. It now carries none, as the
  build's rule counts.
- The repository guard knew only this checkout. outside_work_trees
  (receipt_counts) refuses any path with a .git entry in an ancestor:
  another checkout, a worktree, or this one in other case on a
  case-insensitive volume. The receipt, the pool tool and the engine report
  run it before any work.

María kept lineage as the carrier key for SPI copies (2026-10-18).

New tests run main end to end on a synthetic run. A reverted frs_carriers
or a reverted guard now fails them, and every writer refuses a work tree
before reading anything.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From the #1305 re-review (1305-15 and 1305-05):
- A sampled rung's region with one or two FRS renters still computed its
  own scale, and refused when the scale left SCALE_BAND. Below full scale a
  region holding fewer than ten FRS renters (MIN_SAMPLED_REGION_RENTERS) now
  ranks against, and is scaled on, Great Britain's renters on its own census
  mix, counted in the receipt. A full-scale build still ranks and scales
  every region on its own renters and refuses one with none.
- A cell that fell back on Great Britain also counted as ranked against its
  region. Each cell now counts once: cells_ranked_against_their_region
  holds a region's own fallbacks only.
- The Monte-Carlo level test's tolerance is 1%, not 3%. The unmutated
  ratios sit within half a percent, and a mutant using one category's BRMA
  mix for every category, which moved them by about 3% and passed before,
  now fails on every seed.
- strict_cells is pinned: the rent node is strict exactly at full scale,
  and the build sets full scale from both sample fractions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…1262)

From the #1305 re-review's low items and nits:
- The cross-grain rent reasons and the contract's selector notes say what
  the region-scale texts say elsewhere: the rents are scaled to each
  region's FRS level on the census renter mix, and at design weights a
  region moves by the gap between that mix and where its renters land
  until the tenure rows restore it.
- The B1 changelog no longer says the certification part digests do not
  move as if certification were untouched. The declared gate count and the
  exclusions move with the full-manifest pins, so a release certified
  before this change fails validate_release after it.
- The B3 changelog says what a build withholds of a thin BRMA row, and how
  the totals then read.
- The two Scotland and Wales private-rent validation rows name PIPR's July
  2026 record set, the one the bound rent rows resolve, not the September
  one. Both are report-only; nothing reads the selector.
- The loss-share floors' doc comment sits on its constant again, after
  UK_LOCAL_WEIGHT_GRAIN had come between them.
- Three live descriptions of the local battery say seven gates, not six.
- A test runs build_uk_full_local_problem with a Scottish BRMA row: its
  matrix row holds the Scottish clones only (1305-20).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22
juaristi22 force-pushed the uk-private-rent-area-1262 branch from c5ec9f6 to b80b596 Compare October 18, 2026 02:03
@juaristi22

Copy link
Copy Markdown
Collaborator Author

Addressed in b80b596, rebased onto main after #1307 and #1306 merged. Item by item:

Rebase

Blocking

  • One withheld row breaks the dense build. The manifest projection reads the targets table's new model_figures_withheld flag. Every error summary and family fit covers the published rows, and the solve block counts the withheld ones. While any row is withheld, the size receipt's totals over every row (dense_loss, compact_loss, max_target_scaled_change) read null in the manifest, with a note; the stored receipt keeps them. The scorer skips a withheld row on both sides and counts it (rows_withheld). It refuses a withheld row that carries an estimate. The release contract lets a row carrying the note leave its estimates null, if it carries no other model figure and the payload's withheld block counts it. New tests: a dense CLI run whose terminal table has a withheld row, plus scorer and contract tests.

  • The real local fit gate keeps the error. build_full_gate_context counts each BRMA row's carriers on the installed weights and flags the thin ones; the kernel reads that one set. uk_target_fit_gate still decides on every row, but writes a withheld one by gate key only:

    • its failure line quotes no error and comes after every other line, in name order, so its rank gives nothing away;
    • it moves from failing_targets to withheld_failing_targets;
    • an exclusion it stands on carries no error.

    uk_local_per_family_fit rests its shares, sizes and decision on the published rows and counts the withheld rows by family. The kernel now refuses any report that names a withheld row beside a figure, so a future gate that lists per-row errors fails the build instead of leaking. The first round's rewrite of the generic gate shape goes. The tests run the production bindings, and the terminal kernel on a fixture whose BRMA row is withheld. A binding that drops the flag makes the kernel fail closed.

  • The diagnostics' totals give the row back. María ruled on 18 October that while any row is withheld, the totals cover the published rows only, labelled.

    • final_loss is the published rows' summed contributions, on the solve's own weights, scales and cap. initial_loss is the same at the input weights.
    • fraction_within_10pct and past_cap_census count the published rows, so a withheld pushed-out row leaves the listing.
    • loss_trajectory is empty, since it cannot be restated.
    • uk_diagnostics.model_figures_withheld counts the withheld rows and names the basis. It is an optional schema field.

    A test tries to recover a withheld row from the totals and finds nothing left. Over every row, the restatement reproduces the solve's own totals.

  • The rent receipt has no carrier rule. Each rent row now rests on the distinct source households of positive weight with an entry in it, as the build's rule counts them. A row on fewer than ten stays out of every group, which counts it. Source keys come from the pool table for the design and dense weightings, and from the H5 for the release one; --control-pool gives the control's. A weighting without keys isn't summarised.

Found while fixing these. Two more readers took a withheld row's figures:

  • The sign-off drafter (UK local-area release: publish a K=15, 150,000-household dense dataset in at most two builds #1301). It refused a blank error, so it could not sign such a build. It now signs a withheld cell beyond the bound by key, as the gate names it, and its reasons carry no figure. For its prediction of build 2 it stands each withheld cell in at twice the bound or at zero, as the gate found it; the stand-ins are never written.
  • Orrery's result summary. A staged build publishes this. It now counts each BRMA row's carriers on that result's own weights, blanks a thin row's figures, and restates the final loss over the published rows. The size summary can't tell which rows are withheld, so for a problem that binds BRMA rows it publishes no total over every row.

Should fix

  • Zero-carrier cells. A non-empty side with fewer than ten counted carriers reads "<10", including zero.
  • SPI copies as carriers. María ruled on 18 October to keep lineage as the carrier key. No change.
  • Monte-Carlo tolerance. It is now 1%. The mutant that uses one category's mix for every category fails on all three seeds, and the unmutated test passes.
  • 1305-15. Below full scale, a region with fewer than ten FRS renters ranks against, and is scaled on, Great Britain's renters, and the receipt counts it. Full scale stays strict. Each cell is now counted once, so cells_ranked_against_their_region no longer includes Great Britain's fallbacks.

Low and nits

  • The guard. outside_work_trees (receipt_counts) refuses any path with a .git entry in an ancestor. The receipt, the pool tool and the engine report run it before any work.
  • Thin cells. Complements are withheld until the withheld cells together rest on ten households.
  • Zero weights. A zero-weight row carries nothing, as in the build's rule.
  • Tests that passed with the fix reverted. main now runs end to end on a synthetic run, so reverting frs_carriers or the guard fails. Every writer refuses a work tree before reading anything, the kernel's withholding is tested, and strict_cells is pinned at both ends.
  • 1305-11. The B1 changelog says that a release certified before this change fails validate_release after it.
  • 1305-20. Tests now cover the scorer's statistical_scope grain and build_uk_full_local_problem with a BRMA row.
  • Nits.
    • The floors' doc comment is back on its constant.
    • Three live descriptions say seven gates.
    • The two validation rows name the July record set the rent rows resolve.
    • The cross-grain reasons and the selector notes carry the design-weight qualifier.
    • max_target_scaled_change is covered under Blocking.
    • The B3 changelog says what a build withholds.

Not yet run. The head spine and dense builds, and their receipts. Main now has #1307, so they can start.

🤖 Generated with Claude Code

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant