Repository navigation
Place UK private rent and BRMA at the assigned area; bind Scotland's rents at BRMA grain (#1262, #1090) - #1305
juaristi22 wants to merge 22 commits into
Conversation
Independent adversarial review of this PR at 29e5c5fRun with Claude Code on 16–17 October 2026 at 29e5c5f: seven independent review lenses (correctness, method, generated artifacts, tests, claims against the diff, disclosure, a second fresh-eyes pass), then one to three skeptics per finding who tried to refute it; (k/n) is how many agreed. No builds and no feed-gated tests were run. The head has not moved since. Leave it draft. It is not ready to merge. The BRMA layer, the rent quantile table, the Scottish overlap, the B3 composer and the rent node all check out against their sources, and every touched test passes at head 29e5c5f. Three things block it:
Before the receipts run, close the disclosure gaps in the receipt tool and fix the region-scale claim. Otherwise the first numbers will either be misread or be unsafe to quote. Blocking
Should fix
Minor
Cross-PR
Disclosure checkWhat the disclosure lens checked:
Every figure comes from Census/NRS, DESNZ/ONS, VOA, Rent Officers Wales, SG, RSS, PIPR, DWP admin counts or target-registry counts. The "30.9%" in 6421fe4 comes from Rent Service Scotland FOI lettings: admin data, outside UKDS scope. No committed content carries a FRS/LCFS/WAS/SPI/NTS figure, a record id, a per-record value or an extreme. The FRS level is applied only inside the kernel, the region scales and ranks never leave it, and the receipts round record counts. Gaps, all in tooling that has not run yet, so nothing has been published:
Close these before the receipts run. The PR plans to quote rent fit by region and category, and PIPR by district and bedrooms, from exactly this file. For María to decide (1305-27): the 18 new BRMA rows need a carrier floor before any BRMA-level diagnostics leave the build. The figures in this review are synthetic test values, counts of targets, areas or tests, or come from public census, rent-officer list, PIPR, SG, DESNZ and DWP sources. Tests and tools runAll runs were at head 29e5c5f.
Checked and found sound
Refuted during verificationNo finding was refuted outright. These parts were dropped or narrowed:
Not covered
🤖 Generated with Claude Code |
29e5c5f to
c5ec9f6
Compare
From the #1305 review, before any receipt runs: - carriers come from one column. A cell of spine values counts its FRS-channel households only, since an SPI-channel row's spine rent is drawn from FRS donors; - the top 1%'s weight share needs ten rows over ten households; - an unweighted coherence share needs both sides on ten households, not ten clone rows; - a district or BRMA group is summarised only over ten cells; - the summary, the audit, the pool table and the engine report are refused inside the repository, and .gitignore covers *.npz and audit-*/. The design-weight regional level is now a named check, region_level_at_design_weights. The DWP rent-above-LHA comparison is marked indicative: it was copied by hand from a quotation, and no Chronicle fact carries the table. Tests cover the summary writers: a thin cell and its complement, thin district groups, the top 1%, clone rows, FRS-only carriers and the repository guard. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
María kept the census renter mix for the region scale on 2026-10-17 (the #1305 review, item 01). The scale holds each region's FRS mean rent on the census private-renter mix of its BRMAs. Placement follows all households, so at design weights a region's level moves by the gap between the two mixes until calibration's tenure rows restore it. The kernel and module docstrings, the build-graph doc and the changelog now say so, and the receipt reports that level as a check. The quantile tool and its resource qualify "a factor common to a nation's list cancels". It does not cancel for the 16 LSOAs whose BRMA crosses the border: West Cheshire's 11 Welsh LSOAs read England's list under the Wales scale, and Monmouthshire's 4 and Brecon and Radnor's 1 English LSOAs read Wales's older list under English scales. The resource is rebuilt for the text (--check reproduced the committed one first). The kernel's receipt says rounded record counts only. Tests: - the toy counts give London's categories different BRMA mixes, so a scale that ignored the category fails; - a Monte-Carlo check holds the weighted redrawn mean at the FRS level for renters placed on the census mix; - the toy-graph test asserts that the rent changes; - the London clone test asserts its six clones and both BRMAs unconditionally. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Only the row-metadata rules mapped the 18 Scottish BRMA rows into the local-authority grain. Under grain_equal the raw labels made "brma" a grain of its own, with a quarter of the loss for 18 rows: about 20 times an authority row's weight, and the national share fell from a third to a quarter (#1305 review). UK_LOCAL_WEIGHT_GRAIN now sits in local_doctrine, and uk_local_target_loss_weights maps through it. That covers the solve, the holdout and uk_rule_loss_weights at once, while the alignment check keeps the raw labels. A test holds each grain's share when the rows are added. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
uk_local_brma_enum_domain's structural stop had no test. The classification test gains it, and a new test runs the unstubbed enum binding on a frame with an off-domain BRMA. The gate fails, and at both rungs the build stops with no candidate exported (#1305 review). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From the #1305 review: - --brma-localisation none no longer binds the BRMA rows on the spine's regional draw. The selection leaves them out and its payload counts them; an explicit brma level without area BRMAs is refused. - --target-geographies can name brma, so "country,region,constituency,la" dropping the 18 rows is now a choice the selection receipt records. - The layer and quantile override and sha flags are refused in none mode, where nothing reads them. - On a sampled development rung, a region whose sample holds SPI-channel renters but no FRS renter ranks and scales on Great Britain's FRS renters, on its own census mix, and the receipt counts the cells and regions. A full-scale build still refuses. Tests cover both measurement modes' release refusals, brma in --target-geographies, the none-mode flag refusals and the release-candidate override refusals, the size identity moving with a rent pin, the sampled-rung fallback, the config rule, and the BRMA codes and metric. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
María's ruling of 2026-10-17 on the #1305 review, item 27: a BRMA row's model figures leave a build only when at least ten distinct source households carry them. The support gate covers constituencies and authorities, not the partial BRMA grain. A row's carriers are the distinct source_household_ids of the positive-weight households in its problem row, as the area support counts them. A withheld row keeps its place and its failure everywhere the build writes it, without its estimates or errors: - the calibration diagnostics: model fields null, with a note in metadata. It counts as scored but never as passing, and stays out of the family rollup, so no total can be differenced against the published rows; - the targets CSV and the gate report's target_diagnostics; - the gate report: the fit gates' failure lines quote target and estimate and their details list every row's error and the worst offenders. A withheld row's line keeps its name only, and it leaves the error lists. The gates still evaluate on the full figures. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
) From the #1305 review: - Both private-rent routes in uk/cross_grain_declarations.json declare their level miss. The rows bind PIPR's level while the node holds each region at its lower FRS level, so with tenure bound, calibration closes the gap only by moving weight toward higher-rent renters (item 03). The BRMA route names its own composer, uk_private_rent_brma_mean_to_total (item 28). - The compile-parity tool reads the UC drift rationales' fixture/ours bands off the regenerated rows instead of quoting stale ones, and says what the paid-claims basis leaves (item 09). It routes the support floor's signed deferrals before the per-metric maps, so E06000053 and E09000001 no longer read as Northern Ireland's (item 31). - The parity census names PIPR's July 2026 edition, the record set the selector resolves (item 32). - The Scottish overlap note states the layer's correction counts and where they live: 9 filled, 59 corrected by the postcode lookup, 2 Balloch overrides (item 22). - A pinned digest of the atomic supports' (area, FRS region) pairs lets CI check the committed layer against supports it does not mount; the mounted test re-derives it (item 17). A ladder-free test checks the crosswalk's BRMA level against the committed layer (item 30). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On main after #1302: 25,327 compiled cells. The 18 Scottish BRMA rows are ledger-only by construction, and no signed value moves. The UC drift rationales now read their fixture/ours bands off the rows, and the support floor's deferred cells carry their own signed rationale. The release-blocking preflight gate uk_ledger_compile_parity_local_incumbent_2025 reads this receipt, so whichever of #1305, #1306, #1307 and #1308 lands last regenerates it on the combined tree. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Addressed in c5ec9f6, rebased onto main after #1302 merged. Item by item: Blocking
Should fix
Minor
Cross-PR.
🤖 Generated with Claude Code |
The energy module's docstring said consumption is calendar 2024 kWh. Since the abf0921 re-pin the gas rows are DESNZ's gas year 2024, which the sub-national lookups pair with calendar 2024 (SUBNATIONAL_PERIOD_TYPES); electricity stays on calendar years. Docstring only. The other wording the re-review names in this module (the module docstring's account of the Northern Ireland interim, UK_ENERGY_LEVEL_BASIS and the interim text in the household-share receipt) is left as it is: the two stacked changes that retire the interim, #1305 and #1306, rewrite those same lines, and UK_ENERGY_LEVEL_BASIS is stamped into each kWh cell's metadata, so it moves with their regeneration. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Re-review of this PR at c5ec9f6Run with Claude Code on 17 October 2026, after the rebase onto main (post #1302) and the nine new commits. Three independent reviewers re-checked every first-round finding, attacked the new code with synthetic probes, checked the new public text against the UK Data Service rules, and simulated the merges with #1306, #1307 and #1308. No builds and no feed-gated tests were run. VerdictThe rebase is clean:
But 48529f1, which withholds the model figures of BRMA rows resting on fewer than 10 distinct source households, has four high-severity problems. Two independent reviewers found them. Fix them before the head build:
The response's 1305-27 line ("without estimates or errors … and stays out of every rollup") is not yet true for the reasons below. Blocking (before the head build)
Should fix
Low and nits
First-round dispositions
Cross-PR
Disclosure checkThe new commit messages, added lines and body carry no survey-derived figures. The three leak routes in the new code are under Blocking. Tests and tools run
🤖 Generated with Claude Code |
…1262) uk/brma_area_layer.json places each 2021 LSOA of England and Wales and each 2022 output area of Scotland in one Broad Rental Market Area, so a placed household's BRMA can be read off its assigned area (microcosm#1262, B1). tools/build_uk_brma_area_layer.py rebuilds it from Chronicle's raw registrations, pinned by sha256: - England and Wales: LSOA 2021 population-weighted centroids in the VOA and Rent Officers Wales polygons, with policyengine-uk-data#516's overrides; its region x BRMA support table agrees exactly. - Scotland: output-area centroids in the Scottish Government polygons, corrected by Rent Service Scotland's postcode lookup, with the two Balloch overrides of microcosm#1092; its 63 BRMA x council pairs agree exactly. uk_runtime/brma_area_layer.py loads it and reads a BRMA per household. microcosm#1092 folds in on the same layer. The Scotland BRMA x council overlap now sums Census 2022 UV404 by output area from the pinned feed (PolicyEngine/chronicle#326), so the ward split and its unregistered ward export go: all-household cells are unchanged, private renters total 322,903 against the ward split's 322,995, and the decision note and the rent dispersion receipt are rebuilt on it (BRMA rents still cannot become council targets: up to 30.9% from a council's own mean). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
uk/brma_rent_quantiles.json holds 25 weekly-rent knots (0.005 to 0.995) and the implied mean for each Great Britain BRMA and bedroom category B to E, the shape the private-rent redraw reads (microcosm#1262, B2; the level comes from the FRS, one scale per region). tools/build_uk_brma_rent_quantiles.py rebuilds it from Chronicle's raw registrations and the pinned feed: - England: the VOA list of rents, April 2026. - Wales: Rent Officers Wales, ATISN 25142 Table 2 (October 2023 to September 2024). - Scotland: the Scottish Government's 2025 lower quartile, median and upper quartile per BRMA and size, with the shape of Rent Service Scotland's 2016-21 market evidence; each implied mean stays within 15% of the published one. A cell with fewer than 50 lets borrows its nation's shape at its own median (20 to 49 lets) or a log-additive BRMA plus category median. Category A is never used. uk_runtime/private_rent_localisation.py holds the table's loader and the redraw's pure functions. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…1262) Two post-geography nodes on the local line (microcosm#1262, B1 and B2): - uk.full.geography.brma (--brma-localisation area, the default on the atomic assignment) reads each placed Great Britain clone's BRMA off its LSOA or output area from the committed layer, so its LHA rate describes the same place as its local authority and constituency. Northern Ireland and the national line keep the spine's frs_brma draw. A terminal, release-blocking uk_local_brma_enum_domain gate rechecks the column; the full and dense battery digests, the declared gate count and the certification exclusions move with it. - uk.full.private_rent (--private-rent-localisation brma_list, the default with area BRMAs) redraws each GB private renter's rent with a positive rent, in both channels. Each keeps its rank among its region and bedroom category's FRS-channel renters (SPI rows ranked against that reference) and reads that quantile off its assigned BRMA's list. One scale per region holds the region's FRS mean rent on the census private-renter mix, so the lists set where rents sit within a region, not the level. Category A is never used; everything else stays bit for bit, and receipts carry rounded counts only. Both modes ride the size checkpoint and the orrery geography group, and a release candidate refuses none. Rulings applied: María's of 2026-10-15/16 (A1, A3 per region, A6 both channels, A7 FRS-channel ranks, A8). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The live tree names the retired data package by its short name only; uk-data#516 replaces the full name in the layer tool, its provenance, its test, the runtime module and the changelog fragment. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ONS's PIPR publishes Scotland's rents for its 18 Broad Rental Market Areas only, and they now bind there: ons.rent.private_rent_brma, at the crosswalk's statistical_scope level (the layer's 18 codes and each one's BRMA) and a partial `brma` area type on the local surface. - Each row is 12 x the BRMA's calendar-2025 mean x its private renters, which uk_private_rent_brma_mean_to_total derives after reconciliation: the councils' tenure/private_rent cells in Census 2022's council shares of each BRMA's renters (scotland_brma_la_overlap.json). The counts add up to Scotland's reconciled renters; a missing council refuses. - A placed Scottish household joins its BRMA's cell through the BRMA of its assigned output area; households outside Scotland belong to no BRMA area, which the matrix builder now allows for a partial grain. - The rows weigh inside the local-authority grain's private-rent family (a grain of their own would dilute the energy and bus families below their floors), are signature-incomplete beside the authority rent rows and route as no_higher_control. - The 32 council cells stay signed deferred with the reason restated, the Northern Ireland reason says PIPR has a country mean only, the parity concern resolves to the BRMA rows, and diagnostics gain a `brma` level, optional in reports written before it. - Rent texts say placed renters' rents now come from the BRMA lists at the region's FRS level, so PIPR's pattern is not independent validation; "A17-uprated" becomes SPREE-drifted and reconciled. Local references regenerate once: 23,004 active, the 18 BRMA rows added. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The local compile-parity tool explains the 18 Scottish BRMA rows, which the incumbent has no rows for, as ledger-only, and restates the fixture-only rationale of the private-rent rows. The receipt is regenerated on the head that is built. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`tools/uk_private_rent_receipt.py` measures a local build, and a control build: the rent rows at design, dense and release weights; rents by region and bedrooms; PIPR by district and bedrooms; each region's BRMA mix against the census; and renters' weights and coherence. Its summary keeps only aggregates that pass UKDS disclosure control (ten distinct source households per cell, complementary suppression, rounded counts, nothing for a single district or BRMA); its audit stays with the run. `tools/uk_private_rent_pool.py` writes the clone pool from the graph store with each row's spine rent and BRMA beside the rewritten ones, for the design-weight measures and a matched before and after. `docs/evidence/uk-private-rent-1262/scripts/engine_rent_diagnostics.py` runs the engine for UC, the UC housing element, Housing Benefit, AHC poverty, the LHA rate and rent above LHA against DWP, split into spine, area-BRMA and built variants at the release weights. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From the #1305 review, before any receipt runs: - carriers come from one column. A cell of spine values counts its FRS-channel households only, since an SPI-channel row's spine rent is drawn from FRS donors; - the top 1%'s weight share needs ten rows over ten households; - an unweighted coherence share needs both sides on ten households, not ten clone rows; - a district or BRMA group is summarised only over ten cells; - the summary, the audit, the pool table and the engine report are refused inside the repository, and .gitignore covers *.npz and audit-*/. The design-weight regional level is now a named check, region_level_at_design_weights. The DWP rent-above-LHA comparison is marked indicative: it was copied by hand from a quotation, and no Chronicle fact carries the table. Tests cover the summary writers: a thin cell and its complement, thin district groups, the top 1%, clone rows, FRS-only carriers and the repository guard. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
María kept the census renter mix for the region scale on 2026-10-17 (the #1305 review, item 01). The scale holds each region's FRS mean rent on the census private-renter mix of its BRMAs. Placement follows all households, so at design weights a region's level moves by the gap between the two mixes until calibration's tenure rows restore it. The kernel and module docstrings, the build-graph doc and the changelog now say so, and the receipt reports that level as a check. The quantile tool and its resource qualify "a factor common to a nation's list cancels". It does not cancel for the 16 LSOAs whose BRMA crosses the border: West Cheshire's 11 Welsh LSOAs read England's list under the Wales scale, and Monmouthshire's 4 and Brecon and Radnor's 1 English LSOAs read Wales's older list under English scales. The resource is rebuilt for the text (--check reproduced the committed one first). The kernel's receipt says rounded record counts only. Tests: - the toy counts give London's categories different BRMA mixes, so a scale that ignored the category fails; - a Monte-Carlo check holds the weighted redrawn mean at the FRS level for renters placed on the census mix; - the toy-graph test asserts that the rent changes; - the London clone test asserts its six clones and both BRMAs unconditionally. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Only the row-metadata rules mapped the 18 Scottish BRMA rows into the local-authority grain. Under grain_equal the raw labels made "brma" a grain of its own, with a quarter of the loss for 18 rows: about 20 times an authority row's weight, and the national share fell from a third to a quarter (#1305 review). UK_LOCAL_WEIGHT_GRAIN now sits in local_doctrine, and uk_local_target_loss_weights maps through it. That covers the solve, the holdout and uk_rule_loss_weights at once, while the alignment check keeps the raw labels. A test holds each grain's share when the rows are added. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
uk_local_brma_enum_domain's structural stop had no test. The classification test gains it, and a new test runs the unstubbed enum binding on a frame with an off-domain BRMA. The gate fails, and at both rungs the build stops with no candidate exported (#1305 review). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From the #1305 review: - --brma-localisation none no longer binds the BRMA rows on the spine's regional draw. The selection leaves them out and its payload counts them; an explicit brma level without area BRMAs is refused. - --target-geographies can name brma, so "country,region,constituency,la" dropping the 18 rows is now a choice the selection receipt records. - The layer and quantile override and sha flags are refused in none mode, where nothing reads them. - On a sampled development rung, a region whose sample holds SPI-channel renters but no FRS renter ranks and scales on Great Britain's FRS renters, on its own census mix, and the receipt counts the cells and regions. A full-scale build still refuses. Tests cover both measurement modes' release refusals, brma in --target-geographies, the none-mode flag refusals and the release-candidate override refusals, the size identity moving with a rent pin, the sampled-rung fallback, the config rule, and the BRMA codes and metric. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
María's ruling of 2026-10-17 on the #1305 review, item 27: a BRMA row's model figures leave a build only when at least ten distinct source households carry them. The support gate covers constituencies and authorities, not the partial BRMA grain. A row's carriers are the distinct source_household_ids of the positive-weight households in its problem row, as the area support counts them. A withheld row keeps its place and its failure everywhere the build writes it, without its estimates or errors: - the calibration diagnostics: model fields null, with a note in metadata. It counts as scored but never as passing, and stays out of the family rollup, so no total can be differenced against the published rows; - the targets CSV and the gate report's target_diagnostics; - the gate report: the fit gates' failure lines quote target and estimate and their details list every row's error and the worst offenders. A withheld row's line keeps its name only, and it leaves the error lists. The gates still evaluate on the full figures. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
) From the #1305 review: - Both private-rent routes in uk/cross_grain_declarations.json declare their level miss. The rows bind PIPR's level while the node holds each region at its lower FRS level, so with tenure bound, calibration closes the gap only by moving weight toward higher-rent renters (item 03). The BRMA route names its own composer, uk_private_rent_brma_mean_to_total (item 28). - The compile-parity tool routes the support floor's signed deferrals before the per-metric maps, so E06000053 and E09000001 no longer read as Northern Ireland's (item 31). Its UC drift rationales read their fixture/ours bands off the rows, as #1307's code does (item 09). - The parity census names PIPR's July 2026 edition, the record set the selector resolves (item 32). - The Scottish overlap note states the layer's correction counts and where they live: 9 filled, 59 corrected by the postcode lookup, 2 Balloch overrides (item 22). - A pinned digest of the atomic supports' (area, FRS region) pairs lets CI check the committed layer against supports it does not mount; the mounted test re-derives it (item 17). A ladder-free test checks the crosswalk's BRMA level against the committed layer (item 30). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On main after #1307 and #1306: 27,267 rows, 23,004 of them compiled. The 18 Scottish BRMA rows are ledger-only by construction, and no signed value moves. The 40 cells of the support floor's two authorities, E06000053 and E09000001, carry the floor's own signed rationale; nothing else changes. The release-blocking preflight gate uk_ledger_compile_parity_local_incumbent_2025 reads this receipt, so whichever of #1305 and #1308 lands second regenerates it on the combined tree. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…1262) From the #1305 re-review (Blocking 2): the real local fit gates did not see the first round's withholding. uk_local_target_fit writes its own shape (failure lines quoting each row's error, details.failing_targets keyed family/area/metric), and uk_local_per_family_fit counted withheld rows in the family shares and sizes, which set beside weakest_families gave each row's within-10% and within-25% status back. - build_full_gate_context counts each BRMA row's carriers on the installed weights and flags the rows resting on fewer than ten source households. The evidence lists them, and the kernel reads that one set. - uk_target_fit_gate decides on every row but writes a withheld one by gate key only. Its failure line quotes no error and follows every other line in name order, so its rank among the errors gives nothing away. It moves from failing_targets to withheld_failing_targets, and an exclusion it stands on carries no error. - uk_local_per_family_fit rests its shares, sizes and decision on the published rows and counts the withheld ones by family, so no family rollup includes them (María's ruling of 2026-10-17). - The kernel refuses a gate report that names a withheld row beside a figure, so a gate that writes per-row errors fails the build instead of leaking. The first round's rewrite of the generic gate shape, which matched no UK gate, goes. - The targets table and the embedded rows flag every row while any is withheld. A stored zero in a BRMA row no longer counts as a carrier. The tests run the production bindings on a withheld row, the gate directly, and the terminal kernel on a two-household fixture whose BRMA row is withheld. A binding that drops the flag makes the kernel fail closed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…1262) From the #1305 re-review (Blocking 3): the first round blanked a withheld row's own fields, but the payload's totals still covered every row. final_loss less the published rows' contributions gave the withheld row's contribution, and with its loss weight share and scale, which stay published, its error. The fraction within 10% and the past-cap census, which listed a pushed-out withheld row with its misses, did the same. María's ruling of 2026-10-18: while any row is withheld, the totals cover the published rows only, labelled. - final_loss is the published rows' summed contributions, on the solve's own loss weights, scales and cap. initial_loss is the same at the input weights, from the rows' initial estimates. - fraction_within_10pct and past_cap_census count the published rows. - loss_trajectory is empty: it is the solve's path over every row, and no part of it can be restated. - uk_diagnostics.model_figures_withheld counts the withheld rows and names the basis. The schema gains the block as an optional field, so payloads without withheld rows are unchanged. A test tries to recover a withheld row from the published totals and finds nothing left. The restatement over every row reproduces the solve's own totals, and a withheld row pushed past the cap leaves the census. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ract (#1262) From the #1305 re-review (Blocking 1): one withheld row broke a dense build after its export. The manifest projection read the withheld targets table back, its blank cells became NaN, and max_abs_relative_error, median_abs_relative_error and fit.local_by_family (np.argmax picked the NaN row) went NaN. The manifest's strict JSON writer then raised, leaving no manifest and no build.json. The scorer refused any null final estimate, and the release contract required finite estimates on every row. - The projection reads the table's model_figures_withheld flag. Every error summary and family fit covers the published rows, and the solve block counts the withheld ones and names the totals' basis. While any row is withheld, the size receipt's totals over every row (dense_loss, compact_loss and max_target_scaled_change) read null in the manifest, with a note: compact_loss less the published contributions would give the rows back. The stored receipt keeps them. - The scorer skips a withheld row on both sides and counts it: its error is in no drift row and in no loss. A withheld row that carries an estimate is refused. - The release contract lets a row whose metadata carries the note leave its estimates null. Such a row may carry no model figure, and the payload's withheld block must count exactly these rows. Tests: a dense CLI run whose terminal table carries a withheld row, the size receipt as the manifest publishes it, the scorer with a withheld and a published row on the statistical_scope grain, and the contract's withheld-row rules. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two more readers of a dense build's figures, found while fixing the re-review's Blocking items. - The sign-off drafter (microcosm#1301) refused a targets.csv with a blank relative error, so a build with a withheld BRMA row could not be signed. It now reads the flag, checks the gate's details against the published rows plus the withheld ones, and signs a withheld cell beyond the bound by key, as the gate names it. Its reason never carried a figure, and the private review file has none to give. The prediction of build 2 stands a withheld cell in at twice the bound or zero, as the gate found it, which is all the gate's decision reads; the stand-ins are never written. A withheld row that gives its error is refused. - Orrery's result summary, which a staged build publishes to the public visualization service by default, listed every target row's initial and achieved estimates and errors. It now counts each BRMA row's carriers on the result's own weights, blanks a thin row's figures, and restates the final loss over the published rows, as the diagnostics do. The size summary cannot tell which rows a build withholds, so over a problem that binds BRMA rows it publishes no total over every row. Calibrate's result_summary splits its decoding into summary_result for the UK provider. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…1262) From the #1305 re-review (Blocking 4 and the receipt items): - The quotable summary gave the 18 BRMA rows' fit with no carrier rule. With one of 18 rows withheld, the group median gave back its |error| and the within-10% share times 18, less the published count, its status. Each rent row now rests on the distinct source households of positive weight with an entry in it, as the build's BRMA rule counts them. A row resting on fewer than ten stays out of every group, which counts it; the audit keeps every row with its count. The keys come from the pool table for the design and dense weightings and from the H5 for the release one. A weighting whose households have no keys is not summarised, and --control-pool gives the control's. - A cell with rows but no counted carrier, such as spine values resting on SPI rows only, printed its mean. Any non-empty side with fewer than ten counted carriers, none included, now reads "<10". - Two thin cells resting on fewer than ten households together took no complement. Complements are withheld until the withheld cells together rest on ten. - A zero-weight row counted as a carrier. It now carries none, as the build's rule counts. - The repository guard knew only this checkout. outside_work_trees (receipt_counts) refuses any path with a .git entry in an ancestor: another checkout, a worktree, or this one in other case on a case-insensitive volume. The receipt, the pool tool and the engine report run it before any work. María kept lineage as the carrier key for SPI copies (2026-10-18). New tests run main end to end on a synthetic run. A reverted frs_carriers or a reverted guard now fails them, and every writer refuses a work tree before reading anything. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From the #1305 re-review (1305-15 and 1305-05): - A sampled rung's region with one or two FRS renters still computed its own scale, and refused when the scale left SCALE_BAND. Below full scale a region holding fewer than ten FRS renters (MIN_SAMPLED_REGION_RENTERS) now ranks against, and is scaled on, Great Britain's renters on its own census mix, counted in the receipt. A full-scale build still ranks and scales every region on its own renters and refuses one with none. - A cell that fell back on Great Britain also counted as ranked against its region. Each cell now counts once: cells_ranked_against_their_region holds a region's own fallbacks only. - The Monte-Carlo level test's tolerance is 1%, not 3%. The unmutated ratios sit within half a percent, and a mutant using one category's BRMA mix for every category, which moved them by about 3% and passed before, now fails on every seed. - strict_cells is pinned: the rent node is strict exactly at full scale, and the build sets full scale from both sample fractions. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…1262) From the #1305 re-review's low items and nits: - The cross-grain rent reasons and the contract's selector notes say what the region-scale texts say elsewhere: the rents are scaled to each region's FRS level on the census renter mix, and at design weights a region moves by the gap between that mix and where its renters land until the tenure rows restore it. - The B1 changelog no longer says the certification part digests do not move as if certification were untouched. The declared gate count and the exclusions move with the full-manifest pins, so a release certified before this change fails validate_release after it. - The B3 changelog says what a build withholds of a thin BRMA row, and how the totals then read. - The two Scotland and Wales private-rent validation rows name PIPR's July 2026 record set, the one the bound rent rows resolve, not the September one. Both are report-only; nothing reads the selector. - The loss-share floors' doc comment sits on its constant again, after UK_LOCAL_WEIGHT_GRAIN had come between them. - Three live descriptions of the local battery say seven gates, not six. - A test runs build_uk_full_local_problem with a Scottish BRMA row: its matrix row holds the Scottish clones only (1305-20). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
c5ec9f6 to
b80b596
Compare
|
Addressed in b80b596, rebased onto main after #1307 and #1306 merged. Item by item: Rebase
Blocking
Found while fixing these. Two more readers took a withheld row's figures:
Should fix
Low and nits
Not yet run. The head spine and dense builds, and their receipts. Main now has #1307, so they can start. 🤖 Generated with Claude Code |
Puts private rent and the Broad Rental Market Area (BRMA) at each placed household's assigned area on the local line (#1262, item 4 of #1123), and binds Scotland's PIPR rents at the grain ONS publishes them. Draft until the head rebuilds: one spine and one dense build, compared with the 15 October main build (María's ruling of 16 October). #1307, which defers the high-value-homes cells that stopped a dense build on main, has merged, so the head builds can run. The PR also waits for Max's view on the direction, asked on PolicyEngine/policyengine-uk-data#521.
Rebased onto main after #1307 and #1306 merged.
Closes #1262. Closes #1090. Folds in and supersedes #1092: its overlap table, decision note and rent dispersion receipt, rebuilt on the area-to-BRMA layer.
The Northern Ireland energy interim is not retired here. María ruled on 17 October that #1306 retires it with its share formula, and #1306 has merged; this PR's subtraction commit was dropped.
Summary
B1. BRMA from the assigned area
uk/brma_area_layer.jsonplaces every 2021 LSOA of England and Wales and every 2022 output area of Scotland in one BRMA.tools/build_uk_brma_area_layer.pyrebuilds it from Chronicle's raw registrations, pinned by sha256:uk.full.geography.brma(--brma-localisation area, the default on the atomic assignment) reads each placed Great Britain clone's BRMA off its LSOA or output area, so its LHA rate describes the same place as its local authority and constituency.frs_brmadraw.uk_local_brma_enum_domain, rechecks the column; an off-domain value stops even a development build. The full and dense battery digests move.B2. Private rents from the lists of rents
uk/brma_rent_quantiles.jsonholds 25 knots per GB BRMA and bedroom category B to E. Its sources:uk.full.private_rent(--private-rent-localisation brma_list) redraws each GB private renter's positive rent, in both channels.B3. Scotland's rents at BRMA grain (#1090)
The rows.
ons.rent.private_rent_brmabinds PIPR's 18 Scottish BRMA rows. They sit at the crosswalk's newstatistical_scopelevel and on a partialbrmaarea type: households outside Scotland belong to no BRMA area.The values. Each row is 12 × the BRMA's calendar-2025 mean × its private renters.
tenure/private_rentcells, through Census 2022's council shares of each BRMA's renters (UV404 by output area).The 32 council cells stay signed deferred; councils have no published rent. The parity concern
cross_grain_private_rent_scotland_brmaresolves to the BRMA rows.Selection.
--target-geographiescan namebrma. Without area BRMAs the rows would bind on the spine's regional draw, so they are left out and counted; namingbrmawithout area BRMAs is refused.Weighting. The rows weigh inside the local-authority grain under every rule,
grain_equalincluded.Diagnostics. A BRMA row's model figures leave a build only when at least 10 distinct source households of positive weight carry them (María's ruling of 17 October). A thinner row keeps its place and its failure. Its estimates and errors are blank in:
It stays out of every rollup, the family fit gate included. While any row is withheld, the diagnostics' totals cover the published rows, labelled in
uk_diagnostics.model_figures_withheld(María's ruling of 18 October). The release contract accepts such a row only if all its model fields are null and the block counts it.Active local references move from 22,986 to 23,004.
Texts and receipts
uk_ledger_compile_parity_local_incumbent_2025reads it. UK local surface before the Budget build: high-value homes OBR leg, per-vehicle fuel flag, Northern Ireland energy share and housekeeping (#1278) #1306 and UK local release: publish a K=15, 150,000-household dense dataset in at most two builds (#1301) #1307 have merged, so whichever of Place UK private rent and BRMA at the assigned area; bind Scotland's rents at BRMA grain (#1262, #1090) #1305 and Port the uk-data wave to policyengine-uk 2.134.4: stamp duty, Move to UC, national home values (sixth #1095 PR) #1308 lands second regenerates the receipt again on the combined tree, with the full-battery and dense digests. The support floor's two authorities carry the floor's own signed rationale.For review
uk_local_per_family_fitdecides on the published rows, so a family share never covers a row whose figures are withheld;uk_local_target_fitstill decides on every row. For a problem that binds BRMA rows, Orrery's size summary publishes no total over every row, since it cannot tell which rows a build withholds.grain_equal) and push the energy and bus families below their loss-share floors.brmageography level without a schema bump, as additive levels have before. Reports written before it may leave it out, and no out-of-repo reader validates the level set.validate_releaseafter it. Sequence the merge against the staged national bundle and the frozen build.uk_local_brma_area_layergate was dropped. The BRMA node fails closed on any area it cannot place, or places in another FRS region.Not yet run
obr.housing_benefit), and AHC poverty on the national and local-authority rows;Tests
--checkmodes reproduce their committed resources: the area-to-BRMA layer, the Scottish overlap and the rent quantiles, run locally on 17 October. The quantiles were rebuilt for a text change after their check, and all three resources are byte-identical at this head.grain_equalshares with the BRMA rows;nonemode;mainend to end, its per-row carrier rule and every writer's work-tree guard;strict_cellsat both ends, and the Monte-Carlo level check at 1%;build_uk_full_local_problemwith a Scottish BRMA row.🤖 Generated with Claude Code