UK wave 2: national fiscal/benefit source packages (#133)

[juaristi22]

Part of #133 (wave 2, batch 1 of ~5: the OBR EFO anchor; HMRC, DWP, ONS-misc and the uk_national profile follow in separate PRs per the one-publication-per-package cadence).

Lane: ledger-source-ingestor (.github/ledger-agents.yml) — judges ledger-source-fidelity + ledger-boundary.

What

Two packages porting policyengine-uk-data's OBR EFO calibration surface (targets/sources/obr.py, ~28 realized targets) from publisher truth — 196 source_projection facts at fiscal-year periods 2024–2030:

package	artifact	lines	facts
obr-efo-receipts-march-2026	efo_receipts.xlsx	income tax (3.4 accrued), Class 1 employee/employer + self-employed NICs (3.4), cash-basis NI total/VAT/fuel duties/CGT/SDLT (3.8)	63
obr-efo-expenditure-march-2026	efo_expenditure.xlsx	council tax England/Scotland/Wales + NI domestic rates + total (4.1), 13 welfare lines incl. UC in/outside cap disambiguated by row (4.9), BBC licence fee (4.19)	133

Values are GBP billion in the workbooks (value_scale: 1e9); all rows are table_record_kind: total (each line is a total over its program population). Fiscal-year periods store the opening year (FY2024-25 → 2024), matching the hmrc/vat_firm_targets_2024_25 precedent. Row headers are exact cell text including OBR's footnote markers (Incapacity benefits2, Total net council tax receipts1) — the drift that truncated startswith-matching hides.

Provenance notes for review

• Direct publisher downloads: obr.uk 403s CI/agent IPs (documented in uk-data, and the reason CBO's analogous package fell back to a curated CSV) — but residential connections pass, so both artifacts are the actual publisher workbooks, double-fetched (manual + fetch-artifact) with byte-identical sha256s: receipts f65b6cb7…766a28, expenditure 700bfd70…d9a1.
• Structure vs the CBO precedent: CBO templates {year} with column_by_year, which emits only the bundle-year's facts (the UK enablement tooling: source-inventory de-hardcode, UK consumer-artifact suite, legacy ETL cleanup #135 year-handling wart). These packages materialize explicit per-year record sets instead, so the full 2024–2030 forecast surface lands in every build.
• Deliberately not ported: uk-data's obr/salary_sacrifice_* series are uk-data-computed uprating (3%/2.4% growth applied to base values from the 2025 SPP Review PDF) — the facts-only ADR excludes computed series; the PDF base values become their own small package later in wave 2. obr/private_school_students is ISC census data (wave-1 scope, see UK source packages, wave 1: the production 149-target national surface (ONS SNPP, SLC, OBR) #132).
• The March-2026 EFO current-receipts table moved to sheet 3.8 (3.9 is now APD) — packages pin sheets by name with exact header guards (2024-25 FY labels, guardable as non-digit strings).

Gates

• validate-package clean for both; build-suite + agent_acceptance green end to end (under the disclosed, reverted local storage.r2 sandbox, as in UK wave 1: ONS mid-2023 population estimate source packages, UK countries + England regions (#132) #137); seven facts spot-verified byte-exact against the workbooks (×1e9 scaling included).
• SOURCE_PACKAGE_ALIASES registered; bundle-test constants updated to the post-publication truth measured by a full local bundle build.
• Draft because the consumer export awaits uv run ledger publish-raw --root db/data by a credentialed maintainer (two artifacts), same as UK wave 1: ONS mid-2023 population estimate source packages, UK countries + England regions (#132) #137. Committed manifests are upload-truthful.
• Known conflict-by-design with UK wave 1: ONS mid-2023 population estimate source packages, UK countries + England regions (#132) #137: both PRs touch SOURCE_PACKAGE_ALIASES and the bundle-test constants; whichever merges second re-measures (two-minute mechanical fix).

🤖 Generated with Claude Code

[juaristi22]

Batch 2 pushed — and this PR now carries wave 2 as consolidated batches rather than the separate-PRs plan in the opening description: parallel drafts off main would all conflict pairwise on SOURCE_PACKAGE_ALIASES + the bundle-test constants, and one branch means one publish-raw pass. Retitled accordingly.

New in batch 2

package	facts	notes
ons-savings-interest-income	8	UKEA HAXV annual observations 2018–2025 (£m, ×1e6), 30-06-2026 release. policyengine-uk-data's flat-fill of the last actual through 2029 is a computed extension — excluded per the facts-only ADR; forward-fill is a populace-side declaration.
ons-families-households-2025	80	Families & households 2025 edition, Table 7: 10 household types × 2018–2025 (thousands, ×1e3). LFS-derived → provenance_class: survey_aggregate, survey_instrument: Labour Force Survey. Row guards are exact cell text incl. leading indentation and [note N] markers.

Spot checks byte-exact (HAXV 2023 = £86.04bn; couple-no-children 2023 = 7,956k). Both suites + agent_acceptance green under the disclosed, reverted R2 sandbox.

Fidelity notes surfaced by this batch

1. HAXV and Table 7 values are newer vintages than the June fixture consumed (30-06-2026 UKEA release; 2025 F&H edition carries LFS-reweighted revisions of earlier years). Deltas land in the Consume Ledger UK facts: uk_target_references, mapping module, and a compile-parity gate populace#622 enumeration, as with the MYE swap.
2. ons_tenure is not ported as national facts, deliberately: the SPREE workbook (sheet 1a) is LA-grain only — policyengine-uk-data sums ~300 LA rows in code to make its England totals, and Ledger only stores publisher-emitted aggregation levels. The workbook becomes the wave-3 LA-level package (ons-subnational-dwellings-by-tenure-2024); England totals derive populace-side.
3. policyengine-uk-data's salary-sacrifice download is silently broken: its pinned asset (media/687a294e…/Tables_6_1_and_6_2.csv) now returns {"status":"gone"} — the module's exception handler drops those targets without failing the build. The current asset is media/6a673e0e5e87122783093901/Tables_6_1_and_6_2.csv (Private pension statistics, updated 2026-07-30); packaged in batch 3, base-year values only (the 3%/yr uprating is computed, excluded).

Batch 3 next: HMRC (salary sacrifice, CGT) + DWP first-print set. SPI follows once the ODS parser lands (ledger/sources, in-lane).

🤖 Generated with Claude Code

[juaristi22]

Batch 3: hmrc-salary-sacrifice-relief-2024 — 6 facts from the tidy Tables 6.1/6.2 CSV, tax year 2024-25 (tax_year period 2024, opening-year convention): income tax relief total + by rate band (£8.8bn / £1.6bn / £5.5bn / £1.8bn) and Class 1 employee/employer NICs relief (£1.0bn / £3.4bn). Every row carries six guard cells (tax year, IT/NICs, contribution type, NICs class, sector, scheme). [z]-suppressed by-rate NICs splits are not ported. All gates green; values byte-verified.

Fidelity notes:

• This is a newer vintage than policyengine-uk-data ever consumed: its pinned asset (media/687a294e…) returns {"status":"gone"}, so its runtime download silently drops these targets today; the module's 2023-24-base ×1.03 uprating is a computed series the facts-only ADR excludes anyway. Packaged from the current asset (Private pension statistics, updated 2026-07-30).
• Correction to earlier comments on UK source packages, wave 1: the production 149-target national surface (ONS SNPP, SLC, OBR) #132/UK source packages, wave 2: national fiscal/benefit families + uk_national target profile #133: I referenced a resolve_pending.py "first-print lane" for hand-transcribed values — no such mechanism exists in this repo (verified by grep). Hand-transcribed uk-data families (DWP set, CGT constants, housing, PSE, NTS, ScotGov) will be packaged from their actual publication files instead.

Scoping from this batch:

• HMRC CGT: uk-data's constants (£65.9bn gains, 378k taxpayers, 2023-24) trace to CGT statistics Table 1 — ODS format, like the SPI collated tables. The AEA-gating and PE-parameter uprating in hmrc_cgt.py are populace-side measurement semantics, not facts.
• Next infrastructure item is therefore the ODS parser (ods_used_range in ledger/sources, in-lane per the harness doc's package-expressibility rule; populace's hmrc_income.py has a stdlib ElementTree reference implementation). It unblocks SPI, CGT, and several DWP tables in one move.

🤖 Generated with Claude Code

[juaristi22]

Batch 4: hmrc-cgt-statistics-2025 — and a correction that shrinks the roadmap.

Correction: the ODS parser already existed

Batch 3's comment (and the #133 scoping) claimed the parser registry had no ODS lane. Wrong: ods_used_range has been fully wired all along — dispatch in source_package.py, repeat-aware reader in ledger/sources/cells.py. It had zero consumers, which my parser-usage survey mistook for absence. Consequence: SPI, CGT and the ODS-format DWP tables were never parser-blocked, and no ledger/sources code change is needed. This package is the lane's first consumer and proves it end to end through build-suite.

The package

Full published history from CGT statistics Table 1 (2025 release, ODS): 37 disposal years (1987-88 → 2023-24) × 9 measures = 333 facts — taxpayer counts (thousands, ×1e3), chargeable gains and tax liabilities (£m, ×1e6), each for individuals, trusts, and the publisher's own Total columns. tax_year periods store the opening year. Percentage columns are derivable and not ported; 2023-24 is provisional per the publication notes.

Fidelity: policyengine-uk-data's hardcoded constants (378k taxpayers, £65.9bn gains, "2023-24 outturn") are exactly the publisher's Total columns — a verbatim port, with the individuals/trusts split now available alongside. The AEA gating and PE-parameter uprating in hmrc_cgt.py are populace-side measurement semantics (the PolicyEngine/populace#622 metric-provider layer), not source facts. This is also the package populace's native UK_CGT_TARGET_SPECS single-homes onto.

Spot checks byte-exact (ty2023 totals 378,000 / £65,937m / individuals £62,921m; ty1987 individuals 135,000). All gates green under the disclosed, reverted R2 sandbox; alias registered; bundle constants re-measured.

PR totals: 6 packages, 623 facts. Next: DWP publication-file packages + SPI collated ODS (both now unblocked), then VOA/NTS/ScotGov/land, uk_national profile last.

🤖 Generated with Claude Code

[juaristi22]

Batch 5: SPI + benefit cap — both on the ods_used_range lane, PR totals now 8 packages, 820 facts.

package	facts	notes
hmrc-spi-income-bands-2023	182	SPI collated tables 2023-24, Tables 3.6/3.7: 13 total-income bands × 7 income types (counts ×1e3, amounts ×1e6), tax_year 2023. survey_aggregate with the SPI named as instrument. Numeric ODS band cells make numeric row-header guards workable (unlike text-digit xlsx headers). The ×1.9 property-income scale and uprated projections in hmrc_spi.py are populace-side; means derivable, not ported. Same artifact populace's national HMRC stage parses — single-homing candidate.
dwp-benefit-cap-november-2025	15	Table 4 point-in-time: UC households capped at Nov-2025, GB Total 110,637 + all 13 amount-capped bands. First dwp source, first UK month-period facts (2025-11). uk-data's headline count is exactly this GB Total (UC-only, publisher-emitted); its £320.9m "annualized reduction" is a computed band-midpoint construction, excluded — populace can declare its own annualization over the ported distribution. Cumulative/flows tables → backfill lane.

Remaining DWP scope: two-child limit (Apr-2025), PIP (Jan-2026) and DWP benefit statistics (Feb-2026) publication tables next batch; the Stat-Xplore families (UC by children ×5, Scotland UC child-under-1) stay parked until someone holds a free Stat-Xplore API key. After that: VOA/NTS/ScotGov/land, then the uk_national profile PR.

🤖 Generated with Claude Code