Restore 51 orphaned changelog fragments into their true release sections

[MaxGhenis]

Fixes #9149.

What this does

Commit 1 — historical repair. Restores the 52 changelog entries from the 51 orphaned changelog.d/ fragments (47 in type subdirectories + 4 top-level missing the .<type>. suffix; one file carried two entries) into the CHANGELOG.md sections of the releases that actually first shipped them, then deletes the orphans. Also fixes 32 pre-existing - - double-bullet artifacts (fragments that carried their own leading dash, doubled by towncrier). Formatting-only for existing entries — no released entry's text was altered.

Commit 2 — enforcement. Adds a tree-state step to the Check changelog fragment job: any tracked file under changelog.d/ that isn't .gitkeep or a top-level <name>.<type>.md now fails CI. The existing check only inspects PR-changed files, which is how the orphans accumulated between the towncrier migration (#7488) and the path-validation check (2026-05-27). Landing enforcement in the same PR as the cleanup makes the invariant atomic.

Why historical repair (vs. dumping into the next release, or writing off)

All 50 orphan-adding commits shipped only the orphan — none had a valid fragment alongside — so these entries are the sole changelog record of ~51 changes released in May 2026 (verified: sampled orphan content appears nowhere in any released CHANGELOG.md). Renaming them into live fragments would misdate them ~2 months and force a spurious minor bump from the 13 added entries; deleting them would preserve a permanent gap in the public record. Adjudicated with a second-model review (sol) concurring on repair + same-PR enforcement.

Method and verification

• Mapping: each orphan's adding commit → first Update PolicyEngine US bump commit on main's first-parent chain → the version stamped in that commit's pyproject.toml. Entry type = the original author's declaration (the subdirectory name; the 4 suffixless classified from content, noted below).
• Insertion preserves the canonical heading order (Breaking changes/Added/Changed/Fixed/Removed), creates missing headings in position, appends after the last bullet of an existing heading, and removes No significant changes. from any section gaining content.
• Receipts: script run is deterministic and idempotent (first run inserted=52; rerun inserted=0 already-present=52); post-repair structure checks: 0 sections with heading-order violations or duplicate headings, 0 sections mixing No significant changes. with content, 0 remaining - - artifacts; towncrier build --draft renders normally against the repaired file (prepend pipeline unaffected).
• Suffixless classifications: alphabetize-tanf-gross-income-limits → Changed, chip-age-unit-package-description → Fixed, ct-eitc-bonus-eligibility → Fixed, pure-leaf-inputs → Changed.

Full fragment → release mapping (51 files, 52 entries)

Fragment	Type	Added by	PR/commit subject	Restored under	Result
changelog.d/added/7548.md	added	a4be56977c	Add NM, SC, and WV 529 plan deductions	1.705.10	inserted
changelog.d/added/7946.md	added	20927f088e	Add a tax-only Roth conversion input (#7946)	1.700.0	inserted
changelog.d/added/8188.md	added	07acc70c80	Add ACA cost-sharing reduction actuarial values	1.722.0	inserted
changelog.d/added/8194.md	added	3830c85a8c	Add DC senior disabled property tax relief	1.697.0	inserted
changelog.d/added/8271.md	added	6ef6346b3c	Add Medicaid 1115 adult work requirement scope (#8505)	1.710.2	inserted
changelog.d/added/8294.md	added	29affcd12c	Add TAXSIM total FICA output (#8383)	1.705.5	inserted
changelog.d/added/8349.md	added	07be47bda1	Update NY Essential Plan expansion after 2026 waiver termination	1.722.0	inserted
changelog.d/added/8355.md	added	96c878259b	Backfill HHS SMI parameter with FY2018, FY2020, and FY2021 historical values	1.700.1	inserted
changelog.d/added/8397.md	added	f397006735	Add Minnesota Homestead Credit Refund (#8417)	1.705.22	inserted
changelog.d/added/8441.md	added	a339a03a5f	Add Mississippi working disabled Medicaid buy-in	1.706.16	inserted
changelog.d/added/8442.md	added	d3c35b978d	Add Mississippi Healthier Medicaid waiver eligibility	1.706.16	inserted
changelog.d/added/issue-8267-medicaid-ce-pass-through.md	added	d5b8379b2f	Add Medicaid CE SNAP TANF pass-through	1.706.15	inserted
changelog.d/added/ssa-disability-screen.md	added	fa2594a08f	Add SSI disability criteria input (#8398)	1.706.3	inserted
changelog.d/fixed/1030.md	fixed	6c5eaa554b	Cite Massachusetts Senior Circuit Breaker income guidance (#8436)	1.706.9	inserted
changelog.d/fixed/1272.md	fixed	7857196824	Cite Pennsylvania income tax rate code (#8435)	1.706.12	inserted
changelog.d/fixed/6035.md	fixed	17f66dd22b	Update Alaska PFD amounts (#8421)	1.706.4	inserted
changelog.d/fixed/6975.md	fixed	e2a06f4d25	Fix OK STC capital gains gross income	1.705.7	inserted
changelog.d/fixed/7321.md	fixed	f6b062c420	Add vectorized sum code-health test (#8426)	1.706.6	inserted
changelog.d/fixed/7391.md	fixed	d6ddcd7991	Harden contrib reform tests against CPI updates (#8425)	1.706.5	inserted
changelog.d/fixed/7492.md	fixed	cb3625f94f	Derive monthly hours worked from last week (#8507)	1.710.1	inserted
changelog.d/fixed/7920.md	fixed	6a1b962176	Update income sources docs example (#8434)	1.706.8	inserted
changelog.d/fixed/7968.md	fixed	ca6f5a6b66	Monthlyize SSI payments and resource stocks (#8375)	1.703.2	inserted
changelog.d/fixed/7985.md	fixed	53089b51bc	Add NC FY2025 SNAP phone allowance (#7985)	1.700.0	inserted
changelog.d/fixed/8097.md	fixed	510dfab734	Fix NY Child Health Plus full-premium tier	1.722.0	inserted
changelog.d/fixed/8187.md	fixed	dc37dbb601	Deprecate legacy Marketplace coverage input (#8432)	1.706.7	inserted
changelog.d/fixed/8239.md	fixed	1e187b5567	Fix Oklahoma child credit federal CTC match	1.705.9	inserted
changelog.d/fixed/8241.md	fixed	886022254b	Fix Michigan homestead property tax credit income limit (#8424)	1.706.5	inserted
changelog.d/fixed/8336.md	fixed	f2b90af06a	Fix Social Security benefit aging uprater (#8381)	1.705.8	inserted
changelog.d/fixed/8343.md	fixed	eb27050230	Fix NJ and MT state tax edge cases	1.705.6	inserted
changelog.d/fixed/8344.md	fixed	eb27050230	Fix NJ and MT state tax edge cases	1.705.6	inserted
changelog.d/fixed/8376.md	fixed	c55e08465c	Fix California HSA wage addback (#8378)	1.705.2	inserted
changelog.d/fixed/8379.md	fixed	203913f016	Fix state payroll tax wage bases	1.705.4	inserted
changelog.d/fixed/8388.md	fixed	9013c73de8	Enforce IRA contribution limits (#8416)	1.705.21	inserted
changelog.d/fixed/8389.md	fixed	f261de7bfa	Cap self-employed pension contributions (#8419)	1.706.1	inserted
changelog.d/fixed/8395.md	fixed	00e0f0e62f	Fix NM modified gross income retirement distributions	1.705.11	inserted
changelog.d/fixed/8401.md	fixed	571740b137	Add Minnesota Paid Leave employee payroll tax (#8408)	1.705.13	inserted
changelog.d/fixed/8405.md	fixed	26310eb13e	Split Pennsylvania refundable credits (#8409)	1.705.14	inserted
changelog.d/fixed/8406.md	fixed	742d9e4f04	Add Pennsylvania employee UC payroll tax (#8407)	1.705.12	inserted
changelog.d/fixed/8443.md	fixed	887bed9cc9	Model SSI Medicaid eligibility in 209b states	1.706.16	inserted
changelog.d/fixed/8496.md	fixed	d613be6605	Add MSP fiscal accounting	1.706.16	inserted
changelog.d/fixed/dc-ptc-state-gate.md	fixed	b0f124abfb	Limit DC property tax credit to DC (#8418)	1.705.23	inserted
changelog.d/fixed/fsla-overtime-input.md	fixed	c5a4dfbf2f	Make FLSA overtime premium a data input (#8429)	1.706.14	inserted
changelog.d/fixed/medicaid-ce-exclusions.md	fixed	e71157a6a0	Add Medicaid CE exclusion inputs (#8377)	1.710.6	inserted
changelog.d/fixed/snap-work-requirement-issues.md	fixed	eb6ff02979	Fix SNAP work requirement issues (#8372)	1.710.5	inserted
changelog.d/fixed/ssi-disability-input-only.md	fixed	1c0bd159a5	Make SSI disability criteria input-only (#8428)	1.710.3	inserted,inserted
changelog.d/fixed/ssi-disability-sga.md	fixed	99f65ffd4d	Keep SSI SGA screen separate from disability criteria (#8365)	1.701.1	inserted
changelog.d/fixed/state-payroll-wage-base-organization.md	fixed	6671586661	Organize state payroll wage bases by program (#8382)	1.705.17	inserted
changelog.d/alphabetize-tanf-gross-income-limits.md	changed	2b01631b62	Alphabetize TANF gross income limits (#8439)	1.706.10	inserted
changelog.d/chip-age-unit-package-description.md	fixed	f66aba808c	Fix CHIP age unit and package description (#8437)	1.706.13	inserted
changelog.d/ct-eitc-bonus-eligibility.md	fixed	4ff151180e	Fix Connecticut EITC bonus eligibility (#8438)	1.706.11	inserted
changelog.d/pure-leaf-inputs.md	changed	e58b53d3b3	Make input variables pure leaves (#8162)	1.700.0	inserted

Repair script (one-time, run from repo root)

#!/usr/bin/env python3
"""One-time historical repair: insert orphaned changelog.d fragments into the
CHANGELOG.md sections of the releases that actually first shipped them.

Deterministic and idempotent: reruns insert nothing. Fails loudly on any
unmapped orphan, missing release section, or count mismatch.
Outputs a markdown mapping table (for the PR body) to stdout.
"""

import glob
import re
import subprocess
import sys
from pathlib import Path

REPO = Path.cwd()
HEADING_ORDER = ["Breaking changes", "Added", "Changed", "Fixed", "Removed"]
TYPE_TO_HEADING = {
    "breaking": "Breaking changes",
    "added": "Added",
    "changed": "Changed",
    "fixed": "Fixed",
    "removed": "Removed",
}
# The four suffixless top-level orphans, classified from their content.
SUFFIXLESS = {
    "changelog.d/alphabetize-tanf-gross-income-limits.md": "changed",
    "changelog.d/chip-age-unit-package-description.md": "fixed",
    "changelog.d/ct-eitc-bonus-eligibility.md": "fixed",
    "changelog.d/pure-leaf-inputs.md": "changed",
}


def run(*args: str) -> str:
    return subprocess.run(args, check=True, capture_output=True, text=True).stdout


def orphan_files() -> list[tuple[str, str]]:
    files: list[tuple[str, str]] = []
    for type_dir in ("added", "fixed"):
        for path in sorted(glob.glob(f"changelog.d/{type_dir}/*.md")):
            files.append((path, type_dir))
    for path, type_ in SUFFIXLESS.items():
        if Path(path).exists():
            files.append((path, type_))
    return files


def entry_texts(path: str) -> list[str]:
    """A fragment is usually one entry, but a file whose lines each start with
    '- ' carries multiple entries; keep them as separate bullets."""
    raw_lines = [l.strip() for l in Path(path).read_text().strip().splitlines()]
    raw_lines = [l for l in raw_lines if l]
    if len(raw_lines) > 1 and all(l.startswith("- ") for l in raw_lines):
        return [re.sub(r"^-\s+", "", l).strip() for l in raw_lines]
    text = re.sub(r"^-\s+", "", " ".join(raw_lines)).strip()
    return [text]


def adding_commit(path: str) -> str:
    out = run("git", "log", "--diff-filter=A", "--format=%H", "--", path)
    shas = out.split()
    if not shas:
        sys.exit(f"FATAL: no adding commit for {path}")
    return shas[-1]  # oldest add


def first_bump_after(sha: str, cache: dict[str, str]) -> str:
    if sha in cache:
        return cache[sha]
    out = run(
        "git", "rev-list", "--first-parent", "--reverse", "--format=%H %s",
        f"^{sha}", "upstream/main",
    )
    for line in out.splitlines():
        if line.startswith("commit "):
            continue
        commit, _, subject = line.partition(" ")
        if subject.strip() == "Update PolicyEngine US":
            cache[sha] = commit
            return commit
    sys.exit(f"FATAL: no version-bump commit found after {sha}")


def bump_version(bump_sha: str, cache: dict[str, str]) -> str:
    if bump_sha in cache:
        return cache[bump_sha]
    toml = run("git", "show", f"{bump_sha}:pyproject.toml")
    m = re.search(r'^version\s*=\s*"([^"]+)"', toml, re.M)
    if not m:
        sys.exit(f"FATAL: no version in pyproject.toml at {bump_sha}")
    cache[bump_sha] = m.group(1)
    return m.group(1)


def insert(lines: list[str], version: str, heading: str, bullet: str) -> str:
    """Insert bullet under `### {heading}` in section `## [{version}]`.
    Returns 'inserted' or 'already-present'. Mutates lines."""
    sec_start = None
    for i, line in enumerate(lines):
        if line.startswith(f"## [{version}] - "):
            sec_start = i
            break
    if sec_start is None:
        sys.exit(f"FATAL: no CHANGELOG section for version {version}")
    sec_end = len(lines)
    for i in range(sec_start + 1, len(lines)):
        if lines[i].startswith("## ["):
            sec_end = i
            break
    if bullet in lines[sec_start:sec_end]:
        return "already-present"
    # A section gaining real content must drop its "No significant changes." line.
    for i in range(sec_end - 1, sec_start, -1):
        if lines[i] == "No significant changes.":
            del lines[i]
            if i < len(lines) and lines[i] == "" and lines[i - 1] == "":
                del lines[i]
    # Recompute section end after possible deletion.
    sec_end = len(lines)
    for i in range(sec_start + 1, len(lines)):
        if lines[i].startswith("## ["):
            sec_end = i
            break
    # Find the heading inside the section.
    head_idx = None
    for i in range(sec_start + 1, sec_end):
        if lines[i] == f"### {heading}":
            head_idx = i
            break
    if head_idx is not None:
        # Append after the last bullet of this heading's block.
        j = head_idx + 1
        last_bullet = None
        while j < sec_end and not lines[j].startswith("### "):
            if lines[j].startswith("- "):
                last_bullet = j
            j += 1
        insert_at = (last_bullet + 1) if last_bullet is not None else head_idx + 2
        lines.insert(insert_at, bullet)
        return "inserted"
    # Create the heading at its canonical position among existing headings.
    rank = HEADING_ORDER.index(heading)
    insert_at = None
    for i in range(sec_start + 1, sec_end):
        if lines[i].startswith("### "):
            existing = lines[i][4:]
            if existing in HEADING_ORDER and HEADING_ORDER.index(existing) > rank:
                insert_at = i
                break
    if insert_at is None:
        insert_at = sec_end - 1  # before the trailing blanks preceding next section
        while insert_at > sec_start and lines[insert_at - 1] == "":
            insert_at -= 1
        # keep exactly one blank line between previous content and new heading
        lines[insert_at:insert_at] = ["", f"### {heading}", "", bullet]
        return "inserted"
    lines[insert_at:insert_at] = [f"### {heading}", "", bullet, ""]
    return "inserted"


def main() -> None:
    changelog = REPO / "CHANGELOG.md"
    lines = changelog.read_text().splitlines()
    bump_cache: dict[str, str] = {}
    version_cache: dict[str, str] = {}
    rows = []
    inserted = present = bullets_total = 0
    orphans = orphan_files()
    for path, type_ in orphans:
        texts = entry_texts(path)
        if not texts or not all(texts):
            sys.exit(f"FATAL: empty fragment {path}")
        sha = adding_commit(path)
        bump = first_bump_after(sha, bump_cache)
        version = bump_version(bump, version_cache)
        heading = TYPE_TO_HEADING[type_]
        subject = run("git", "log", "-1", "--format=%s", sha).strip()
        results = []
        for text in texts:
            bullets_total += 1
            result = insert(lines, version, heading, f"- {text}")
            results.append(result)
            if result == "inserted":
                inserted += 1
            else:
                present += 1
        rows.append((path, type_, sha[:10], subject, version, ",".join(results)))
    # Repair pre-existing "- - " double-bullet artifacts (fragments that
    # carried their own leading dash, doubled by towncrier).
    double_bullets = 0
    for i, line in enumerate(lines):
        if line.startswith("- - "):
            lines[i] = line[2:]
            double_bullets += 1
    changelog.write_text("\n".join(lines) + "\n")
    print(
        f"orphans={len(orphans)} bullets={bullets_total} inserted={inserted} "
        f"already-present={present} double-bullets-fixed={double_bullets}\n"
    )
    print("| Fragment | Type | Added by | PR/commit subject | Restored under | Result |")
    print("|---|---|---|---|---|---|")
    for path, type_, sha, subject, version, result in rows:
        subject = subject.replace("|", "\\|")
        print(f"| `{path}` | {type_} | `{sha}` | {subject} | {version} | {result} |")
    if inserted + present != bullets_total:
        sys.exit("FATAL: count mismatch")


if __name__ == "__main__":
    main()

🤖 Generated with Claude Code