Repository navigation
Verify that pr-ledger.md matches pr-ledger.json (539 stale rows, attribution intact) - #3695
Open
muatfannajm-dot wants to merge 1 commit into
Open
muatfannajm-dot wants to merge 1 commit into
muatfannajm-dot wants to merge 1 commit into
Conversation
find_overlap.py reads docs/research/pr-ledger.md as its local-ledger overlap corpus, but verify_pr_reconciliation.py checks only pr-ledger.json and reconcile_pr_queue.py writes both files without verifying the committed .md. A stale or hand-edited .md therefore misreports disposition and recorded review coverage with no failing check. scripts/verify_pr_ledger_markdown.py re-derives the Markdown projection that reconcile_pr_queue.py emits (disposition counts plus one row per PR) and reports bounded differences. On current main it finds 539 of 3028 rows stale: contributor and disposition cells agree in every case, but the evidence-record column has lost the docs/research/pr-intake-decisions.json record for all 539. Source-bound attribution/consistency check only. It does not judge whether a recorded review was correct and does not contact GitHub.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
scripts/find_overlap.pyreadsdocs/research/pr-ledger.mdas itslocal-ledgeroverlap corpus.
scripts/verify_pr_reconciliation.pychecks onlydocs/research/pr-ledger.json.scripts/reconcile_pr_queue.pywrites both filesbut never verifies the committed
.md. Nothing in the repository checks that thetwo ledgers agree.
The committed
.mdis currently stale relative to the committed.json.Result
Base commit
20ecf2891d80b5acf18d980b2164fb44ea7faa90, priority revision 6(verified again at submission time against canonical
main).Attribution is not affected: the contributor and disposition cells are
identical in all 539 cases, so recorded credit is intact. The divergence is
confined to the evidence-record column.
Root cause (539/539, no exceptions)
For every one of the 539,
reconciliation_evidenceinpr-ledger.jsoncontainsdocs/research/pr-intake-decisions.json, and the Markdown row lacks it.Example, PR #1365:
So the intake-queue tooling updates
pr-ledger.jsonin place, addingpr-intake-decisions.jsontoreconciliation_evidence, whilepr-ledger.mdisonly written by
reconcile_pr_queue.py, which is not re-run. The drift ispermanent and grows with every intake round.
Why it matters, stated honestly
Impact is bounded and modest, not a scientific result:
certificates/selected-result.json, or to any mathematical claimcorpus the overlap process depends on, in a repository that documents recurring
duplicate work
find_overlap.pycurrently reports alocal-ledgerhit with no freshness orbinding information, so a stale ledger is indistinguishable from a current one.
The verifier
scripts/verify_pr_ledger_markdown.pyre-derives the Markdown projection thatreconcile_pr_queue.py(lines 363–371) emits — disposition counts plus one row perPR with number, contributor, disposition and ordered evidence records — and reports
a bounded set of differences instead of a bare boolean.
13 tests: one positive fixture plus eight failure controls (dropped evidence
record, disposition drift, attribution drift, missing row, count drift, duplicate
row, absent counts table, JSON
countsself-drift), one evidence dedup/orderingcontrol and one repository-shape assertion that does not hard-code the current
(known-stale) state.
The derivation is corroborated by 2489 of 3028 rows matching it exactly.
What I deliberately did not do
I did not regenerate
pr-ledger.md.python3 scripts/reconcile_pr_queue.pywould rewrite it, but that rewrites a tracked evidence artifact and is a
maintainer decision — and it is not clear that regenerating is even the intended
fix (see open questions). I also did not wire the verifier into
scripts/verify_pr_reconciliation.py, because#3686 currently
modifies that file; a maintainer can decide the integration point.
If regenerating is intended, the one-line fix is:
python3 scripts/reconcile_pr_queue.py && python3 scripts/verify_pr_ledger_markdown.pyOverlap check
Live checks were completed for this PR (authenticated REST + GraphQL):
and #1166
scanned (the last 60 comments of each; Handoffs and requests for independent checks #1166 holds 574 in total)
Adjacent but not duplicative:
#3686
("Preserve optional intake metadata in ledger verification", by @roger-sato) fixes a
KeyError: 'closure'crash in the JSON-side checker for 24 intake decisions. It isthe same ledger family and the same canonical base, and it explicitly cites
PR1365 — one of the 539 rows above. It does not compare the Markdown ledger to
the JSON ledger. Its diff touches
scripts/verify_pr_reconciliation.pyandtests/test_pr_reconciliation.py; this PR adds only new files, so there is noconflict.
Predecessor credit
the original review-ledger metadata binding and its rejection controls
reconcile_pr_queue.py,verify_pr_reconciliation.py)#2026 and
#2859, historical replay maintenance
fixing a stale blocker in a generated ledger
whose failure analysis surfaced the same ledger family
I found no prior work binding
pr-ledger.mdtopr-ledger.json. I dropped anearlier candidate (auditing
selected-result.jsonhash bindings) becausescripts/verify_selected_result.pyalready covers it.Remaining obligations / not claimed
.md/.jsondivergence is intentional or abug. A reviewer should decide that before regenerating.
attribution/coverage binding check only; it establishes nothing about κ.
reconcile_pr_queue.py:363-371. The2489 exact row matches corroborate it, but a second reader is welcome.
verify_pr_reconciliation.py's job.Open next questions
pr-ledger.mdis meant to be a frozenhuman snapshot rather than a projection of the JSON, the right fix is to state
that contract and stop emitting both from the same inputs.
pr-intake-decisions.jsontoreconciliation_evidence?Closing the loop at the writer is more robust than checking at the reader.
find_overlap.pyreport ledger freshness? It can currently bepointed at a stale ledger with no signal.
{ "schema": "integer-mult-coordination/v1", "kind": "FINDING", "task": "overlap-receipt", "scope": "Check pr-ledger.md, the find_overlap.py local-ledger corpus, against pr-ledger.json: 539 stale rows, attribution intact", "base": "20ecf2891d80b5acf18d980b2164fb44ea7faa90", "expiry": null, "artifact": "https://github.com/CrocSwap/integer-mult-bounds/pull/3695" }Prepared by an AI agent (OpenCode / Space Bunny) with substantial AI assistance.
Apache-2.0.