Tooling: search open-PR overlap by changed file path in find_overlap (no new kappa)

[navinor]

Result (tooling; no new kappa)

scripts/find_overlap.py can now answer which open PRs touch a given file path — the changed-file index requested in the workflow feedback thread ("the overlap helper can't search by file path ... cache each open PR's changed-file list and make it searchable by path").

python3 scripts/find_overlap.py --path docs/research/current-status.md
python3 scripts/find_overlap.py --path scripts/find_overlap.py --path tests/test_research_onboarding.py

Answered live at the measured snapshot: docs/research/current-status.md is touched by exactly one open PR (#1232); research/lean-external-review is touched by #1365 and #1494, the stacked level-model phases of #442.

Scope

Track verifier-engineering, card replay-runtime, scope key b1dd41940c60d240b47a, chooser seed a0107446c288cf7a8989cfea985c3344 (OS-random draw). Base 1f327c6f3e3a35271c5c141d381b540bab8ad005; contribution head 7df8e542 on branch research/agent-20-overlap-path-index-a0107446; tooling commit 04aecc3d, package commits c05e8f30 and 7df8e542. Scope status refined, within the card's allowed variation "design one source-bound cache boundary": the measured check is the overlap helper's own remote refresh, the coordination check the feedback thread names as expensive and unable to see code paths. Task card: research/overlap-path-index/TASK.json. This is verifier engineering: no mathematical claim, kappa, supplier, certificate, replay coverage or selected-frontier change.

Delta

Two commits, five files plus one package:

• scripts/find_overlap.py: new fetch_pr_files (paginated GraphQL changed-file inventory, third snapshot source pr_files, fetched only for --path queries so body searches are byte-identical in behavior and cost), new path_report, and a --path PATTERN flag (repeatable; fnmatch pattern or literal substring). With both axes given, matches is the term-and-path intersection while path_matches stays the superset of all path hits (no silent false negative). PRs whose changed-file list is unavailable (e.g. Conditional κ=0.0011208649522582786: split centres with complete signed admission #376 at the measured snapshot) are recorded empty and flagged truncated: true; PRs beyond GitHub's 100-file listing are flagged truncated: true. A failed or partial refresh keeps the previous snapshot and warns; it never reports "no overlap" from missing data without a warning. Diff contents remain unsearched; the report carries a path_limits statement.
• tests/test_research_onboarding.py: six new fixture tests (exact/substring/glob matching with a rejection control, pagination with truncation flags, null file-list and null node handling, transient-failure retries, fetch-only-on-demand with cache reuse and offline warnings, refresh-failure keeping old records).
• research/overlap-path-index/: registered package (TASK.json with the card and scope-change reason, SOURCES.json exact pins, VERIFY.json, verify.py).

Evidence

Measured on Windows 11, CPython 3.13.15, this host:

Check	Before	After
Body-only overlap query (cached, offline)	0.48 s	0.48 s (pr_files never fetched)
Full changed-file index build	impossible (no path data)	87.7 s: 745 open PRs, 20,117 changed-path entries
Cached offline --path query	manual per-PR inspection	0.48 s

Page-size evidence: 25-PR pages (up to 2500 nodes) exceed GitHub's ~10 s gateway limit and returned HTTP 502 (failing calls died at 10.5–10.7 s); 10-PR pages completed in 0.8–4.4 s and are what the tool uses, with up to four attempts per page.

Verification (reproducible commands and output):

python3 scripts/research_package.py --package research/overlap-path-index --check-only
# PASS_PACKAGE_METADATA_AND_PINS (not a mathematical acceptance check)
python3.12 scripts/research_package.py --package research/overlap-path-index
# PASS_OVERLAP_PATH_INDEX  (2 pinned inputs, 11 unit tests, 7 controls)
# PASS_REGISTERED_RESEARCH_PACKAGE research/overlap-path-index outcome=completed
python3.12 -B -m unittest discover -s tests -p 'test_research_*.py'
# Ran 26 tests, FAILED (errors=6)

The 6 errors are the pre-existing PackageTests Windows CRLF-vs-autocrlf fixture failures documented by #1298; they are untouched here. Baseline recorded from the base commit's tree on this host: Ran 20 tests ... FAILED (errors=6) — the same 6 errors, so this change adds 6 passing tests and no new failure. The package verifier's seven controls (positive, substring, glob, rejection, term-and-path, mutation, missing-index) run hermetically against a fixture snapshot cache placed outside the repository.

Not checked: no hosted-CI run, no Linux/macOS execution, no independent human review, no full selected cold replay, no closed-PR path coverage.

Overlap

Fresh find_overlap.py searches before submission for overlap-path-index, path index, changed-file index, find_overlap.py, searchable by path, changed-file list, file inventory cache matched only the request itself, this package and this PR's own claim (#1166 (comment)). The new --path mode reports the nearest textual neighbors: #1298 (Windows/UTF-8 portability of the same script and tests; complementary, different concern, possible textual conflict) and #1299 (PYTHONOPTIMIZE assertions in the same test file). Replay optimizations in #1461, #1584 and #1101 concern different objects. No matching active claim was found; claims are advisory and no-match is not a novelty certificate.

Open next questions

In research/overlap-path-index/README.md and structured in its TASK.json: closed-PR path coverage via the ledger's COVERAGE.json.gz inventories (blocked on an explicit coverage label); cheaper index refresh via per-page caching keyed by PR head sha (blocked on GraphQL having no ETags for this connection); exact base/head stacking chains beyond shared-path approximation (blocked on stacked PRs naming parents only in prose).

Credit

• Douglas Colkitt: the overlap helper, its snapshot design and coverage contract (Apache-2.0, retained; changed only as described here).
• Mister-Meeseeks: the changed-file request, discussion 1200 item 2.
• rtyu193-rgb (Tooling: Windows portability for research onboarding scripts and tests (no new kappa) #1298): onboarding UTF-8/Windows fixes to the same script; complementary and not included here.

Contribution by Aaron Wang / navinor, with help from opencode (substantial AI assistance in design, implementation, tests, measurements and writing). Apache-2.0. No merge or selected-frontier change requested.

Workflow feedback (brief)

Task replay-runtime refined after the draw; the chooser's card did not reflect that both suggested objects had live coverage nearby. Biggest friction: GitHub's ~10 s GraphQL gateway limit 502ing large changed-file pages (cost two debugging rounds; recorded above as evidence). Suggested improvement: the card's first_step could link the feedback thread's open tooling requests the way it links the baseline command — this task came from there, not from the catalogue.

[Mister-Meeseeks]

Batch review at 7df8e5429e1aa9f8acb3ca02761b98e86cd374c2.

Merged to main, preserving the original contributor commits and package attribution.

Reviewed scope: Optional overlap search by changed-file path with paginated GraphQL index, explicit per-PR truncation/unknown coverage and mocked failure controls. Combined onboarding tests required with PR1702; search remains advisory, not a novelty certificate.

The pinned scoped command passed; the batch ledger contains its command, input hashes, environment and log. This incorporates the research/tooling contribution and does not change the selected κ or discharge the inherited multiplication-theorem assumptions.

Batch review, attribution and evidence. Thank you for the contribution.