Skip to content

feat(pr-review-cli): add a generated-data input for files no review reads - #971

Merged
kyle-sexton merged 3 commits into
mainfrom
feat/review-generated-data
Nov 3, 2026
Merged

kyle-sexton merged 3 commits into
mainfrom
feat/review-generated-data

Conversation

@kyle-sexton

@kyle-sexton kyle-sexton commented Nov 3, 2026 •

Copy link
Copy Markdown
Member

Refs: melodic-software/claude-code-plugins#12558

Summary

pr-review-cli.yml gains a generated-data input: the repository path of a
list file (one path or glob per line, blank lines and # comments skipped)
naming generated data files the review never reads. melodic-software/claude-code-plugins#12558
adds a 184 KB generated verify-legacy-fails.json, which no review reads
whole, so every head fails publish-review with failure-class=partial
("read 2 of 3 changed file(s)"). generated-copies, the only exemption today,
needs a source file.

Fix

  • The list is read at the merge base only (git cat-file blob "$base:$PATH"),
    as generated-copies is, so a pull request's own copy of the list declares
    nothing (see Residuals for the limits of that). A list missing there warns
    and declares nothing.
  • Full-diff step, all three legs (key, subscription, fallback): the declared
    files are computed first, and one :(exclude,literal)<path> pathspec array
    goes to every git call (the pr.diff tee, the deleted-bytes count, the
    files.txt list, the --raw feeding the copy groups). A data file leaves
    the diff, the list, the file total, the size and the copy groups together,
    so the split and extract steps' copy arithmetic still agrees. The glob
    function the copy groups use is now shared by both awk programs.
  • Scope step, all three legs: an incremental review applies the same rule to
    the change from the merge base and drops data files from
    incremental-files.txt, its count and incremental.diff. It does so before
    the missing-patch check, so a data file whose large patch the API omitted
    keeps the push incremental; a missing patch on any other file still forces
    a whole review. A push of data files alone skips as no-change with its own
    reason.
  • A file is left out only as a modification or addition of a non-executable
    regular file at a path of letters, digits and ._/@+-, whose diff section
    is over DATA_FLOOR_BYTES (7500). Basis for 7500: one review reads at most
    READ_BUDGET (300000 bytes) across MAX_CHUNK_FILES (40) files, so 7500 is
    one file's even share; a smaller change, such as one deleted FAIL record,
    fits a review and is reviewed. An executable, a symlink, a rename, or a
    hold-paths or SENSITIVE_PATHS match is still reviewed, and an empty
    SENSITIVE_PATHS declares nothing, as the copy groups fail closed.
  • Visible record: the extract step adds coverage.data_files (the names) and
    data files left out: N to its job summary. A count its list cannot back
    makes the review incomplete. publish.cjs validates the list and writes
    ; N data file(s) not reviewed: <path>, ... into the marker's status line
    and the shadow report's coverage line, naming the first 10 and then
    and M more.
  • The claude-lane-review-complete notice is unchanged: both merge-gate
    parsers anchor on $ (claude-code-plugins
    .github/scripts/pr-merge/clear-escalations.mjs and
    babysit_review_evidence.py), so a new field there would break them.
  • Gitleaks and every other non-review check are untouched and still scan the
    file.

Verification

  • node --test over every .github/**/*.test.cjs: 1057 pass, 0 fail.
  • Size floor: the fixture's declared files change 400 records each (well over
    7500 bytes) and leave the review; data/small.json, also declared, loses one
    record and stays in the list. Setting the floor to 0 failed both the
    full-diff and incremental tests.
  • Missing patch: an incremental push whose data file has no patch stays
    scope=incremental; the same push with the code file's patch missing is
    scope=whole.
  • The status line names 10 files exactly, and 12 as the first 10 and
    and 2 more.
  • New tests, on one real-git fixture: a file the base list declares leaves
    files.txt, files=, pr.diff and the copy groups; no list, a list the base
    lacks, and a list only the head holds declare nothing; an executable, a
    SENSITIVE_PATHS match, a hold-paths match, a symlink and a non-plain path are
    not exempted. The same rules hold in the incremental scope step. The extract
    step names the files, and a mismatched list is incomplete. The status line
    and shadow report name the file.
  • Sabotage check: dropping the SENSITIVE_PATHS exception in the bash rule, then
    in the JS rule, failed the matching test each time.
  • actionlint, zizmor, biome, markdownlint and typos are clean on the changed
    files.

Residuals (accepted, not built)

Raised by the Automation Lanes review and accepted for this pull request:

  • Guards with no test. The empty-SENSITIVE_PATHS case, rename and type
    change (R/T) statuses, and a path over 500 characters have no test.
    Removing s == 0 / sensitive.length === 0, NF != 2 / rest.length > 0,
    or the length cap leaves every test green. The incremental bytes assertion
    computes its expected value from the same section format the code writes,
    so it checks itself.
  • "A pull request cannot declare its own files" is too strong. The list
    is read at the merge base, but the input itself is set by the caller
    workflow in the pull request's own head (a change there is reviewed, since
    .github/** is a SENSITIVE_PATHS match). A branch cut from an older main
    also keeps globs that main later removed. Also requiring the list at
    pull_request.base.sha would close both.
  • A literal exclude matches a directory too. :(exclude,literal)<path>
    also drops paths under a base-side directory of the same name.

Related

Consumer order: this merges first, then claude-code-plugins adds the list file
and generated-data: to its pr-review-cli.yml callers. Only once the claude-code-plugins pull request's merge base holds the
list does its review leave the file out.

https://claude.ai/code/session_01VcupomBPg7kuobwLoNpxEW

🤖 Generated with Claude Code

kyle-sexton and others added 2 commits November 2, 2026 20:32
…eads

A generated data file no review can read whole, such as
claude-code-plugins#12558's 184 KB verify-legacy-fails.json, made every
review partial. The new generated-data input names a list file of paths
or globs, read at the merge base only. A changed file it declares leaves
the diff, files.txt, the file total, the size and the copy groups in the
full-diff step, and the incremental list, count and diff in the scope
step, in all three legs. An executable file, a symlink, a rename, a
hold-paths or SENSITIVE_PATHS match, or a path outside plain characters
is still reviewed. The extract step names each left-out file in the
coverage (data_files), and the marker's status line and the shadow
report state the count and names. The completion notice is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VcupomBPg7kuobwLoNpxEW
…t the scope

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VcupomBPg7kuobwLoNpxEW
@kyle-sexton
kyle-sexton marked this pull request as ready for review November 3, 2026 01:41
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-03T01:44:15.118285Z 151999f Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

github-actions Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

claude-security-review reviewed b099c9c4ecced81c98c9ebdf03cfc5e4f1556138: 0 finding(s), 0 as inline comments.

No security findings in the change since 151999f.

I read .claude-lane/incremental.diff for pr-review-cli.yml, publish.cjs, both test files and README.md. The change adds DATA_FLOOR_BYTES (7500). A declared generated data file now leaves the review only when its diff section is over that size. The status line names the first 10 such files and counts the rest.

  • The exemption gets narrower. A data file with a small change is now reviewed, so the diff does not widen what reaches an unreviewed path. The other eligibility checks in the awk filter are unchanged, and a missing or non-numeric DATA_FLOOR_BYTES makes both the shell and the JS legs declare no data files. That fails closed to a full review.
  • Path handling is safe. git diff ... -- ":(literal)$p" takes only paths the awk filter has already limited to [A-Za-z0-9._/@+-] and 500 characters. The path is quoted and passed through the literal pathspec magic, so there is no injection surface.
  • Dropped patches stay safe. A data file whose API patch was dropped still leaves the scope. Any other file with a dropped patch still forces a whole review, and the new tests cover both cases.
  • No new data crosses a trust boundary. The status line still prints only filenames that passed the plain-character filter, so publish.cjs has no new markdown or injection surface.
  • Trust controls are untouched. The change touches no trusted-actor list, token scope, kill switch, merge hold or sensitive-path list, and permissions: and the triggers are unchanged. It adds no plugin manifest changes.

Pre-existing design, not part of this diff: a PR author can still put content in a data file the base branch already declares, and it goes unreviewed once its diff exceeds the floor. The floor narrows this rather than widening it.

critical-findings: 0

@github-actions

github-actions Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

claude-review reviewed b099c9c4ecced81c98c9ebdf03cfc5e4f1556138: 0 finding(s), 0 as inline comments.

No findings. I reviewed the incremental change (.claude-lane/incremental.diff) against b099c9c. I read the diff and the workflow source around it. I did not run the tests; the PR's 1056-pass claim is the author's and unverified by me.

What changed:

  • publish.cjs names only the first 10 data files in the status line and counts the rest.
  • pr-review-cli.yml adds DATA_FLOOR_BYTES (7500). A declared data file now leaves the review only when its diff section is larger than that, so a small edit to it is still reviewed.
  • The floor is applied in all three legs. The full-diff step compares the size of git diff for the single :(literal) path. The scope step compares the byte length of the API patch, and a dropped patch counts as over the floor.
  • Tests and the README follow.

Checked and found sound:

  • Floor value: 300000 / 40 = 7500, matching the READ_BUDGET and MAX_CHUNK_FILES the comment cites.
  • Validation: DATA_FLOOR_BYTES is validated by the same ^[0-9]{1,9}$ rule in bash and JS, and a bad value declares nothing, which fails closed. It is workflow-level env, so every step sees it.
  • patches.get(file.filename) in the scope step: it cannot be undefined for a file in data. dataFiles admits only plain M/A files, so those have no previous_filename, and names(file) is just filename, which must be a key of patches for the file to be in changedFiles.
  • Dropped patches: a data file moved into left before the missing-patch check no longer forces a whole review. A dropped patch on any other file still does, and the new test covers both.
  • Empty array: "${candidates[@]}" is safe on the runner's bash 5.
  • Pathspec order: the only exclude pathspec is still built from the filtered data array, so the diff, list, counts and copy groups stay consistent.
  • Merge-base trust: declarations still come from the merge base, so a pull request cannot add its own exemptions or lower the floor.

Not a finding: the full-diff step sizes a file by its whole-PR diff, and the incremental step by the change since the last review. A file's classification can differ between the two on edge cases, such as a file that was left out and later gets a small edit. Both outcomes are safe, because the file is reviewed rather than silently skipped.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 151999fb61

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/pr-review-cli.yml Resolved
… data patches incremental

- A declared data file leaves the review only when its diff section is
  over DATA_FLOOR_BYTES (7500: one review's 300000-byte read budget over
  the 40 files it reads whole), so a one-record edit is still reviewed.
- sinceLastReview drops data files before the missing-patch check, so a
  data file whose large patch the API omitted keeps the push incremental.
- The status line names the first 10 data files, then "and N more".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VcupomBPg7kuobwLoNpxEW
@kyle-sexton
kyle-sexton added this pull request to the merge queue Nov 3, 2026
Merged via the queue into main with commit 51bd90b Nov 3, 2026
54 checks passed
@kyle-sexton
kyle-sexton deleted the feat/review-generated-data branch November 3, 2026 02:04
kyle-sexton added a commit to melodic-software/standards that referenced this pull request Nov 3, 2026
Refs: melodic-software/claude-code-plugins#12558
<!-- session: poteto -->

## Summary

Adds `generated-data` to the reviewed `allowedInputs` of
`melodic-software/ci-workflows/.github/workflows/pr-review-cli.yml` in
`components/runner-policy/policy.json`.

## Fix

melodic-software/ci-workflows#971 (merged) added the opt-in
`generated-data` input. Without this entry, a caller that passes it
fails `runner-policy` with "the reusable workflow call has inputs absent
from its reviewed contract: generated-data" (seen on
claude-code-plugins#12601). The input names a repository file listing
generated data files the review skips. It carries no secret and does not
affect runner routing.

## Verification

- `node --test components/runner-policy/runner-policy.test.mjs`: 226/226
pass.

## Related

- melodic-software/ci-workflows#971 (the input);
melodic-software/claude-code-plugins#12601 (the caller opt-in, waiting
on this through the sync). Same shape as #940.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01VcupomBPg7kuobwLoNpxEW

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant