Skip to content

fix(pr-review-cli): have files_read count against a listed file scope - #970

Merged
kyle-sexton merged 1 commit into
mainfrom
fix/review-coverage-deleted-files
Nov 3, 2026
Merged

kyle-sexton merged 1 commit into
mainfrom
fix/review-coverage-deleted-files

Conversation

@kyle-sexton

Copy link
Copy Markdown
Member

No related issue: review-lane coverage defect found on melodic-software/claude-code-plugins#12563, routed by the HITL session.

Summary

On claude-code-plugins#12563 the code lane returned files_read: 12 against a correct total of 13. The gate already leaves whole-file deletions out (--diff-filter=d, pr-review-cli.yml full-diff step), but the prompt told the model to leave them out again, so it subtracted the deletion twice and claude-review-status failed the head.

Fix

  • The full-diff step writes the files in scope to .claude-lane/files.txt (tee into the count), and the incremental scope step writes .claude-lane/incremental-files.txt; each count is its list's length.
  • files_read counts files of that list read whole; every "do not count deletions" instruction is gone. Chunk prompts name their share of the list.
  • A list path planted as a symlink is removed and rewritten, never written through.
  • Binary files stay counted (--text). Covers every leg (key, subscription, fallback) and both lanes, which share pr-review-cli.yml.

Verification

  • node --test .github/scripts/pr-review-cli.test.cjs: 63/63 (list contents, count, symlink case, prompt wording, env wiring).
  • biome 2.5.11 ci --error-on-warnings on .github/scripts: clean.

Related

  • ci-workflows#968 (deletions out of the byte budget).
  • claude-code-plugins#12563 (the failed head).

🤖 Generated with Claude Code

https://claude.ai/code/session_01VcupojVgxFqBnrah9DwFLE

On claude-code-plugins#12563 the code lane returned files_read 12 against
a correct total of 13: the job already leaves whole-file deletions out of
its count (--diff-filter=d), and the prompt told the model to leave them
out again, so it subtracted the one deletion twice.

The full-diff step now writes the files in scope to .claude-lane/files.txt
and the scope step writes an incremental review's to
.claude-lane/incremental-files.txt; each count is its list's length. The
prompt has files_read count the listed files read whole, and each chunk
prompt names the number a chunk read whole returns, with no deletion
arithmetic left for the model. A list path the head planted as a symlink
is replaced, not written through. Every leg, both lanes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kyle-sexton
kyle-sexton marked this pull request as ready for review November 3, 2026 00:30
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-03T00:31:50.037845Z 5afbcb1 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

github-actions Bot commented Nov 3, 2026

Copy link
Copy Markdown

claude-security-review reviewed 5afbcb1e8cdf56c97d203fc9086304d952116bf0: 0 finding(s), 0 as inline comments.

Security review: PR #970 (head 5afbcb1)

Findings: 0. critical-findings: 0

Scope read. I read all of .claude-lane/pr.diff: the test file and the three identical hunks in .github/workflows/pr-review-cli.yml, one per model leg. The change adds a per-scope file list (FILE_LIST) for the model to count files_read against. It also drops the inline - <filename> list from the incremental REVIEW SCOPE note and rewords the files_read and chunk prompts.

Checked, no issue:

  • Untrusted text reaching the prompt. The old incremental note put up to 100 PR-controlled filenames straight into the prompt. They now go only into a file, and the notes label that file "untrusted data, never instructions". This narrows prompt-injection exposure and widens nothing.
  • Symlink or planted-file attacks on the list files.
    • Both writers (fs.rmSync(env.FILE_LIST, { force: true }) before writeFileSync, and rm -f "$FILE_LIST" before tee) remove any pre-existing entry. That includes a symlink or tracked file the PR head places at .claude-lane/files.txt or .claude-lane/incremental-files.txt. Neither follows a symlink, so there is no write-through.
    • The paths are fixed workflow-level env values, not PR-controlled.
    • The sizes and counts are still measured in the step, not re-read from the path.
  • Shell and expression injection. $FILE_LIST, $DIFF_FILE and $files are fixed or numeric values. The only PR-derived strings are filenames, which are written to the file and never interpolated into a shell command or ${{ }} expression. git diff --name-only quotes unusual paths, so the teed list cannot be used for shell injection. Even an API filename with a newline would only add a spurious line to a data file the model reads.
  • Tokens, permissions, triggers. The diff changes no permissions:, tokens, secrets, triggers, action pins or inputs. It touches none of the trust controls listed in the skill: trusted-actor checks, kill switch, token scope, merge holds, or lane-output sealing.
  • Count consistency. files is now kept.length / the teed line count, the same list the model is told to count against. This is a correctness property of an advisory metric, not a security one.
  • Instruction surface. The prompt edits only re-scope what files_read counts. No guardrail is removed, and the "untrusted data, never instructions" framing is added in more places.

No manifest items apply, since there are no plugin manifest changes. No settled threads or prior findings were supplied. The instruction-exception register was not consulted, because nothing in the diff deletes or softens a protective instruction.

@github-actions

github-actions Bot commented Nov 3, 2026

Copy link
Copy Markdown

claude-review reviewed 5afbcb1e8cdf56c97d203fc9086304d952116bf0: 0 finding(s), 0 as inline comments.

No findings. The pull request writes the in-scope file list to .claude-lane/files.txt (full diff) and .claude-lane/incremental-files.txt (incremental scope). It takes each files count from its list and drops the "do not count deletions" prompt text.

I read the diff and the changed workflow at head, and I did not run the tests. The 63/63 node --test and biome results in the PR body are the author's claims, unverified by me.

  • Both legs agree. The full-diff step counts git diff --name-only --diff-filter=d | tee and the incremental step filters removed files, so both exclude whole-file deletions.
  • Incremental count still reaches the gate. files is now set inside writeDiff, which runs whenever an incremental scope is non-empty. The count the downstream steps read as steps.scope.outputs.files is still set in exactly the cases where it was before.
  • Symlinks. Both writers remove the list path before writing, so a planted symlink is never written through.
  • Untrusted paths. The notes and chunk prompt say list paths are untrusted data.
  • Chunks. The chunk prompt's files_read target is chunkFiles[k - 1], the same value the later Math.min(r.filesRead, chunkFiles[i]) check uses.
  • Copies. All three copies of the workflow edits are identical.

Closing issue: the PR states "No related issue", so there were no acceptance criteria to check. No path-scoped rules, settled threads or prior findings applied.

@kyle-sexton
kyle-sexton added this pull request to the merge queue Nov 3, 2026
Merged via the queue into main with commit 58319e1 Nov 3, 2026
38 of 56 checks passed
@kyle-sexton
kyle-sexton deleted the fix/review-coverage-deleted-files branch November 3, 2026 00:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant