Skip to content

Fix consistency review PR delta detection#2018

Open
roji wants to merge 1 commit into
mainfrom
roji-fix-consistency-pr-diff
Open

Fix consistency review PR delta detection#2018
roji wants to merge 1 commit into
mainfrom
roji-fix-consistency-pr-diff

Conversation

@roji

@roji roji commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator

Summary

The consistency review agent can choose an incorrect base commit for understanding what changed in a PR, example.

  • require the SDK consistency reviewer to fetch the complete changed-file list and diff through pull_request_read
  • treat GitHub pull-request API data as authoritative and use the checkout only for context and cross-SDK comparison
  • prohibit inferred ranges based on HEAD^, merge-parent ordering, commit history, or local refs; fail closed with missing_data
  • regenerate the gh-aw lock metadata with v0.82.10

Root cause

In SDK Consistency Review run 29561137136 for #2010, the depth-1 checkout had no usable origin/main. The prompt only suggested fetching changed files with GitHub tools, so the agent fell back to local history. It eventually mislabeled the PR head's first parent (0846618) as the base and analyzed 0846618...3745ae2, which is the merge-from-main delta rather than the pull-request delta from ff18f7e...3745ae2. That caused features already on main to be attributed to the PR.

The GitHub MCP server already exposes the authoritative pull_request_read get_files and get_diff methods, so the fix is prompt-only rather than changing checkout behavior.

Validation

  • gh aw compile --strict
  • git diff --check

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 107c6541-f2ca-4f5b-b7f5-9cd70eab5844
Copilot AI review requested due to automatic review settings July 17, 2026 07:36
@roji
roji requested a review from a team as a code owner July 17, 2026 07:36

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the SDK consistency reviewer to identify PR changes exclusively through authoritative GitHub API data.

Changes:

  • Requires complete changed-file and diff retrieval with pagination.
  • Prohibits unreliable local-history delta inference and fails closed on missing data.
  • Refreshes generated workflow metadata.
Show a summary per file
File Description
.github/workflows/sdk-consistency-review.md Adds authoritative PR-delta instructions.
.github/workflows/sdk-consistency-review.lock.yml Updates the generated body hash.

Review details

  • Files reviewed: 2/2 changed files
  • Comments generated: 0
  • Review effort level: Medium

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants