Skip to content

fix: match dependency snapshots on update pull requests - #2492

Draft
groupthinking wants to merge 5 commits into
codex/fix-dependency-review-evidencefrom
codex/fix-dependency-snapshot-parity
Draft

groupthinking wants to merge 5 commits into
codex/fix-dependency-review-evidencefrom
codex/fix-dependency-snapshot-parity

Conversation

@groupthinking

@groupthinking groupthinking commented Oct 28, 2026 •

Copy link
Copy Markdown
Owner

Current status — failed real dependency test

Do not merge this draft. Temporary diagnostic PR #2493 combined the earlier full-tree workflow approach with the fast-uri lockfile delta from #2452 against current main (ba2777d1). Its full-tree scan found npm, PipReport and UvLock components, but GitHub still reported 3 base snapshots versus 1 head snapshot for all 300 seconds of retry. The live dependency diff showed 55 unrelated uv.lock removals. The evidence recorder from #2487 correctly failed, so no complete review artifact was produced. #2493 was closed unmerged.

A second temporary diagnostic, #2494, repeated the same fast-uri update using component-detection action v0.2.0, whose upstream release includes a manifest-mapping fix. Its hosted run submitted one head snapshot successfully, yet the exact compare still warned 3 base versus 1 head after 300 seconds. The API returned 1,482 changes, including 1,347 removals under ../../_temp/dependency-review-base/. The evidence recorder again refused to save a misleading result; #2494 was closed unmerged. The action update alone therefore has not verified snapshot parity on this base.

A third controlled off-main canary #2495 submitted matching base and head full-tree snapshots. Its hosted check initially failed because GitHub still reported no head snapshot after more than five minutes, and no evidence artifact was saved. A later independent read of the same exact off-main commits returned an empty snapshot-warning header and only the intended fast-uri 3.1.7 → 3.1.9 version delta; GitHub commit compare shows only package-lock.json changed. This verifies eventual graph availability for the controlled comparison, but does not pass the hosted check or prove parity against main. #2495 remains closed unmerged.

The integration head 998b19ac8087d4dfbb6a519d5e4b4b4ad7bb12f4 uses the verified upstream v0.2.0 action pin from #2397 for its single full-tree scan. It also carries #2487's rename-safe, lockfile-aware evidence guard. Upstream v0.2.0 assigns dependency scope using each manifest's own graph. This addresses a source-level risk found in the older action; no hosted main-based dependency comparison has yet verified its effect here.

The current workflow on dependency PR #2452 also submits a focused base snapshot to the same main SHA using correlator focused-lockfile-review; the main push submitted a full-tree snapshot. These are distinct manual snapshot streams on the base. Automatic dependency submission is enabled in repository settings, but the source of the third base snapshot has not been verified. The new workflow removes the focused base submission; its main-based outcome still needs a real hosted test.

The 18 focused local tests and this draft's hosted code-only checks validate syntax and code paths, but they did not model multiple snapshot submissions associated with the current main SHA. The approach below is the current candidate, not a verified fix. Revise it and repeat the real manifest-changing test before any merge decision.


Canonical issue

Closes #2490 on merge. This is a stacked draft on #2487 (codex/fix-dependency-review-evidence), which remains unmerged. Retarget to main only after #2487 lands and then validate on a real dependency PR before merging this PR and closing the issue.

Observed failure

GitHub's live compare for dependency PR #2452 returned HTTP 200 with a warning that the base commit ba2777d1b008c2148d413e07cc497791966310e0 had 3 snapshots while the head 5e5864e8cbf7ebdd606fd8c3fd8ea8624d0b4403 had 1. That comparison may show unexpected removals. A successful API status alone is incomplete evidence.

The current main full-tree snapshot was submitted at 08:20 UTC. A focused Uv base submission from #2486 was followed by a 2-base/1-head warning; a focused npm base submission from #2452 was followed by a 3-base/1-head warning. This sequence supports removing focused base submissions. A warning-free comparison on a real update PR against a fresh main SHA remains unverified.

Change

  • Remove the focused one-ecosystem snapshot staging/submission path from .github/workflows/dependency-review.yml.
  • Exclude Dependabot update configuration (.github/dependabot.yml and .yaml) from installed dependency input detection. Real manifests and locks remain subject to review; a config-only PR takes the no-delta path.
  • Submit the same full-tree component-detection scan on main pushes and same-repository dependency PR heads, bound to the exact PR head SHA/ref. Pin the verified upstream v0.2.0 commit, which maps packages and scopes per manifest; this incorporates the action update reviewed in fix: bump advanced-security/component-detection-dependency-submission-action from 0.1.6 to 0.2.0 #2397.
  • Keep dependency review severity/license policy and warning rejection from fix: save dependency review evidence for large updates #2487. Retain the 120-second per-file Python report timeout and declared-dependency fallback; the fallback may omit transitive dependencies.
  • Update workflow tests for manifest selection, matching submission, the v0.2.0 pin, policy preservation and warning rejection.

Paths changed

  • .github/workflows/dependency-review.yml
  • tests/unit/test_dependency_review_workflow.py

Local validation on integration head 998b19ac8087d4dfbb6a519d5e4b4b4ad7bb12f4

Exact-diff and hosted validation on integration head 998b19ac8087d4dfbb6a519d5e4b4b4ad7bb12f4

  • Running the new detector against the actual main → fix: extend Dependabot coverage to Python SDK and MCP server #2447 diff found only .github/dependabot.yml and its test, and returned changed=false, count=0. This corrects the config-only false scan in the candidate workflow; fix: extend Dependabot coverage to Python SDK and MCP server #2447 itself has not yet rerun on this workflow.
  • The prior draft head bb083e0a passed its hosted dependency-review check via the no-delta path because this PR changes workflow code and tests, not a package manifest. Hosted checks on the integration head are being verified; see the live check list. Neither run proves snapshot parity or scope accuracy for a real dependency update.

Warning-free comparison is not sufficient by itself

The earlier #2495 warning-free response labeled the removed root fast-uri 3.1.7 entry runtime, although both exact root lockfiles mark fast-uri dev: true. The older v0.1.6 action caches a package by PURL across manifests; the newly pinned v0.2.0 action uses each manifest's dependency graph. The current #2487 guard now rejects that warning-free but semantically wrong #2495 response without writing evidence, and also rejects the 3-base/1-head warned comparison for #2452. It accepts the controlled v0.2.0 #2496 comparison below. The guard validates changed manifest paths and npm lockfile package/version/scope claims against the exact base and head files, including both sides of a renamed manifest. Main-based parity and non-npm metadata accuracy remain unverified. An empty warning header and matching snapshot counts alone do not prove semantic accuracy.

Controlled v0.2.0 scope result — 2026-10-28 15:16 UTC

Temporary #2496 used this draft’s exact v0.2.0 pin on an isolated off-main base and head. Base run 40713010942 and head run 40713661006 both submitted full-tree snapshots successfully. The head dependency-review check passed and uploaded a two-entry artifact. An independent GitHub dependency comparison returned HTTP 200 with no X-Github-Dependency-Graph-Snapshot-Warnings header and exactly the root package-lock.json fast-uri 3.1.7 removal and 3.1.9 addition, both scope: development, no vulnerabilities or unrelated removals. The API result matches the hosted artifact and its recorded SHA-256. Both root lockfile revisions mark fast-uri dev: true; the unchanged docs fetch-MCP lockfile retains 3.1.7 as runtime. This corrects the specific #2495 scope mismatch in the controlled fixture. #2496 was closed unmerged and its temporary remote branches deleted after readback.

This is off-main scope evidence only. It does not clear the three-base/one-head snapshot warning observed on real PRs against current main; the gates below remain open.

Remaining gates

  • fix: save dependency review evidence for large updates #2487 merged and this PR retargeted to main.
  • Hosted checks on the final base/head pass.
  • A real dependency PR produces matching full-tree base/head snapshots with no GitHub snapshot warning; inspect the exact compare for correct changed package names, versions, manifests, scopes, vulnerabilities and licenses against the actual dependency files, with no unrelated removals, then verify the complete evidence artifact.
  • Required dependency review verdict reflects the actual changed packages, including severity and license policy.
  • Review findings resolved and merge decision made on the final version.

The current draft CI on this stacked base can test code and workflow syntax, but its own diff contains no dependency manifest change, so it cannot establish the real dependency PR parity outcome.

@vercel

vercel Bot commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
v0-uvai Ready Ready Preview, v0 Oct 28, 2026 3:58pm UTC

@coderabbitai

coderabbitai Bot commented Oct 28, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

This branch was successfully deployed

1 active deployment
Preview — 998b19ac Deployed Oct 28, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants