Require complete API review coverage despite sub-agent failures

[Copilot]

Eight auditor batches returned no usable output, leaving a grep-based review of only three principles. This change requires evidence-backed coverage across every component and principle before replacing the ongoing review issue.

Changelog

New

• Coverage contract: Track each component/principle pair, including rest-prop placement, prop merging, ref handling, visibility naming, and mutually exclusive booleans.
• Publication gate: Preserve the existing issue with noop when coverage is incomplete; summarize coverage when complete.

Changed

• Delegation: Pilot batches of five components, cap concurrency at two, inherit the parent model, and retry missing coverage once with a single component.
• Recovery: Stop delegation after two consecutive unusable responses and inspect remaining coverage directly with the same evidence requirements.

Removed

• Bare none responses as sufficient proof of a completed audit.

Rollout strategy

• Patch release
• Minor release
• Major release; if selected, include a written rollout or migration plan
• None; workflow-only change.

Testing & Reviewing

• Verify recovery on the next workflow run: empty or partial auditor responses must not become passes or erase prior findings.
• The underlying sub-agent failure remains unconfirmed; this change addresses recovery and audit completeness.

[changeset-bot]

⚠️ No Changeset found

Latest commit: b73a244

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[joshblack]

@copilot use the feedback from: https://github.com/primer/react/actions/runs/34279566580 to update these instructions as it seems like they lead to a noop

[Copilot]

@copilot use the feedback from: https://github.com/primer/react/actions/runs/34279566580 to update these instructions as it seems like they lead to a noop

Updated in 1898d34. The logs show inherited was sent as a literal model and rejected for missing pricing; the auditor now uses claude-sonnet-5, which worked for the parent in that run. Incomplete coverage now permits labeled partial updates that preserve unchecked findings, report remaining coverage, and prioritize those gaps next time rather than forcing noop. Compilation, formatting/lint, security scan, and review passed. Please rerun to verify runtime behavior.

[joshblack]

@copilot let's update this with the following findings:

• Create a checklist with an overview of proposed API changes that link to findings
• Each finding should be a comment on the issue and corresponds to a specific component
• Group within component comments by findings
• Use tables to organize evidence, impact, and recommendation
• Hide run details, past findings, or remaining coverage list behind details/summary elements (progressive disclosure this kind of stuff)
• The summary should be at most two sentences

[Copilot]

@copilot let's update this with the following findings:

• Create a checklist with an overview of proposed API changes that link to findings
  ...

Implemented in c1df664, with dry-run handling in 6b63004. The overview now has a maximum two-sentence summary and API-change checklist linking to reusable component comments. Findings use evidence/impact/recommendation tables; run details, past findings, and remaining coverage are collapsed. The publisher resolves real comment links and avoids duplicate component threads. Seven focused tests, lint/format checks, strict compilation, and CodeQL passed; live rendering still needs a workflow run.

[joshblack]

@copilot it seems like not all the comments were created, why is that?

[Copilot]

@copilot it seems like not all the comments were created, why is that?

This is a bug in my publisher integration. In run 34388304097, the agent queued 19 component comments, but gh-aw’s output ingestion rejected 18 with:

Too many items of type 'publish_component_findings'. Maximum allowed: 1.

Only Breadcrumbs reached the publisher. My 100-comment guard runs inside the handler—after ingestion—so it cannot override that limit. The run stayed green because these rejections were warnings; the overview update and single accepted comment both succeeded, leaving the other checklist links unresolved.

The tests exercised the handler directly and missed ingestion. I recommend changing this to one bounded batch publishing call and testing the ingestion path too. Would you like me to make that change?

[joshblack]

@copilot go for it

[Copilot]

@copilot go for it

Implemented in b73a244. The publisher now receives one bounded batch instead of one output per component. The regression test runs pinned gh-aw ingestion: it reproduces the 18 dropped outputs, then verifies all 19 comments survive ingestion and reach the publisher in one batch. Eight tests, strict compilation, lint/format checks, review, and security scanning passed. Please rerun the workflow to verify live publishing.

[github-actions]

⚠️ Action required

👋 Hi, this pull request contains changes to the source code that github/github-ui depends on. If you are GitHub staff, test these changes with github/github-ui using the integration workflow. Check the integration testing docs for step-by-step instructions. Or, apply the integration-tests: skipped manually label to skip these checks.