Fix agentic triage comment submission and reject incomplete summaries

[priyankatiwari08]

Summary

Repair the safe-output submission failure behind #4704. In the recorded run for #4193, jq payload preparation was denied by the shell allowlist. The agent then submitted a diagnostic test message please ignore comment, consumed the one-comment quota, and could not publish its real triage summary.

• Allow jq and document separate, checked JSON payload preparation followed by exactly one safeoutputs add_comment . submission. Explicitly prohibit diagnostic writes, -/@- body experiments, and retries after a submission error.
• Add a trusted pre-publication validation step that checks the entire safe-output batch before any comments or labels are applied. Reject placeholder/unfinished comments, quoted or fenced templates, missing check rows or sections, multiple comments, label-only results, and mixed success/incomplete outcomes. Preserve explicit no-op and failure reporting.
• Keep the existing trigger conditions, one-comment quota, label allowlist, on-demand label behavior, and read-only agent permissions. Checkout only the trusted validator at the event SHA, with persisted credentials disabled.
• Regenerate the workflow lock with the existing gh-aw v0.88.2 compiler and add regression coverage to the lock-verification workflow.

Validation

• All 11 Node.js regression tests pass, including the exact jq encoding/check expressions with quotes, backticks, backslashes, shell-like text, and newlines.
• The original Entra ID authentication broken under NativeAOT in v7.0 — Extensions.Azure reflection-based discovery is AOT-incompatible #4193 agent output is rejected both with its incomplete report and as a placeholder-only batch.
• gh aw compile succeeds; repeating compilation leaves the generated lock byte-for-byte unchanged.
• git diff --check passes.

Rollout and recovery

Related to #4704; intentionally do not close that tracker on merge. The previously reported completion count included workflow-attributed placeholders and is being re-audited. This PR fixes the workflow, not the existing backlog results.

After this reaches the default branch, verify a small representative pilot, retry only genuinely incomplete issues in completion-gated batches within the existing daily credit budget, and delete confirmed placeholder comments only after replacement summaries succeed. Keep #4704 open until recovery is verified. No backlog retries or comment deletions are included in this PR.

Checklist

• Tests added or updated
• Public API changes documented — N/A; workflow-only change
• Verified against customer repro (if applicable) — recorded workflow failure replayed; no product repro applies
• Ensure no breaking changes introduced — no public API changes; malformed triage output is intentionally rejected

[Copilot]

🟡 Changes recommended

The validator still accepts collector errors, embedded placeholders, and contradictory no-op/incomplete batches.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Repairs issue-triage safe-output submission and adds pre-publication validation.

Changes:

• Enables safe JSON payload creation with jq.
• Validates triage summaries before publishing comments or labels.
• Adds regression tests and regenerates the workflow lock.

File summaries

File	Description
.github/workflows/verify-aw-lock.yml	Runs validator tests during lock verification.
.github/workflows/issue-triage.md	Adds safe submission guidance and validation steps.
.github/workflows/issue-triage.lock.yml	Regenerates the executable workflow.
.github/scripts/validate-triage-output.cjs	Implements output validation.
.github/scripts/tests/validate-triage-output.test.cjs	Adds validator regression coverage.
.github/scripts/tests/README.md	Documents test execution and scope.

Review details

Suppressed comments (2)

.github/scripts/validate-triage-output.cjs:36

• The validator ignores the collector's top-level errors array. gh-aw records rejected/over-quota safe-output calls there while retaining any valid items, so a valid-looking comment plus a failed label or second submission is currently published as success. Reject any non-empty output.errors before processing items, and cover that mixed batch in the regression tests.

function validateTriageOutput(output) {
    if (!output || !Array.isArray(output.items) || output.items.length === 0 ||
        output.items.some(item => !item || typeof item.type !== 'string')) {
        throw new Error('Missing or malformed triage output: expected non-empty result items.');

.github/scripts/validate-triage-output.cjs:50

• A batch containing both noop and report_incomplete has no comment, enters this branch, and returns successfully. Those are contradictory completed/no-action and failed outcomes, so this still permits a mixed success/incomplete batch. Reject incomplete && noop before the no-comment return and add a regression test.

    if (comments.length === 0) {
        if (!incomplete && !noop) {
            throw new Error('Triage output has no summary or explicit no-op/incomplete result.');
        }
        if (output.items.some(item => ['add_labels', 'remove_labels'].includes(item.type))) {

• Files reviewed: 6/6 changed files
• Comments generated: 1
• Review effort level: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.