Repository navigation
Structure generated issues with a normalized report AST - #69756
Conversation
Render failure summaries, diagnostics, attribution, expiration, and machine-readable markers through a single escaping and normalization pipeline. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Add semantic closing references, structured data, and safe hidden metadata nodes. Route REST, GraphQL, and provider issue creation through shared report normalization with compatibility regression coverage. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Generated failure issue previewsThese previews render the three original failure examples through the new report AST. Original run links, diagnostics, footer attribution, expiration, and failure-match metadata are retained; the generic diagnostic fallback uses the updated wording. The source issues have not been modified. Missing required tools: Smoke CodexSource: #69737 New title: What happenedSmoke Codex is missing required tools. View run #39046125927 · Workflow source · Previous failures Branch: Next stepReview the failure details below for a targeted fix, or assign this issue to a coding agent to investigate. Failure details
Missing Tools:
Copy a debugging promptPaste this into your coding agent: If the repository includes the Manage failure notificationsTo stop creating failure issues for this workflow, set this in its frontmatter and recompile: safe-outputs:
report-failure-as-issue: falseThis only disables issue reporting; it does not fix the failure.
Unknown failure cause: Smoke GeminiSource: #69732 New title: What happenedSmoke Gemini failed (cause unknown). View run #39046126074 · Workflow source · Previous failures Branch: Pull Request: #69581 Next stepReview the failure details below for a targeted fix, or assign this issue to a coding agent to investigate. Failure detailsRuntime evidence
Error details:A comment or other safe output does not confirm that the requested task was completed. Only inferred runtime status and error codes are reported; agent text is excluded. Failure DiagnosticsAgent job: No additional engine error was captured. Check the reported failure signals and linked run for details. Only runtime metadata is shown; agent-generated text is excluded. Copy a debugging promptPaste this into your coding agent: If the repository includes the Manage failure notificationsTo stop creating failure issues for this workflow, set this in its frontmatter and recompile: safe-outputs:
report-failure-as-issue: falseThis only disables issue reporting; it does not fix the failure.
Missing model pricing: Smoke Copilot - AOAI (Entra)Source: #69743 New title: What happenedSmoke Copilot - AOAI (Entra) has no AI credits pricing for model (o4-mini-aw). View run #39046722029 · Workflow source · Previous failures Branch: Next stepReview the failure details below for a targeted fix, or assign this issue to a coding agent to investigate. Failure details
This is a configuration issue — retrying will not help. The model must have pricing before the workflow can run. How to fix thisOption 1 — Add pricing in the workflow frontmatter: models:
providers:
github-copilot:
models:
'o4-mini-aw':
cost:
# Placeholder values — replace with actual pricing for this model
input: "0e0" # $0.00 per million input tokens
output: "0e0" # $0.00 per million output tokens
cache_read: "0e0" # $0.00 per million cache-read tokens
cache_write: "0e0" # $0.00 per million cache-write tokensUse the provider key matching your engine: Option 2 — Map the model to a known model using the If models:
'o4-mini-aw':
- claude-sonnet-4-5Option 3 — Switch to a model already in the built-in pricing table: Replace Runtime evidence
Error details:A comment or other safe output does not confirm that the requested task was completed. Only inferred runtime status and error codes are reported; agent text is excluded. Failure DiagnosticsAgent job: No additional engine error was captured. Check the reported failure signals and linked run for details. Only runtime metadata is shown; agent-generated text is excluded. Copy a debugging promptPaste this into your coding agent: If the repository includes the Manage failure notificationsTo stop creating failure issues for this workflow, set this in its frontmatter and recompile: safe-outputs:
report-failure-as-issue: falseThis only disables issue reporting; it does not fix the failure.
|
Render report sections as H3 and keep imported diagnostic headings at H4 or below. Cover the heading hierarchy with regression assertions. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
✅ Ponytail Reviewer completed successfully!
|
|
🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅
|
|
✅ Test Quality Sentinel completed test quality analysis. Test Quality SentinelOutcome: The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
|
✅ PR Code Quality Reviewer completed the code quality review.
|
|
✅ Design Decision Gate 🏗️ completed the design decision gate check. See the comment below for the result and any generated ADR draft. Design Decision Gate 🏗️Outcome: The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
There was a problem hiding this comment.
L82-90: yagni: full GitHub Report AST constructed only to re-extract a title and literal ADF description. Call normalizeReportTitle/raw text normalization directly before requiredString; Jira never renders or consumes report nodes.
net: -7 lines possible.
Generated by ✂️ Ponytail Reviewer for #69756 · codex · gpt56 · 22.8 AIC · ⌖ 5.35 AIC · ⊞ 14.6K
Comment /ponytail to run again
There was a problem hiding this comment.
Skills-Based Review 🧠
Applied /improve-codebase-architecture and /codebase-design (pr-triage: architecture_change). This is a well-executed deepening refactor — one minor cleanup item noted inline, otherwise approving.
📋 Key Themes & Highlights
Key Themes
- Dead code after refactor:
generateFailureMatchMarkerinhandle_agent_failure.cjsis superseded bybuildFailureMatchMetadata+ the newbuildFailureReportFooterboundary and is no longer called, exported, or tested. Flagged inline for removal.
Positive Highlights
- ✅ Deep module design:
report_ast.cjsexposes a narrow surface (normalizeReport,renderReport, a few helpers) while internally handling escaping, HTTPS validation, depth limits, and redaction — exactly the "simple interface, rich behavior" shape/improve-codebase-architecturelooks for. - ✅ Single publication boundary:
issue_report.cjsconsolidates GitHub REST/GraphQL/Linear/Jira issue creation throughrenderIssueParameters/createReportIssue. Verified no production creator callsclient.rest.issues.createdirectly outside this boundary (only work-queue test-check fixtures do). - ✅ Explicit opt-in metadata parsing:
parseReportMarkdown'sallowMetadataflag cleanly separates trusted footer/import parsing from diagnostic-fragment parsing, preventing diagnostic text from forging XML/JSON markers — a solid answer to the injection risk this refactor set out to close. - ✅ Consistent vocabulary: new AST node names (
xml_marker,json_marker,closing_reference,structured_data) are used uniformly acrossreport_ast.cjs,report_markdown.cjs, andreport_footer.cjs, and are documented clearly in the updated README — satisfies/grill-with-docsconcerns about new concepts needing shared vocabulary. - ✅ Strong test coverage:
report_ast.test.cjsandissue_report.test.cjscover normalization idempotency, escaping/redaction, structured-data edge cases (getters, sparse arrays, cycles), and the publication-boundary guarantee.
Scope note
The pre-fetched diff (3000 lines) truncated partway through report_markdown.cjs; I read the full file directly from disk to confirm behavior, so no gaps in this review.
🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · copilot · sonnet50 · 95.5 AIC · ⌖ 17.1 AIC · ⊞ 10.1K
Comment /matt to run again
Reuse shared report title and text normalization directly instead of constructing temporary report nodes. Preserve literal ADF behavior and add staged/live normalization, footer, and provider-limit regressions. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
In response to the Test Quality Sentinel result: |
|
In response to the PR Code Quality Reviewer notice: this is an in-progress notification without a substantive code-quality finding, so no code change is needed. It is not being treated as review completion or approval. |
|
In response to the generated previews: all three examples already show the H3 report sections, H4 diagnostics, and retained metadata. This informational preview requests no further code change. |
|
In response to the Matt Pocock Skills Reviewer notice: this is an in-progress notification without an actionable finding, so no code change is needed. It is not being treated as a completed review or approval. |
|
In response to the Ponytail Reviewer completion notice: the substantive Jira simplification is addressed in its inline review thread. This completion notice itself requests no additional change. |
|
In response to the Design Decision Gate result: the notification contains no ADR draft or requested design change, so no change is needed. |
There was a problem hiding this comment.
Verdict
No merge-blocking issue in the changed lines stood up to static review.
Notes
I reviewed the prefetched diff plus the key new report AST and markdown importer paths. I did not run code or tests in this review pass, and I did not find a changed-line correctness or security regression with enough confidence to block merge.
🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt54 · 384.4 AIC · ⌖ 5.74 AIC · ⊞ 12.7K
Comment /review to run again
Represent imported Markdown soft breaks explicitly while keeping scalar metadata normalized. Restore the original multiline issue-body assertion and cover deterministic rendering and scalar injection boundaries. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved security and content-compatibility regressions remain in the shared publication path.
11 open issues
Reapply URL policy after repeated entity decoding · New Avoid escaping hyphens in JSON numeric syntax · New Redact complete fragments before reference tokenization · New Preserve literal Unicode in Jira fields · New Preserve Markdown hard breaks during whitespace normalization · New Preserve ordinary double hyphens in workflow IDs · New Require left boundaries when escaping mentions · New Preserve unknown placeholders in custom failure footers · New Import Markdown images as validated image nodes · New Parse balanced parentheses in Markdown link destinations · New Import bare HTTPS URLs as validated links · New
What changed in this PR
Centralizes gh-aw issue publication around a structured report model and makes workflow failures easier to act on.
Changes:
- Adds report nodes, normalization, Markdown importing, and rendering.
- Restructures failure reports with consistent titles, diagnostics, and footers.
- Migrates issue publishers and adds compatibility coverage.
| File | Description |
|---|---|
actions/setup/README.md |
Documents structured reporting. |
actions/setup/md/agent_failure_issue.md |
Reorganizes failure-report sections. |
actions/setup/js/work_queue_issues_checks.cjs |
Updates creation fixtures. |
actions/setup/js/work_queue_issues_acceptance_checks.cjs |
Updates retry fixtures. |
actions/setup/js/work_queue_issue_api.cjs |
Normalizes GraphQL issue creation. |
actions/setup/js/sanitize_content.cjs |
Exports mention-alias expansion. |
actions/setup/js/run_validate_workflows.cjs |
Uses shared issue publication. |
actions/setup/js/run_operation_update_upgrade.cjs |
Migrates upgrade notifications. |
actions/setup/js/run_activity_report.cjs |
Migrates activity reports. |
actions/setup/js/report_markdown.cjs |
Imports supported Markdown into nodes. |
actions/setup/js/report_footer.test.cjs |
Tests footer compatibility. |
actions/setup/js/report_footer.cjs |
Builds structured failure footers. |
actions/setup/js/report_failed_jobs.test.cjs |
Updates fence assertions. |
actions/setup/js/report_failed_jobs.cjs |
Migrates failed-job reports. |
actions/setup/js/report_ast.test.cjs |
Tests normalization and rendering. |
actions/setup/js/report_ast.cjs |
Defines and renders report nodes. |
actions/setup/js/push_to_pull_request_branch.cjs |
Migrates fallback issues. |
actions/setup/js/missing_issue_helpers.cjs |
Migrates missing-information issues. |
actions/setup/js/messages_footer.cjs |
Exposes footer templates and metadata. |
actions/setup/js/linear_safe_outputs.test.cjs |
Tests normalized Linear content. |
actions/setup/js/linear_create_issue.cjs |
Renders structured Linear descriptions. |
actions/setup/js/jira_handlers.test.cjs |
Tests normalized Jira content. |
actions/setup/js/jira_handlers.cjs |
Normalizes literal Jira fields. |
actions/setup/js/issue_report.test.cjs |
Tests publication and migration coverage. |
actions/setup/js/issue_report.cjs |
Adds shared publication boundary. |
actions/setup/js/index.test.cjs |
Tests staged/live output parity. |
actions/setup/js/handle_noop_message.cjs |
Migrates tracking-issue creation. |
actions/setup/js/handle_detection_runs.cjs |
Migrates detection tracking issues. |
actions/setup/js/handle_create_pr_error.cjs |
Migrates error-report issues. |
actions/setup/js/handle_agent_failure.test.cjs |
Updates failure-report expectations. |
actions/setup/js/handle_agent_failure.cjs |
Builds structured failure reports. |
actions/setup/js/failure_report.test.cjs |
Tests report layout and escaping. |
actions/setup/js/failure_report.cjs |
Builds reports from failure templates. |
actions/setup/js/create_steering_issue.cjs |
Migrates steering issues. |
actions/setup/js/create_pull_request.test.cjs |
Updates rendered-body assertions. |
actions/setup/js/create_pull_request.cjs |
Migrates PR fallback issues. |
actions/setup/js/create_issue.test.cjs |
Tests rendering and mention compatibility. |
actions/setup/js/create_issue.cjs |
Integrates structured issue publication. |
actions/setup/js/create_forecast_issue.cjs |
Migrates forecast issues. |
actions/setup/js/check_workflow_recompile_needed.cjs |
Migrates recompile notifications. |
actions/setup/js/check_version_updates.cjs |
Migrates version-block notifications. |
.github/scripts/test_aw_issue_clustering_publish.cjs |
Updates clustering publication coverage. |
Review effort: Max
Canonicalize content before URL policy and redact complete fragments before tokenization. Preserve JSON numbers, multilingual Jira text, hard breaks, workflow identifiers, images, complete URLs, and custom footer placeholders. Remove the obsolete failure-marker helper and add regression coverage. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
🎉 This pull request is included in a new release. Release: |


Why
Automated
[aw]failure issues buried actionable causes beneath diagnostics and repeated debugging instructions. Issue creators also assembled Markdown independently, making structure, escaping, and hidden metadata inconsistent.Approach
[aw] Workflow: reasontitles and What happened, Next step, and Failure details sections. Keep classified causes visible, collapse secondary runtime evidence, and provide one copyable debugging prompt.Compatibility
The Markdown importer intentionally supports the runtime template dialect rather than all CommonMark syntax. Unsupported HTML is rendered literally. Existing authentication, retry behavior, mention budgets, and deliberately escaped closing references are retained. This changes future generated reports; existing issues are not rewritten.
Validation
TMPDIR=/tmpand a longer timeout for macOS.make agent-report-progresspassed.Full generated-body previews for the original missing-tools, unknown-cause, and missing-model-pricing failures are provided in a separate PR comment.