From 6c1a9bee809fac64edb35dfd4a9a72dd5c5c1341 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Wed, 9 Sep 2026 13:52:46 +0000 Subject: [PATCH] Extract PR reviewer skill-recovery detour into its own side quest - Move the 'skill not applied' recovery checklist out of Step 14b's ready-for-review testing section into a new optional side quest (side-quest-14b-01-skill-recovery.md). - Replace the extracted checklist in 14b-pr-reviewer-workflow.md with a short summary and an Optional Side Quest callout linking to the new file. - List the new side quest in workshop/README.md under Optional Side Quests, noting it branches from Step 14b. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- workshop/14b-pr-reviewer-workflow.md | 17 +++-- workshop/README.md | 1 + workshop/side-quest-14b-01-skill-recovery.md | 70 ++++++++++++++++++++ 3 files changed, 79 insertions(+), 9 deletions(-) create mode 100644 workshop/side-quest-14b-01-skill-recovery.md diff --git a/workshop/14b-pr-reviewer-workflow.md b/workshop/14b-pr-reviewer-workflow.md index ac337478..9b59004a 100644 --- a/workshop/14b-pr-reviewer-workflow.md +++ b/workshop/14b-pr-reviewer-workflow.md @@ -137,16 +137,15 @@ To test the manual path, add a `/review` comment to the pull request. After push > [!NOTE] > If no run starts, confirm that the workflow is on your default branch and that you changed the pull request from draft to ready. Opening a pull request as ready does not emit the `ready_for_review` event. -If the run completes but the review does not mention the `pr-review-standards` skill or does not cite changed files and lines, the reviewer likely could not find the skills directory. Use this checklist to recover: +If the run completes but the review does not mention the `pr-review-standards` skill or does not cite changed files and lines, the reviewer likely could not find the skills directory. -1. Confirm `.github/skills/agentic-workflows/` exists and was pushed. Run `ls .github/skills/` in your terminal. If the directory is missing, run `gh aw init`, commit the generated files, and push. -2. If the directory exists but the skill was still not applied, ask the agent to reinforce the instruction: - -```prompt -/agentic-workflows Update .github/workflows/pr-reviewer.md so the pr-reviewer agent explicitly searches for and applies the pr-review-standards skill before returning findings. -``` - -1. Compile, commit, and re-trigger `/review` to confirm the skill is now applied. +> [!TIP] +>
+> Optional Side Quest: Want the full recovery checklist for a reviewer that skipped its skill? +> +> Work through [Side Quest: Recovering When a Reviewer Skill Isn't Applied](side-quest-14b-01-skill-recovery.md), then come back here. +> +>
## Improve One Layer diff --git a/workshop/README.md b/workshop/README.md index 0d14b9a6..9d18df6e 100644 --- a/workshop/README.md +++ b/workshop/README.md @@ -68,6 +68,7 @@ A hands-on workshop that takes you from zero to a fully automated, AI-powered wo - [Frontmatter Deep Dive — Part A](side-quest-11-01-frontmatter-deep-dive.md) — walkthrough of the opening, trigger, and permissions sections of `gh-aw` frontmatter, with predict-and-try activities; branches from [Step 7](07-your-first-workflow.md). - [Workflow File Structure at a Glance](side-quest-11-01b-workflow-structure.md) — visual map of the two-file structure (`.md` + `.lock.yml`), the frontmatter sections, and the Markdown body; branches from [Step 7](07-your-first-workflow.md). - [Frontmatter Deep Dive — Part B](side-quest-11-08-frontmatter-tools-outputs.md) — walkthrough of the tools, safe-outputs, closing fence, and agent body sections, with predict-and-try activities; continues from Part A. +- [Recovering When a Reviewer Skill Isn't Applied](side-quest-14b-01-skill-recovery.md) — troubleshooting checklist for a PR reviewer run that completes but skips its `pr-review-standards` skill; branches from [Build a PR Reviewer with an Agent and Skill](14b-pr-reviewer-workflow.md). - [Pattern: Auto-Label PRs by Content](side-quest-13-01-pr-labeler-pattern.md) — apply labels automatically based on which files changed in a pull request; branches from [Step 14b](14b-pr-reviewer-workflow.md). - [Pattern: Generate a PR Summary Comment](side-quest-13-02-pr-summary-pattern.md) — post a structured, changelog-ready summary comment when a pull request opens; branches from [Step 14b](14b-pr-reviewer-workflow.md). - [Pattern: PR Review Checklist](side-quest-13-03-pr-checklist-pattern.md) — evaluate pull requests against a quality checklist and post a pass/fail table; branches from [Step 14b](14b-pr-reviewer-workflow.md). diff --git a/workshop/side-quest-14b-01-skill-recovery.md b/workshop/side-quest-14b-01-skill-recovery.md new file mode 100644 index 00000000..e7b2ab65 --- /dev/null +++ b/workshop/side-quest-14b-01-skill-recovery.md @@ -0,0 +1,70 @@ + + +# Side Quest: Recovering When a Reviewer Skill Isn't Applied + +> _Optional: work through this recovery checklist if your PR reviewer's review doesn't cite the `pr-review-standards` skill, then return to [Build a PR Reviewer with an Agent and Skill](14b-pr-reviewer-workflow.md)._ + +## :dart: What You'll Do + +You'll diagnose why the `pr-reviewer` inline agent skipped its `pr-review-standards` skill, confirm the skills directory is actually present and pushed, and reinforce the instruction so the agent reliably applies the skill on the next run. + +## :clipboard: Before You Start + +- You completed the workflow creation and testing steps in [Build a PR Reviewer with an Agent and Skill](14b-pr-reviewer-workflow.md). +- Your **PR Reviewer** run completed, but the submitted review does not mention the `pr-review-standards` skill or does not cite changed files and lines for its findings. + +## Steps + +### Recognize the symptom + +A run that completes successfully but produces a review missing evidence-based findings — no cited file, no cited line, generic feedback — is a sign the `pr-reviewer` agent never discovered or applied the `pr-review-standards` skill. This is different from a workflow that fails to trigger at all; the run itself succeeds, but the *quality* of the output is missing the skill's guidance. + +### Confirm the skills directory exists and was pushed + +The most common cause is a missing or unpushed skills directory. Confirm it locally: + +```bash +ls .github/skills/ +``` + +If `agentic-workflows/` is missing, regenerate and push it: + +```bash +gh aw init +git add . +git commit -m "Initialize agentic workflow skills" +git push +``` + +Re-trigger `/review` on your pull request and check whether the review now cites the skill's guidance. + +### Reinforce the instruction in the workflow brief + +If the directory exists but the skill still wasn't applied, the `pr-reviewer` agent's instructions may not be explicit enough about discovering and applying the skill. Ask your AI agent to strengthen that instruction: + +```prompt +/agentic-workflows Update .github/workflows/pr-reviewer.md so the pr-reviewer agent explicitly searches for and applies the pr-review-standards skill before returning findings. +``` + +Review the diff. The updated `pr-reviewer` agent brief should now explicitly instruct the agent to look for and apply the skill, rather than assuming it will discover it on its own. + +### Compile, commit, and re-verify + +```bash +gh aw compile +git add . +git commit -m "fix: reinforce skill discovery in pr-reviewer agent" +git push +``` + +Re-trigger `/review` on the same pull request (or comment `/review` again after a new commit) and confirm the new review cites evidence from the changed lines and reflects the `pr-review-standards` guidance. + +## :white_check_mark: Checkpoint + +- [ ] You identified that a completed run with a generic, non-evidence-based review indicates a skipped skill +- [ ] You confirmed `.github/skills/agentic-workflows/` exists and is pushed to your default branch +- [ ] You reinforced the skill-discovery instruction in the `pr-reviewer` agent brief +- [ ] You compiled, committed, and pushed the updated workflow +- [ ] A re-triggered `/review` produced a review that cites changed files and lines + +**Return to the main adventure:** [Build a PR Reviewer with an Agent and Skill](14b-pr-reviewer-workflow.md)