perf: cut the bodies of the most-invoked skills through progressive disclosure - part 1 of 2 #12243

Activity

1. melodic-automation-lanes

   added the

   Task

   issue type on Nov 2, 2026
2. melodic-automation-lanes

   added a parent issue on Nov 2, 2026

   • perf: cut the bodies of the most-invoked skills through progressive disclosure #12240
3. melodic-automation-lanes

   marked this as blocking on Nov 2, 2026

   • perf: cut the bodies of the most-invoked skills through progressive disclosure - part 2 of 2 #12244
4. github-actions

   added

   needs-triageNot yet classified. Floor until a type and one priority tier are set.Not yet classified. Floor until a type and one priority tier are set.

   on Nov 2, 2026
5. github-actions commented on Nov 2, 2026

   

   github-actionsBot

   on Nov 2, 2026 – with GitHub Actions

   Last edited by melodic-automation-lanes

   Contributor

   More actions

   Proposed work class: work-class: scoped
   Basis: automated intake triage; the lanes App applied this class.

   Size: Medium
   Complexity: Medium
   Uncertainty: Knowable
   Flags: none
   Blocked by: none

   Understood: this is part 1 of 2 of #12240, split at intake. It asks for the bodies of the most-invoked skills to be cut through progressive disclosure. Action-specific detail would move into reference files read on demand, and the hub would keep routing and gates. Each cut needs eval evidence of no loss.

   Two things are missing from this issue. It says "the six skills" without naming them, and it doesn't say which skills belong to part 1 and which to part 2. The parent issue #12240 or the sibling #12244 may settle both. Neither #12240 nor #12244 is a duplicate; they are the parent and the sibling split.

   _{Automated triage, applied by the lanes App.}
6. github-actions

   added

   priority: mediumReal value, no hard deadline; normal backlog flow.Real value, no hard deadline; normal backlog flow.

   and removed

   needs-triageNot yet classified. Floor until a type and one priority tier are set.Not yet classified. Floor until a type and one priority tier are set.

   on Nov 2, 2026
7. melodic-automation-lanes

   added

   status: readyTriaged, unblocked, and fully specified; eligible to pick up.Triaged, unblocked, and fully specified; eligible to pick up.

   work-class: scopedA briefed fix or small feature; blast radius bounded by the brief, tests exist.A briefed fix or small feature; blast radius bounded by the brief, tests exist.

   on Nov 2, 2026
8. melodic-automation-lanes

   set

   Work status

   to

   ready

   on Nov 2, 2026
9. melodic-automation-lanes

   added

   admission-deferredAdmitted but waiting for WIP room; the hourly work-unblock sweep labels it agent-ready.Admitted but waiting for WIP room; the hourly work-unblock sweep labels it agent-ready.

   agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.Fully specified and briefed; eligible for autonomous pickup from the frontier.

   and removed

   admission-deferredAdmitted but waiting for WIP room; the hourly work-unblock sweep labels it agent-ready.Admitted but waiting for WIP room; the hourly work-unblock sweep labels it agent-ready.

   on Nov 2, 2026
10. melodic-automation-lanes

    changed

    Work status

    to

    agent-ready

    on Nov 2, 2026
11. github-actions commented on Nov 2, 2026

    

    github-actionsBot

    on Nov 2, 2026 – with GitHub Actions

    Last edited by github-actions

    Contributor

    More actions

    No description provided.
12. 1 remaining item

    Load more
13. melodic-automation-lanes

    added

    host-runNeeds an unattended run on a host; the host runner takes it, not a person or the issue lane.Needs an unattended run on a host; the host runner takes it, not a person or the issue lane.

    and removed

    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.Fully specified and briefed; eligible for autonomous pickup from the frontier.

    on Nov 2, 2026
14. melodic-automation-lanes commented on Nov 2, 2026

    

    melodic-automation-lanesBot

    on Nov 2, 2026

    ContributorAuthor

    More actions

    The work-resolve-issue lane stopped on this issue (host-only): the run reports the work needs a host run the lane cannot make.
    It added host-run and removed agent-ready: an agent run on a host builds it, and it waits on no decision from a person.

    In the run's words:

    host-only: each of the six body cuts must ship with eval evidence of no loss (docs/review/skills-and-agents.md:6-8; plugins/playbooks/skills/skill-authoring/reference/skill-criteria.md:24-28 and 278-285: run the skill's cases with and without the removed text via /evals:plugin-eval --baseline <before.json> --margin <m>, and ship only under the non-inferiority rule). /evals:plugin-eval runs billed claude plugin eval sessions under a sandbox host check (plugins/evals/skills/plugin-eval/SKILL.md), and this lane's Bash runs only the named wrappers, none of which runs an eval. Without that evidence a cut cannot ship, so no edit was made.

    What the run says it did not do:

    • Progressive-disclosure cuts of the six skill bodies: not started, because the required no-loss eval evidence (/evals:plugin-eval --baseline per skill) needs a host run this lane cannot make.
    • Tidy, review and prep steps: skipped because there is no change to tidy or review.

    Run: https://github.com/melodic-software/claude-code-plugins/actions/runs/41383383772

    To hand it back to the lane, remove host-run and add agent-ready again.
15. kyle-sexton commented on Nov 2, 2026

    

    kyle-sexton

    on Nov 2, 2026

    Member

    More actions

    Claim. Backlog HITL host-run worker is taking this now.
16. kyle-sexton

    added a commit that references this issue on Nov 2, 2026

    fix(source-control): point the last owner citations at the nesting re…

    ...

    68eba76
17. kyle-sexton commented on Nov 2, 2026

    

    kyle-sexton

    on Nov 2, 2026

    Member

    More actions

    Host run. I cut the /source-control:worktree skill body through progressive disclosure and showed no loss on an eval round that is VALID on both arms. This is cut 1 of the 6.

    What ran

    • Claude Code 2.1.316, on the subscription login.
    • Scope: one skill. Each cut needs its own paired eval run, and this lane runs at most two suites at once.
    • The cut:
      • The section "The nesting invariant, dated measurement" moves verbatim to skills/worktree/reference/nesting-invariant.md. The SSOT test, hooks, scripts, context/create.md, the fixtures and the README now point at the new owner.
      • The quoted isolation-enforcement passage becomes our rules plus a docs pointer, with an as-of date and a recheck trigger.
      • Body bytes with frontmatter stripped, as perf: cut the bodies of the most-invoked skills through progressive disclosure #12240 measures them: 38,158 before, 31,615 after.
    • /docs-hygiene:compress compare (fresh compare-labeler agent): VERDICT PASS, with no SEMANTIC LOSS. It raised two AMBIGUITY rows, and both were fixed before the evals ran.
    • Evals: 3 rounds. Each round ran the before and after copies at the same time with --case 'worktree-*' --ablation none --runs 2 -j 2 --scaffold, on Opus 5.5, in capped scopes with a short TMPDIR. The ~/.docker links were the same before and after.
      • Validity: main's run-validity.py --runs 2 --scaffolded (with fix(evals): check case helper plugins per case and excuse plugin-data compound denials #12388) marks round 1 VALID on both arms.
      • Round 1 result: 8/8 cases at 1.000 in both arms. The noise report gives a paired change of +0.00 (interval +0.00 to +0.00), and its verdict is non-inferior at margin 0.05.
      • Rounds 2 and 3 each stayed INVALID on one arm: a with-arm Read of the sandbox settings.json was denied, and round 3 also had one unfired Skill call. Pooled over all 3 rounds, mean case score is 0.984 before and 0.979 after.
    • Ready phase:
      • origin/main merged into the branch.
      • Verify: check-affected.sh --tests passed on the merged head (14 passed, 2 skipped). pre_ready disables this step, but it was run anyway.
      • The code and security reviews were left to CI, because both lanes read ci healthy.
      • Holds were checked twice and none was found. The PR was flipped to ready, and a ready CI run was confirmed.
    • Review round 1 on the ready head:
      • Security lane: 0 findings.
      • Code lane: 2 findings, both VALID and fixed in the final head. Three owner citations still named SKILL.md. The reference file quoted the hooks docs and restated an upstream issue's state.
      • Both replies are posted, and the bot thread is resolved.
      • The fix touches no eval-read text, so the round-1 no-loss result still holds for the hub.
      • Final head: 68eba768ab. Lanes on that head were still pending at handoff to PR Lane Automation.

    Acceptance criterion (#12243)

    • Each of the six skills cut, with eval evidence of no loss: PARTIAL, 1 of 6.
      • Done: /source-control:worktree, in PR perf(source-control): move the worktree nesting measurement out of the skill hub #12387.
      • Not started: /source-control:pull-request, /session-flow:handoff, /discovery:research, /evals:plugin-eval, /source-control:commit.

    PR: #12387

    Target labels: add none; remove none.
18. kyle-sexton commented on Nov 3, 2026

    

    kyle-sexton

    on Nov 3, 2026

    Member

    More actions

    Claim. Backlog HITL host-run worker is taking this now.
19. kyle-sexton commented on Nov 3, 2026

    

    kyle-sexton

    on Nov 3, 2026

    Member

    More actions

    Slice plan. The five remaining cuts are filed as one issue per skill.

    Slice	Skill	Body bytes	Cases	Note
    done	/source-control:worktree	31,615 after	worktree-*	#12387, merged
    #12655	/source-control:pull-request	50,149	pull-request-* (19)	largest load: 161 invocations in #12240's window
    #12657	/session-flow:handoff	36,785	ho-* (4)
    #12659	/discovery:research	44,641	1 (rs-*)	blocked by #12647, which edits the same SKILL.md; a contract test pins a byte window in it (#12631); needs cases added first
    #12661	/evals:plugin-eval	41,783	pe-* and others naming it
    #12666	/source-control:commit	28,967	commit-* (6)	smallest

    Basis:

    • Body bytes are SKILL.md with frontmatter stripped at origin/main 3bd42cabfe, the method perf(source-control): move the worktree nesting measurement out of the skill hub #12387 used (it reproduces perf(source-control): move the worktree nesting measurement out of the skill hub #12387's 31,615 for worktree).
    • perf: cut the bodies of the most-invoked skills through progressive disclosure #12240's table gives 32,568 for pull-request, which this method does not reproduce (the file has not changed since b1439e9809), so perf: cut the bodies of the most-invoked skills through progressive disclosure - part 2 of 2 #12244 should measure before and after with one stated method.
    • One skill per PR, not one PR for all five: every directory fits a cut under the 262,144-byte cap, but each cut needs its own paired eval round VALID on both arms (skill-criteria.md, "Cutting text"), and perf(source-control): move the worktree nesting measurement out of the skill hub #12387 took about 3 h of host time for one.

    #12243 is done when the five slices merge.
20. kyle-sexton

    added sub-issues on Nov 3, 2026

    • perf(source-control): cut the pull-request skill body through progressive disclosure #12655
    • perf(session-flow): cut the handoff skill body through progressive disclosure #12657
    • perf(discovery): cut the research skill body through progressive disclosure #12659
    • perf(evals): cut the plugin-eval skill body through progressive disclosure #12661
    • perf(source-control): cut the commit skill body through progressive disclosure #12666
21. kyle-sexton

    removed

    host-runNeeds an unattended run on a host; the host runner takes it, not a person or the issue lane.Needs an unattended run on a host; the host runner takes it, not a person or the issue lane.

    on Nov 3, 2026
22. kyle-sexton commented on Nov 3, 2026

    

    kyle-sexton

    on Nov 3, 2026

    Member

    More actions

    Rule that applies to every slice here (agent note, not an owner decision). Each slice is a skill-body cut, so plugins/playbooks/skills/skill-authoring/reference/rewrite-a-plugin.md:60-66 applies: copy the before plugin, delete only the changed skill's procedure steps, run it on one model against a before run launched together, and the suite detects a loss when the noise report calls a case drop or the mean case score falls by 0.15 or more. Line 79: "An unmeasurable part (step 4's negative control) ships its before text, whatever its own pass shows." Line 22: preregister that control in the draft PR body before the first paid run. #12387 shipped without a control and scored 8/8 in both versions, so its suite has not shown it can detect a loss. It needs the control after the fact, and the cut reverts if the control cannot detect. The slices' Done-when (#12655 #12657 #12659 #12661 #12666; draft #12704) names no control, but the protocol requires one. Basis: that file at origin/main; the owner's 2026-11-02 negative-control decision, which landed through #12222. The 2026-11-02 stop on #6476 lists only #6599-#6619.