From b17215d7fe25a1be93822d062fd85ad34130c017 Mon Sep 17 00:00:00 2001 From: Kyle Sexton <153232337+kyle-sexton@users.noreply.github.com> Date: Wed, 4 Nov 2026 09:00:50 -0500 Subject: [PATCH 1/2] docs(skill-quality): record what the renaming control shows about description cuts Co-Authored-By: Claude Opus 5.5 --- ...2491-description-control-record-a7c41e93.md | 3 +++ .../reference/invocation-probes.md | 18 ++++++++++++++++-- 2 files changed, 19 insertions(+), 2 deletions(-) create mode 100644 .changes/skill-quality/docs-12491-description-control-record-a7c41e93.md diff --git a/.changes/skill-quality/docs-12491-description-control-record-a7c41e93.md b/.changes/skill-quality/docs-12491-description-control-record-a7c41e93.md new file mode 100644 index 00000000000..8f4f63108bd --- /dev/null +++ b/.changes/skill-quality/docs-12491-description-control-record-a7c41e93.md @@ -0,0 +1,3 @@ +--- +bump: none +--- diff --git a/plugins/skill-quality/reference/invocation-probes.md b/plugins/skill-quality/reference/invocation-probes.md index 04432675e1e..d5e1511ff6b 100644 --- a/plugins/skill-quality/reference/invocation-probes.md +++ b/plugins/skill-quality/reference/invocation-probes.md @@ -164,8 +164,22 @@ Opus 5.5, 3 runs per probe, all passes VALID, every rival on the init line) four | `architecture:map-flow` | debugging, discovery, visualization | 18/18 / 18/18 | | `architecture:map-deployment` | discovery, visualization | 18/18 / 18/18 | -All four are unmeasurable, so they keep the before text. What makes the control fire was not -tested: the listing always keeps a skill's name, which the generic description leaves in place. +All four are unmeasurable, so they keep the before text. + +The name is what keeps such a control firing, at least on Opus. On the same version and model, +three `code-metrics` leaves ran the description-only control, then a control that also renamed the +skill's folder (each row 27 runs per arm, 9 should-trigger probes at 3 runs, all passes VALID): + +| Skill | Should-trigger fired, shipped / description control / name and description control | +|---|---| +| `code-metrics:audit-coverage` | 27/27 / 27/27 / 0/27 | +| `code-metrics:audit-duplication` | 27/27 / 27/27 / 0/27 | +| `code-metrics:audit-suppressions` | 27/27 / 27/27 / 0/27 | + +A renaming control shows that a probe set can fail, and nothing about the description: it removes +the skill from view, so it falls all or nothing and never certifies a cut. These three are +unmeasurable and keep the before text. `naming:name-it-better` above shows the description control +can still drop, on Sonnet 5.5. ## Outside the marketplace checkout From da1ce77d29235840cc4f6351bb00adcb91def152 Mon Sep 17 00:00:00 2001 From: Kyle Sexton <153232337+kyle-sexton@users.noreply.github.com> Date: Wed, 4 Nov 2026 09:03:05 -0500 Subject: [PATCH 2/2] docs(skill-quality): say why the control record needs no release Co-Authored-By: Claude Opus 5.5 --- .../docs-12491-description-control-record-a7c41e93.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.changes/skill-quality/docs-12491-description-control-record-a7c41e93.md b/.changes/skill-quality/docs-12491-description-control-record-a7c41e93.md index 8f4f63108bd..e9a2abb6840 100644 --- a/.changes/skill-quality/docs-12491-description-control-record-a7c41e93.md +++ b/.changes/skill-quality/docs-12491-description-control-record-a7c41e93.md @@ -1,3 +1,5 @@ --- bump: none --- + +No release: this adds recorded runs to a reference file and changes no behavior.