From 4e1882f2973cf12c872a4f282c36acc5c68cbc92 Mon Sep 17 00:00:00 2001 From: Sarah Soutoul Date: Wed, 14 Oct 2026 14:10:18 -0600 Subject: [PATCH 1/3] Point CLI audit guidance to canonical skills repo --- .agents/skills/audit-clerk-skill | 1 - .claude/skills/audit-clerk-skill/SKILL.md | 152 ---------------------- AGENTS.md | 2 + 3 files changed, 2 insertions(+), 153 deletions(-) delete mode 120000 .agents/skills/audit-clerk-skill delete mode 100644 .claude/skills/audit-clerk-skill/SKILL.md diff --git a/.agents/skills/audit-clerk-skill b/.agents/skills/audit-clerk-skill deleted file mode 120000 index 5a0f59352..000000000 --- a/.agents/skills/audit-clerk-skill +++ /dev/null @@ -1 +0,0 @@ -../../.claude/skills/audit-clerk-skill \ No newline at end of file diff --git a/.claude/skills/audit-clerk-skill/SKILL.md b/.claude/skills/audit-clerk-skill/SKILL.md deleted file mode 100644 index 116f43673..000000000 --- a/.claude/skills/audit-clerk-skill/SKILL.md +++ /dev/null @@ -1,152 +0,0 @@ ---- -name: audit-clerk-skill -description: Audits the Clerk CLI source tree and proposes updates to the `clerk-cli` skill in clerk/skills so it stays in sync with the binary. Use when the user says "audit the clerk-cli skill", "update the clerk-cli skill", "check the skill against the code", "resync clerk-cli skill", or after adding/renaming/removing CLI commands, flags, or agent-mode behavior. -effort: high -user-invocable: true -disable-model-invocation: true -argument-hint: "[--apply]" -metadata: - internal: true ---- - -# Audit the clerk-cli Skill - -Cross-check the `clerk-cli` skill against the actual CLI source in `packages/cli-core/` and propose precise edits wherever they have drifted. The binary is the source of truth; the skill is documentation that must track it. - -**The skill lives in another repo.** #315 removed the bundled copy, so it is now `skills/core/clerk-cli/` in [clerk/skills](https://github.com/clerk/skills), versioned independently of the CLI. This audit reads the CLI here and proposes edits there; they land as a PR against clerk/skills. Set `$SKILL_ROOT` to a local clone before starting, and stop if there is not one: - -```sh -SKILL_ROOT=../skills/skills/core/clerk-cli # adjust to the checkout -ls "$SKILL_ROOT/SKILL.md" || echo "clone clerk/skills first" -``` - -Every skill path below is relative to `$SKILL_ROOT`. - -**ultrathink** on this task. It requires building a full command tree, comparing two representations of it, and making judgment calls about what belongs in the skill vs. in `references/*.md` vs. in `--help`. Shallow passes miss drift. - -## Inputs - -- **Source of truth**: `packages/cli-core/src/commands/**` (one directory per top-level command), plus `packages/cli-core/src/cli.ts`, `cli-program.ts`, `mode.ts`, and anything in `packages/cli-core/src/lib/` referenced by commands (runner preference, agent mode, doctor checks, key resolution). -- **Target**: `$SKILL_ROOT/SKILL.md` and `$SKILL_ROOT/references/*.md`. - -## Workflow - -### 1. Inventory the CLI - -Walk `packages/cli-core/src/commands/` and build a structured inventory. For each top-level command and subcommand capture: - -- Full path (e.g. `clerk config patch`, `clerk api ls`). -- Purpose (one line, from the command's description / help string in source). -- All flags with short form, type, default, and whether they are destructive. -- Exit codes beyond the default if the command overrides them. -- Agent-mode branches: anything gated on `isAgentMode()` / `CLERK_MODE` / TTY detection. Note what changes (prompts skipped, JSON forced, `--fix` ignored, handoff printed, etc.). -- Whether the command mutates remote state (candidate for `--dry-run` / `--yes` guidance). - -Read the per-command `README.md` (`packages/cli-core/src/commands//README.md`) as a **secondary** source. These are dev-facing (documented API endpoints hit, mocked surfaces, internal contracts per `.claude/rules/commands.md`). Use them to: - -- Flag commands or sub-paths marked mocked/stubbed (blockquote at top of the README). The skill should not document these as production-ready. -- Cross-check Clerk API endpoint claims in `references/recipes.md`. - -Do **not** propose bundling the READMEs into `$SKILL_ROOT/references/` (symlinks or text imports). They ship internal detail agents do not need, and inflate the compiled binary. They are a reference for the audit, not for the skill. - -Also capture cross-cutting behavior: - -- Runner preference logic (`preferredRunner` in `lib/`): the lockfile -> runner mapping that the skill's Invoking-the-CLI table must mirror. -- Auth / key resolution order and `--app` / `--instance` resolution (cross-ref `references/auth.md`). -- `doctor` check list and the shape of `--json` output (cross-ref `references/agent-mode.md`). -- `clerk init --prompt` output (the skill claims it is an agent handoff, not an integration guide; verify). - -Don't memorize output. Prefer reading the source directly over running the binary, since the binary in the worktree may be stale or unbuilt. When uncertain about runtime behavior, fall back to the tests in `packages/cli-core/src/**/*.test.ts` or `packages/cli-core/src/test/`. - -### 2. Extract the skill's current claims - -Read `$SKILL_ROOT/SKILL.md` and each file under `$SKILL_ROOT/references/`. Extract every concrete claim: - -- Every command mentioned in the "Core commands at a glance" table and the Invoking-the-CLI table. -- Every flag called out by name. -- Every agent-mode behavior bullet. -- Every exit code / error-format claim. -- Every cross-reference to a `references/*.md` file. - -### 3. Diff source vs. skill - -Produce a structured diff with four buckets: - -1. **Missing from skill**: commands, subcommands, flags, or behaviors that exist in the source but are not mentioned anywhere in the skill. (Example seen historically: `completion`, `deploy`, `open`, `skill`, `switch-env` were absent from the core-commands table.) -2. **Stale in skill**: claims in the skill that no longer match source - renamed flags, changed defaults, removed commands, shifted exit codes, agent-mode branches that were reworked. -3. **Thin in skill**: commands mentioned but under-specified relative to their real complexity or footgun surface (destructive mutations without `--dry-run` callouts, flags with non-obvious interactions, agent-mode divergence not documented). -4. **Over-specified**: details the skill encodes that `clerk --help` or the referenced files already cover better. Candidates for deletion to stay under the 500-line budget. - -For every entry in buckets 1-3, cite the source file and line and the target skill location. - -### 4. Decide placement - -Not every gap belongs in `SKILL.md`. Apply this routing: - -- **Core loop / mental model / safety**: stays in `SKILL.md`. -- **Per-command flag tables, recipes, edge cases**: belong in `references/recipes.md` or a new reference file. -- **Agent-mode branches**: belong in `references/agent-mode.md`; `SKILL.md` gets a summary bullet only. -- **Auth / targeting / key resolution**: `references/auth.md`. - -**Prefer `--help` over enumeration.** `clerk --help` is generated from the Commander tree and can't drift; it also renders each command's `setExamples([...])` block. Before expanding a flag table, ask whether the agent could just run `--help` once. Route as follows: - -- Flags with non-obvious interactions, destructive semantics, or agent-mode divergence: document in the skill. -- Flags that are self-explanatory from `--help`: do not enumerate. If the skill currently lists them and there's no hidden gotcha, propose a `polish` edit that removes the enumeration and points at `--help`. -- Example syntax: prefer a sentence like "see `clerk --help` for examples" over copying examples into the skill (which risks drifting from the source `setExamples([...])`). - -Cuts that shrink the skill toward `--help` are first-class proposals, not optional cleanup. A smaller skill that routes agents to `--help` ages better than a larger one that tries to mirror every flag. - -If a new reference file is warranted (e.g. a `references/commands.md` table), propose it explicitly rather than bloating `SKILL.md`. - -### 5. Propose edits - -Emit a review-ready proposal. For each change: - -- Path (`$SKILL_ROOT/SKILL.md` or `$SKILL_ROOT/references/.md`). -- Why (source citation: `packages/cli-core/src/commands//.ts:`). -- Either a unified diff (preferred) or a before/after block for prose sections. -- Severity: `drift` (factually wrong today), `gap` (missing coverage), `polish` (clearer wording, better placement, or cuts that route the agent to `--help` instead of duplicating it). - -Group the proposal by file. Do **not** rewrite sections that are still accurate just because they are near a change. - -### 6. Apply or hand back - -Default: present the proposal and stop. The user reviews, then says apply. - -If invoked as `/audit-clerk-skill --apply`, apply `drift` and `gap` edits directly to `$SKILL_ROOT` but still list `polish` suggestions for review. Edits land in the clerk/skills clone, not this repo, so commit them on a branch there and open a PR against clerk/skills. Nothing in this repo changes. - -## Guardrails - -- **Never invent flags.** If a flag appears in a test but not in the command's argument parser, treat it as test-only and flag it for human review. -- **Preserve voice.** The existing skill is terse and third-person; match it. No first- or second-person drift. -- **No em-dashes** anywhere in proposals (repo style rule). -- **Respect the template.** The Invoking-the-CLI runner table is generated from `preferredRunner` logic; if that logic changes, update the table, otherwise leave it alone. -- **Stay within the 500-line guidance** for `SKILL.md`. When the budget is tight, move content to `references/` rather than deleting it outright. - -## Output shape - -Return the proposal as: - -``` -# clerk-cli skill audit — - -## Summary - - -## $SKILL_ROOT/SKILL.md -###
-- [drift|gap|polish] - - source: packages/cli-core/src/commands/<...>: - - - -## $SKILL_ROOT/references/.md -... - -## New files (if any) -... - -## Open questions - -``` - -Keep it skimmable. The user should be able to approve or reject each entry individually. diff --git a/AGENTS.md b/AGENTS.md index 5a38ae8ca..7995f77ca 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -15,6 +15,8 @@ This is a Bun workspace monorepo: See [docs/releasing.md](docs/releasing.md) for the full release flow, channels, and safeguards. See [CONTRIBUTING.md](CONTRIBUTING.md) for development setup, pre-release install methods, and PR guidelines. +The canonical `audit-clerk-skill` maintainer instructions live in [clerk/skills](https://github.com/clerk/skills/blob/main/.agents/skills/audit-clerk-skill/SKILL.md). Use that skill when auditing the CLI source against `skills/clerk-cli/`; do not maintain another copy here. + ## Bun Default to using Bun instead of Node.js. From a00e3770af4f3c9d9b01d1018743379ce6c555b3 Mon Sep 17 00:00:00 2001 From: Sarah Soutoul Date: Fri, 16 Oct 2026 11:33:35 -0600 Subject: [PATCH 2/3] Keep CLI audit skill as pointer to canonical guidance --- .agents/skills/audit-clerk-skill | 1 + .claude/skills/audit-clerk-skill/SKILL.md | 16 ++++++++++++++++ 2 files changed, 17 insertions(+) create mode 120000 .agents/skills/audit-clerk-skill create mode 100644 .claude/skills/audit-clerk-skill/SKILL.md diff --git a/.agents/skills/audit-clerk-skill b/.agents/skills/audit-clerk-skill new file mode 120000 index 000000000..5a0f59352 --- /dev/null +++ b/.agents/skills/audit-clerk-skill @@ -0,0 +1 @@ +../../.claude/skills/audit-clerk-skill \ No newline at end of file diff --git a/.claude/skills/audit-clerk-skill/SKILL.md b/.claude/skills/audit-clerk-skill/SKILL.md new file mode 100644 index 000000000..5493feae2 --- /dev/null +++ b/.claude/skills/audit-clerk-skill/SKILL.md @@ -0,0 +1,16 @@ +--- +name: audit-clerk-skill +description: Audit the Clerk CLI source against the canonical clerk-cli skill in clerk/skills. Use when asked to audit or update the clerk-cli skill after CLI commands, flags, or agent-mode behavior change. +effort: high +user-invocable: true +disable-model-invocation: true +argument-hint: "[--apply]" +metadata: + internal: true +--- + +# Audit the clerk-cli Skill + +The audit instructions are maintained in [clerk/skills](https://github.com/clerk/skills/blob/main/.agents/skills/audit-clerk-skill/SKILL.md). Read that entire skill before starting; do not use this pointer as a substitute for its workflow. + +Run the canonical audit against this CLI checkout as the source (its `--source ` option) and `skills/clerk-cli/` in a clerk/skills checkout as the target. Do not look for the target skill in this repository. If you cannot access the canonical instructions, stop and ask for access instead of relying on a stale copy. From aa8891b74882f0cad6efdf0ce66338b5141f6e3f Mon Sep 17 00:00:00 2001 From: Sarah Soutoul Date: Fri, 16 Oct 2026 11:48:53 -0600 Subject: [PATCH 3/3] Resolve skills checkout before CLI skill audit --- .claude/skills/audit-clerk-skill/SKILL.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/.claude/skills/audit-clerk-skill/SKILL.md b/.claude/skills/audit-clerk-skill/SKILL.md index 5493feae2..415ba7b62 100644 --- a/.claude/skills/audit-clerk-skill/SKILL.md +++ b/.claude/skills/audit-clerk-skill/SKILL.md @@ -13,4 +13,6 @@ metadata: The audit instructions are maintained in [clerk/skills](https://github.com/clerk/skills/blob/main/.agents/skills/audit-clerk-skill/SKILL.md). Read that entire skill before starting; do not use this pointer as a substitute for its workflow. -Run the canonical audit against this CLI checkout as the source (its `--source ` option) and `skills/clerk-cli/` in a clerk/skills checkout as the target. Do not look for the target skill in this repository. If you cannot access the canonical instructions, stop and ask for access instead of relying on a stale copy. +Find a local `clerk/skills` checkout containing `skills/clerk-cli/SKILL.md`; check sibling directories of this CLI checkout first. If none exists and network access is available, clone `https://github.com/clerk/skills.git` into a temporary directory. Run the canonical audit from that checkout, passing this CLI checkout's path with `--source`. Do not look for the target skill in this repository. If you cannot access `clerk/skills`, stop and ask for a checkout instead of relying on a stale copy. + +If using `--apply`, keep the skills checkout until its edits are committed or handed off.