ci(docs): catch docs that drift from the CLI by ukimsanov · Pull Request #5933 · heygen-com/hyperframes

ukimsanov · 2026-10-20T06:09:37Z

What

Two CI checks so the docs can't quietly drift from the code, plus the stale facts they found.

CLI references (docs.yml, job cli-references): every hyperframes <command> --flag shown in docs/**/*.mdx must exist in the real CLI. The source of truth is the CLI's own command modules (scripts/cli-command-tree.ts: 88 commands, 668 flags), not --help text. Unknown commands or flags fail with a "did you mean"; deprecated or hidden ones warn. Quoted CLI output can opt out with {/* cli-references: ignore */}.
Docs impact (docs-impact.yml): a PR that changes CLI commands, flags or public package exports must touch docs/, or carry the docs-not-needed label, or a docs-impact: none line in the PR body.
Docs fixes found by the audit: Linux install steps lead with the .deb (what the download page hands Linux browsers by default; file names checked against the live release feed), the command finder lists models and clean, and the update-notice example shows current versions.

Why

The docs had drifted in small ways nobody noticed (Linux install, missing commands). Agents read these pages too, so a stale command becomes a failed run.

How

Check 1 scans fenced blocks (joining \ continuations), inline code and <InstallCommand> props; handles npx/bunx/env prefixes, placeholders (<x>, ..., $X) and anything after --; flag spelling follows reject-unknown-flags.ts.
Check 2 compares merge base with HEAD. The CLI side builds the base tree in a temp worktree only when packages/cli/src changed; exports are read from git with the TypeScript parser (no build). Measured on the last 149 main commits: 6 changed exports, 1 touched docs.
Separate docs-impact.yml (no path filter; reruns on edited/labeled/unlabeled), not on merge_group because a merge group has no PR body.

Test plan

27/27 new tests; bun run test:scripts 781/781 node + 109/109 vitest
typecheck:scripts, oxlint (0), oxfmt --check, check-test-reachability, fallow audit --base origin/main
Today's docs: 586 CLI invocations resolve, 0 errors, 2 intentional deprecation warnings
Replayed past PRs: c0636d0 (added snapshot --keep, touched docs) passes; d5162beb5 (3 new core exports, no docs) fails, passes with the label
Not run in GitHub Actions yet: label/body wiring is tested via env locally only
Documentation updated

🤖 Generated with Claude Code

… not have scripts/cli-command-tree.ts loads every command module that cli.ts's commandLoaders registers and prints the citty command tree (subcommands, flags, aliases, value types, hidden and deprecated markers) as JSON. It reads the real definitions, not --help text, so it sees exactly what citty parses. scripts/check-docs-cli-references.mjs scans docs/**/*.mdx (fenced blocks with backslash continuations, inline code, and the catalog's InstallCommand command props) for `hyperframes ...` / `npx hyperframes ...` invocations and resolves each one the way citty does: the first non-flag word that is not a string flag's value names the subcommand. Unknown commands, subcommands and flags are errors with a "did you mean"; deprecated commands and flags and hidden flags are warnings. Placeholders (<dir>, [opts], {id}, $VAR, ...), comments, `--`, other tools (`npx skills add heygen-com/hyperframes`, `npm i -g hyperframes`) and the release logs are left alone. Quoted CLI output opts out with {/* cli-references: ignore */}. The Docs workflow runs it on docs and CLI source changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…ocs or say why not scripts/check-docs-impact.mjs compares the merge base with HEAD on two surfaces and fails when one changed and nothing under docs/ did: - CLI: every visible command, subcommand, flag, alias and flag value type, from scripts/cli-command-tree.ts. The base tree is built in a temporary worktree (bun install + core build), only when packages/cli/src changed outside tests and fixtures. - Public exports: for every package.json that is not private and declares exports (package-subpaths.mjs's rule), its exports subpaths, bin names and the names its root entry exports through `export *` chains, read from git with the TypeScript parser (scripts/public-api-surface.mjs). A PR opts out with the docs-not-needed label or a `docs-impact: none` line in its body, passed in as PR_LABELS / PR_BODY. --dry-run prints the verdict and exits 0; --base-tree/--head-tree take prebuilt trees. The Docs impact workflow runs it on every PR to main, including label and body edits, so changing the opt-out re-runs it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…to the workflow tests Splits codeSpans, the flag and subcommand walk, the export walker and the docs-impact entry point into small functions so fallow's audit passes, adds scripts/cli-command-tree.test.ts (cli.ts's loader block and a citty command described end to end), and lists docs-impact.yml among the workflows that read the PR body on an edit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

… clean The Studio download page hands Linux browsers the .deb by default and the AppImage only on request, so the install steps now match what people get. models and clean had full sections but no row in the command finder. The quoted auto-update line is output, not a command, so the CLI-references check skips it, and its versions now look current. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

mintlify · 2026-10-20T06:09:40Z

Preview deployment for your docs. Learn more about Mintlify Previews.

Project	Status	Preview	Updated
hyperframes	🟢 Ready	View Preview	Oct 20, 2026, 6:15 AM

💡 Tip: Enable Automations to automatically generate PRs for you.

github-actions · 2026-10-20T06:25:54Z

Edit accuracy: accurate 271 (base branch 271), smooth 150 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (0)

Unstable (1)

seqplainkeys-none-pct-r0-root-z100: tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo false, teleport true / tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true / tracking 0.06, pressJump 0, drop 0, reload 0, render 0.03, renderKey -, undo true, teleport true

ukimsanov · 2026-10-20T08:16:37Z

Closing. The doc fixes moved to #5946. The two CI checks are parked: the docs-impact gate would add a rule to every engineer's PRs, which needs a team conversation first, and the CLI-references check is ~900 lines for what can be pitched lean if the team wants it. The branch stays for reference.

ukimsanov and others added 4 commits October 20, 2026 02:09

mintlify Bot deployed to staging - docs October 20, 2026 06:15

View deployment

This was referenced Oct 20, 2026

docs(cli): document every command, and check it in CI #5938

Merged

docs: install the Studio app on Linux from the .deb #5946

Merged

ukimsanov closed this Oct 20, 2026

Navigation Menu

Repository navigation

ci(docs): catch docs that drift from the CLI - #5933

ci(docs): catch docs that drift from the CLI#5933
ukimsanov wants to merge 4 commits into
mainfrom
docs/freshness-checks

ukimsanov commented Oct 20, 2026

Uh oh!

mintlify Bot commented Oct 20, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 20, 2026

Uh oh!

ukimsanov commented Oct 20, 2026

Uh oh!

This branch was successfully deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

ukimsanov commented Oct 20, 2026

What

Why

How

Test plan

Uh oh!

mintlify Bot commented Oct 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 20, 2026

Edit accuracy: accurate 271 (base branch 271), smooth 150 of those

Uh oh!

ukimsanov commented Oct 20, 2026

Uh oh!

This branch was successfully deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

mintlify Bot commented Oct 20, 2026 •

edited

Loading