Skip to content

docs: verify-ai-tokenizer: fail empty demo-site counts, fix site-up BUN doc - #17

Open
ThomasK33 wants to merge 1 commit into
mainfrom
docs/verify-skill-followups
Open

ThomasK33 wants to merge 1 commit into
mainfrom
docs/verify-skill-followups

Conversation

@ThomasK33

Copy link
Copy Markdown
Member

Why

Two low findings from the last verification of #11 (verify-ai-tokenizer skill). Stacked on #11. Merge #11 first.

What changes

  1. journeys/demo-site.sh: a check passed when both the expected and observed token counts were empty, for example when the page or the oracle failed. Each count check now needs a non-empty expected count.
  2. SKILL.md: site-up reads BUN from the run's meta.env, so a BUN= prefix on site-up had no effect. The doc now says that site-up uses the bun recorded at launch.

Proof

  • Against an unreachable URL, all 7 count checks now print FAIL. Before, 4 printed PASS with empty values.
  • A normal run (launch, site-up, drive demo-site, cleanup) gives the same results as before: 6 of 8 checks pass. The 2 special-token checks fail until fix: count special-token text as ordinary text #9 lands.

Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: high

…UN doc

- A demo-site check passed when both the expected and the observed
  token count were empty. It now needs a non-empty expected count.
- site-up uses the BUN that launch recorded, so SKILL.md no longer
  shows a BUN= override for it.

_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
@ThomasK33
ThomasK33 added this pull request to stack #18 November 3, 2026 12:30
@ThomasK33
ThomasK33 requested a review from kylecarbs November 3, 2026 12:30
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex security review

@ThomasK33
ThomasK33 removed this pull request from stack #18 November 3, 2026 12:45
@ThomasK33
ThomasK33 changed the base branch from docs/verify-ai-tokenizer-skill to main November 3, 2026 12:45

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant