Conversation
SECURITY.md states its guarantees as FAIL IF conditions so a machine can execute them, but nothing executed them. A single Opus agent now runs every one as a mechanical check with evidence, then reads the auth and infrastructure code adversarially, and publishes one report. The port of Dormouse's nightly audit drops everything that only exists because that repo has three domains: no orchestrator, no subagents, no fragment merge, no separate status file. The verdict is the report's own first line, and a `<!-- END OF REPORT -->` sentinel is what says the report is finished — so an agent cut off after rewriting that line reads as inconclusive rather than as a pass. Three outcomes, not two: an audit that never decided files under the same label as a finding and holds the check run red, but says in its own issue that it is not a security finding. The push-to-main trigger is load-bearing rather than convenience. A consumer that vendors a packed tarball recovers the commit from dist/provenance.json; it can now confirm that commit was audited by reading the security-audit check run on it, which is why the job id and its name are both that string. Claude Code is invoked directly instead of through claude-code-action, which throws `Unsupported event type: push` — its context parser accepts only workflow_dispatch, repository_dispatch, schedule and workflow_run, and a step-level GITHUB_EVENT_NAME cannot stand in because the runner writes the real GITHUB_* values over a step's env. The CLI keeps the per-commit check run and makes CI and the local runner the same command over the same prompt files. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The reporting step is the only thing that decides whether a commit's security-audit check run goes green, and it was enforced by nobody: a `VERDICT: PASS` widened to a prefix match, a dropped sentinel test, or a sink removed from the redactor's argument list would all have passed review and shipped. `pnpm test:scripts` extracts the shipped shell out of the workflow by indentation and runs it — a renamed step or a moved block scalar throws rather than quietly handing the test the rest of the file. The sinks the redactor must cover are read from the archive step's own path list, so a file that starts being published without starting to be redacted fails here instead. The local runner is covered against the same grammar, and `.github/audit/_preamble.md` is pinned as the producer of the sentinel every reader waits for, since nothing else writes that literal. Verified by mutation: widening the PASS match, dropping the sentinel requirement, and removing a sink from the redactor each turn the suite red. `clamp-issue-body.mjs` and its self-test come from Dormouse unchanged but for their pointers; GitHub rejects an over-long issue body outright, which loses the whole finding rather than truncating it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #9 (
security-spec), which is on #8.What
A single-domain port of Dormouse's nightly security audit, executing
SECURITY.md:.github/workflows/security-audit.yml— job and check run both namedsecurity-audit; runs nightly, onworkflow_dispatch, and on every push tomain, so everymaincommit carries a check run a consumer can read (gh api repos/diffplug/pgstencil/commits/<sha>/check-runs). That is the contract dormouse#745 verifies for the commit it vendors..github/audit/_preamble.md+.github/audit/security.md) runs everyFAIL IFmechanically (db:verify, unit, the Better Auth integration suites under Docker,packages:verify), then an adversarial pass, appending toaudit-report.mdas it goes. Verdict grammar: exactVERDICT: PASSplus the<!-- END OF REPORT -->sentinel is the only pass; aVERDICT: FAILprefix is a finding even when cut off; anything else is INCONCLUSIVE. FAIL and INCONCLUSIVE file or append to asecurity-audit-failureissue (retitled upward only); PASS closes it.scripts/security-audit-local.shruns the same prompt files locally;scripts/security-audit.test.mjs(28 tests,pnpm test:scripts, now incheck.yml) pins the reporting grammar, the retitle rule, the redactor, and the local runner, mutation-checked.SECURITY.mdgains theFAIL IFfor this workflow and a "How this file is checked" section.One design difference from Dormouse
anthropics/claude-code-actionthrowsUnsupported event type: push, and apushtrigger is what gives each commit its check run. So this job installs a pinned Claude Code (2.1.278, the same installer the action uses) and runsclaude -pwith the same model and tool flags. No GitHub App install, noid-token: write; CI and the local runner now execute the same binary over the same prompts. The headless flag set has not been exercised in a fresh CI HOME yet; a misbehaving run leaves no sentinel and reports INCONCLUSIVE, never a false PASS.Operator step
Add
CLAUDE_CODE_OAUTH_TOKENas a repository secret. Until then every run fails at the verify step with a report naming the missing secret.Verified
pnpm format:check,typecheck,test:scripts(28),test:unit,actionlinton both workflows,bash -non everyrun:block. The audit itself was not run.🤖 Generated with Claude Code