ci: run acceptance tests only when CLI or test code changes

[leggetter]

Closes #240.

The acceptance suite ran on every PR to main/next. A README-only PR spent three API-key-backed matrix jobs (12m timeout each) plus the telemetry job testing a binary nobody had touched. This adds a paths: allowlist to .github/workflows/test-acceptance.yml.

What triggers the suite

paths:
  - "main.go"
  - "go.mod"
  - "go.sum"
  - "pkg/**"
  - "test/**"
  - "tools/**"
  - ".github/workflows/**"

Derived from what the suite actually builds, not from a guess:

• main.go, go.mod, go.sum, pkg/** — test/acceptance/helpers.go runs go build -o <tmp> . and go run main.go. That build resolves the root package, every pkg/ package and the module graph.
• test/** — the tests and their helpers. test/ contains only test/acceptance, so test/** is exact, not over-broad.
• tools/** — tools/generate-reference derives docs from Cobra --help metadata; a change there can move output tests assert on.
• .github/workflows/** — the run is a two-file chain (test-acceptance.yml → acceptance.yml). Matching the directory means adding another reusable workflow to that chain does not require remembering to extend this list. Workflow edits are rare, so the extra runs are cheap.

Note the issue proposed cmd/**; there is no top-level cmd/, the commands live in pkg/cmd/, which pkg/** covers.

What does not trigger it

README.md, CHANGELOG.md, KNOWN_ISSUES.md, LICENSE, docs/, plans/, .plans/, .gitignore, .tool-versions, .cursor/, .agents/, .claude/, .github/dependabot.yml.

Also deliberately excluded, because they feed packaging, not the acceptance binary — and each already has its own workflow:

• .goreleaser/**, scripts/completions.sh → test.yml build jobs, release.yml
• package.json, bin/hookdeck.js, Dockerfile, test-scripts/**, .github/actions/test-npm-install/ → test-npm-build.yml, test-npm-install.yml, test-homebrew-build.yml

The two judgement calls

REFERENCE.md — excluded. It is generated from --help text, so a change to it alone is either generator output or a hand edit the generator will overwrite; either way it cannot change CLI behaviour. No acceptance test reads it. The guard that does exist — pkg/cmd/reference_metrics_doc_test.go, which asserts the documented gateway metrics subcommands are real — is a unit test under pkg/, and it is not -short gated. It runs in test.yml, which has no path filters at all. So a REFERENCE.md-only PR is still checked, by the job that checks it today.

AGENTS.md / CLAUDE.md — excluded. Instructions to coding agents. They are not compiled, not embedded (no go:embed anywhere in the repo) and not read by any test. They cannot change CLI behaviour.

Why an allowlist rather than paths-ignore

An allowlist fails safe. A new top-level directory nobody thought about does not trigger the suite, and that omission is visible — someone notices the run is missing. A denylist fails open in the opposite direction: it keeps running the full matrix on the new directory forever until somebody remembers to add an exclusion, which is the cost this issue is about.

Skipping is safe here specifically because unit-test is the only required status check on main's ruleset (confirmed via gh api repos/hookdeck/hookdeck-cli/rulesets), so a PR that skips acceptance is not left permanently pending on a check that will never report.

workflow_dispatch is untouched

It stays a bare workflow_dispatch: {} sibling of pull_request, carrying no paths. It is the escape hatch for exactly the case this filter creates, and the only way to exercise a stacked release branch (#395). A comment in the file says so, to stop anyone "tidying" a filter onto it later.

Verification

actionlint .github/workflows/test-acceptance.yml passes (exit 0), and the file parses to the expected structure — workflow_dispatch is a sibling of pull_request, so no filter applies to it.

Scenarios walked against the pattern set:

Change	Outcome	Matched by
pkg/cmd/listen.go	runs	pkg/**
test/acceptance/connection_test.go	runs	test/**
README.md only	skips	—
AGENTS.md only	skips	—
pkg/listen/listen.go + README.md	runs	pkg/** (any match wins)
.github/workflows/test.yml	runs	.github/workflows/**
REFERENCE.md only	skips	— (still covered by unit-test)
go.mod + go.sum (dependabot)	runs	go.mod, go.sum
tools/generate-reference/main.go	runs	tools/**
main.go	runs	main.go
.github/dependabot.yml	skips	—
test-scripts/, .goreleaser/, package.json, Dockerfile	skips	— (own workflows)
manual workflow_dispatch	runs	no path filter applies

This PR itself touches only .github/workflows/, so it exercises the new filter: the acceptance suite should run on it.

🤖 Generated with Claude Code

https://claude.ai/code/session_01BnrKWZQASV7bFJ4oGWwmo9

[leggetter]

Folded in a second change: acceptance now also runs on merge to main.

The suite has only ever triggered on pull_request — zero runs in history with headBranch: main — so main's head is strictly a commit nothing has run acceptance against. The PR gate is inherited on merge, and that inheritance is only as current as the PR's last pull_request event.

Not hypothetical: merging a PR into another PR's head branch fires no pull_request event, so #378 sat with 20 commits and 75 changed files against a stale acceptance run, and the code about to ship had never been through its own gate. That is also why workflow_dispatch was added.

test.yml already runs on push to main for precisely this reason:

Run on merge to main so the merged result is tested, not just each PR branch in isolation. A PR based on stale main can merge into a combination that no PR run exercised.

That argument applies at least as strongly to acceptance as to the unit and build jobs; it was just never extended.

It reuses the same paths filter as the pull_request trigger, so a docs merge does not spend a full suite run — which is what makes it affordable, and why it belongs here rather than in a separate PR.

actionlint clean; structure verified: three events, push scoped to main with the same 7 paths, workflow_dispatch still unfiltered.

(Added by Claude on Phil's behalf.)

[leggetter]

Third change folded in: acceptance runs are now serialised repo-wide.

concurrency:
  group: acceptance-suite
  cancel-in-progress: false

Placed on the reusable acceptance.yml rather than each caller, so every entry point — pull requests, merges to main, manual dispatch, and the release gate — shares one queue. They all exercise the same three projects with the same three API keys, and the API allows 240 requests/minute while a full slice makes thousands of calls.

This has now bitten twice in two days. First as all four jobs timing out with zero assertion failures. Then yesterday on this very PR: run 34952534514 failed with TestListenUsesHookdeckAPIKeyInsteadOfGuestAccount reporting "" does not contain "project_id" — which reads as a genuine auth regression. It overlapped the v2.6.0 release workflow 09:32→09:44, which runs the same suite against the same projects. The log carries 27×401 and 11×429. Re-run with nothing else in flight: all four jobs green.

The second shape is the dangerous one. A clean 429 wipeout is recognisable; a single named assertion failure is indistinguishable from a real bug and sends you into the wrong code.

cancel-in-progress: false is deliberate — release.yml calls this as its publish gate, and cancelling a release run to make room for a PR is the wrong trade. Queueing costs wall-clock and nothing else. With the path filters in this PR, the suite runs less often anyway.

Worth noting: I documented this hazard in #418 the same morning and then walked into it while cutting the release. A README note tells you how to recognise it afterwards; it does not prevent it.

actionlint clean apart from four pre-existing @v3 action warnings, identical on main.

(Added by Claude on Phil's behalf.)