Skip to content

feat: report did-not-run checks as their own result - #7

Merged
asgeirf merged 3 commits into
mainfrom
fix/report-did-not-run
Sep 13, 2026
Merged

asgeirf merged 3 commits into
mainfrom
fix/report-did-not-run

Conversation

@asgeirf

@asgeirf asgeirf commented Sep 13, 2026

Copy link
Copy Markdown
Contributor

kapi check exits 4 when a check did not run (neokapi/neokapi#2656, #2661, #2662). The action reported that as "kapi check failed operationally (exit 4)". It still fails the step, and now reports the outcome and its cause.

What changed

  • The check step sets a result output for every outcome: passed (exit 0), failed (exit 3), did_not_run (exit 4) or error (any other code).
  • On exit 4 it sets did_not_run_cause and prints an ::error:: naming the cause. checker_invalid reads "a checker failed its canary, so this run's result cannot be trusted"; nothing_to_check reads "there was nothing in scope to check"; content_not_checked has its own sentence.
  • The cause comes from the output the step already captures: did_not_run_cause from a JSON report, the top-level YAML key, or the "(cause)" ending the text sentence, with ANSI color codes removed. No cause, or a value that is not a cause code, reports unknown. The command line is passed through unchanged.
  • Exit 0, 3, 1 and other codes keep their behavior, so kapi releases that never exit 4 behave as before apart from the new result output.
  • README: outputs table rows and a "When a check does not run" section. The ship-gate example drops commit: "false", an input the action no longer declares.

Tests

  • test/stub/kapi prints a fixture and exits with a chosen code.
  • Fixtures in test/check-output/ are captured from kapi built at neokapi 67d143a61: a pass, a failed ship gate, nothing_to_check in text, JSON, YAML and color, and content_not_checked from check --ship --gate voice in text and JSON. The checker_invalid, future-cause and unreadable-cause fixtures follow the same shapes by hand, since a missed canary cannot be provoked from outside kapi.
  • test/check-outcomes.sh runs the action's own step script, read out of action.yml, for 15 cases. It asserts the exit code, result, gate, did_not_run_cause, the annotation text, and that kapi receives the caller's arguments unchanged.
  • test.yml adds a job that runs the action through uses: ./ against the stub for 8 outcomes, a job for the harness, and result assertions on the existing jobs that run real kapi 1.2.0-rc14.
  • Must-fail: with exit 4 sent back to the generic error branch, the harness fails 33 assertions across its 13 exit-4 cases. The same change pushed here as 9e7317a failed both stub jobs in CI on their assertions (run 34764655241), while the four jobs running real kapi passed. d90b2eb reverts it, and the branch tree equals 7ef7ee3's.

No tag or release comes with this PR.

🤖 Generated with Claude Code

https://claude.ai/code/session_01VRid8i4qnuNfE7Lio73Uu4

asgeirf and others added 3 commits September 13, 2026 17:06
kapi check exits 4 when a check did not run. The check step still fails
with that code, and now sets result=did_not_run and did_not_run_cause and
names the cause in its error annotation and job summary. The cause is
read from the report kapi printed: the JSON or YAML key, or the "(cause)"
that ends the text sentence. result is set for every check outcome; exit
0, 3 and other codes behave as before.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VRid8i4qnuNfE7Lio73Uu4
Temporary, reverted in the next commit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VRid8i4qnuNfE7Lio73Uu4
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VRid8i4qnuNfE7Lio73Uu4
@asgeirf
asgeirf merged commit d266e4d into main Sep 13, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant