Skip to content

Make reporters cope with sparse streams and plain TAP - #16

Merged
trans merged 1 commit into
modernizefrom
fix/reporters
Nov 20, 2026
Merged

trans merged 1 commit into
modernizefrom
fix/reporters

Conversation

@trans

@trans trans commented Nov 20, 2026

Copy link
Copy Markdown
Member

Stacked on #15 (its base is modernize); merge that first.

Found by running every reporter against real streams: rubytest 0.11's TAP-Y and TAP-J, minitap's on Minitest 5.25, and plain TAP.

Reporters raised on tests that say little

TAP-Y/J requires only a message in a non-passing test's exception section.

  • markdown raised for a failure without an exception class, html for one without a backtrace. Rubytest gives no class, and a backtrace only for errors.
  • Every reporter raised for a failed test with no exception section, which is what a failed test in plain TAP becomes.
  • outline and markdown raised for tests outside of any case, as in plain TAP.

The exception section is now filled in before a reporter sees the test, and the two reporters start with a level.

tap reporter and #14

It printed the plan and then no line for omitted or to-do tests, so a stream with either came out short. It now prints ok N - label # SKIP reason and not ok N - label # TODO reason. With no count at the start of the stream, the plan goes at the end. Closes #14.

outline now lists those tests too, with the reason.

Plain TAP input

  • TAP version 13 as the first line was rejected as "Not a recognized TAP stream". It is accepted, as is a stream that starts with a test line or a comment.
  • # SKIP and # TODO directives become omit and todo, with the reason as the message.
  • A test line needn't have a number.
  • The final counts include errors, omissions and to-dos, and a total when the stream has no plan.

Checked

Three new demos cover this: all ten reporters on a sparse TAP-Y stream and on plain TAP, the adapter's handling of the version line and directives, and #14. qed -Ilib demo/ passes (5 demos, 42 steps) on Ruby 3.4.9. All ten reporters also run without raising on seven sample streams from rubytest, minitap and plain TAP.

Not changed

A test with the status skip, which minitap emits and which isn't in the specification, is still treated as a to-do, though minitap counts it as an omission.

🤖 Generated with Claude Code

A test that didn't pass need only give a message, and not every
producer gives even that. Rubytest gives no exception class, and a
backtrace only for errors, which made the markdown and html reporters
raise. A failed test from plain TAP has no exception section at all,
which made every reporter raise. The exception section is now filled
in before a reporter sees the test.

tap reporter: print a line for omitted and to-do tests, with SKIP and
TODO directives, so the lines match the plan (#14). outline: list them
too. outline and markdown: handle tests outside of any case.

Plain TAP input: accept a version line and streams that open with a
test line or comment, turn SKIP and TODO directives into omit and
todo, number unnumbered tests, and give complete final counts.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@trans
trans merged commit fdbd4a2 into modernize Nov 20, 2026
14 checks passed
trans added a commit that referenced this pull request Nov 21, 2026
Fix reporters on sparse streams and plain TAP input (#16 onto master)
@trans trans mentioned this pull request Nov 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant