Skip to content

Add -testQuiet to report only failing tests#1204

Closed
Donach wants to merge 1 commit into
wurstscript:masterfrom
Code-Fixxers:feat/test-quiet
Closed

Add -testQuiet to report only failing tests#1204
Donach wants to merge 1 commit into
wurstscript:masterfrom
Code-Fixxers:feat/test-quiet

Conversation

@Donach

@Donach Donach commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Motivation

On a project with hundreds of @test functions, wurst -runtests prints two or more
lines per passing test plus everything the tests themselves print. In CI that buries the
handful of lines that actually matter, even though the summary at the end already reports
what passed.

-compactOutput (#1188) shortens the output, but it does so by dropping detail for
failures too: no assertion message, no stack trace, no output from the failing test.

Change

Adds -testQuiet (short form -tq), which reports only what is wrong:

  • Output of a passing test is collected and dropped once the test passes.
  • A failing test prints exactly what it prints today: its own output, the assertion
    message, the stack trace and, for timeouts, the stack frames.
  • The summary is always printed.

Composes with -compactOutput; neither flag changes the behavior of the other.

Fixes to the interpreter output redirection

RunTests.redirectInterpreterOutput was dead code, so output the tests themselves printed
never reached RunTests and went straight to System.err. Three separate reasons, all
fixed here, because quiet mode can only suppress output that it actually receives:

  • ProgramStateIO shadowed ProgramState.outStream with its own field and overrode
    getOutStream/setOutStream without forwarding to the native providers. Removed; the
    base class already implements both and does the forwarding.
  • ReflectionNativeProvider.setOutStream was an empty method, so OutputProvider (which
    implements println, BJDebugMsg, DisplayTextToPlayer, ...) kept writing to
    its default System.err. It now forwards to the OutputProvider it owns.
  • The PrintStream wrapping the redirect had no autoflush, so anything short of 8 KB
    stayed in its buffer forever. Now autoflush with an explicit UTF-8 charset.

The visible consequence is that output printed by tests during -runtests now goes to the
stream RunTests prints to (stdout on the CLI) instead of System.err, which is what the
redirect was written to do. It also makes -compactOutput suppress that output, as
intended.

Tests

RunTestsQuietTests compiles a package with one passing and one failing test and asserts
on the produced output: quiet mode hides the passing test and its output, keeps the failing
test's output and assertion message, and keeps the summary; the default mode still reports
passing tests. Flag parsing is covered for both -testQuiet and -tq.

Running a project with hundreds of test functions prints several lines per
passing test, which buries the failures in CI. -testQuiet (short form -tq)
buffers the output of the running test, discards it when the test passes and
prints it in full when the test fails. The summary is always printed.

Also repairs the interpreter output redirection this relies on, which never
reached the tests' own output:

- ProgramStateIO shadowed ProgramState.outStream and overrode the accessors
  without forwarding to the native providers.
- ReflectionNativeProvider.setOutStream was empty, so OutputProvider kept
  writing to its default System.err.
- The PrintStream wrapping the redirect had no autoflush, so its buffer was
  never emptied.

As a result, output printed by tests during -runtests now goes to the stream
RunTests prints to instead of System.err, as the redirection intended.
@Donach

Donach commented Jul 25, 2026

Copy link
Copy Markdown
Contributor Author

Closing in favour of #1205.

The -testQuiet flag here is redundant: Grill's --quiet already covers this use case (it passes -compactOutput and filters the output, keeping failure headers and Test failed: lines), which is the layer our CI actually invokes.

What is worth keeping are the three bugs I ran into while testing the flag, all of which make RunTests.redirectInterpreterOutput dead code: ProgramStateIO shadowing ProgramState.outStream, the empty ReflectionNativeProvider.setOutStream, and the missing autoflush on the PrintStream. #1205 contains only those, with a regression test.

@Donach Donach closed this Jul 25, 2026
@Donach
Donach deleted the feat/test-quiet branch July 25, 2026 12:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant