Fix output capture breaking on multi-byte characters

[davidanthoff]

Fixes #101.

What was wrong

The output demultiplexer in src/testprocess.jl — the loop that attributes a test process's output to test items by the \x1f… frame markers — mixed character counts (length(buffer)) with byte indices (buffer[i], SubString(buffer, i, j)). Three consequences:

• The test item id was sliced as "up to one byte before the "", which is not a character boundary when the byte before the quote is the tail of a multi-byte character. That is the StringIndexError in the report ([119]=>'✔', [122]=>'"'). Once the reading task dies, nothing the process prints is forwarded for the rest of its life.
• With multi-byte text in the buffer the loop stopped before the end of the buffer, delaying output by a chunk and silently dropping the tail at process exit.
• A pipe read that split a multi-byte character forwarded two malformed fragments.

The " terminator was also fragile: the server wrote the marker and the id as separate print arguments, i.e. separate libuv writes with a yield between them, so output from another task (or from stdout, which is a different fd into the same pipe) could land inside the frame header. Genie's own ✔ "…" logging is the likely origin of the ✔" in the report. A test item whose name contains a " was truncated the same way.

What changed

• src/testprocess.jl: the parser is now a small, pure OutputDemuxer (feed!/flush!) that works on bytes throughout. It holds back a partial marker, an id without its terminator, and an incomplete trailing UTF-8 sequence for the next chunk, so every forwarded string is valid UTF-8 when the process's output is, and it never throws on any byte sequence. The @async reader is now a thin loop over it, and forwards the buffered tail at EOF.
• testprocess/TestItemServer/src/TestItemServer.jl: the frame header is written as one string — marker * id * '\x1f' — so it goes out in a single write, and the id terminator is \x1f (a control character that cannot appear in an id or in ordinary output) instead of ". Server and parser ship together via the [sources] path, so the wire format changes in lockstep.
• docs/src/internals.md: describes the frame and the parsing rules.
• Tests: test/test_output_demux.jl unit-tests the demuxer directly (the Capture gets irritated by unicode output #101 stream, ids with quotes and multi-byte characters, every chunk split point, UTF-8 hold-back, stray \x1f bytes, a random-bytes fuzz that checks nothing is lost and nothing throws). test/test_output.jl gains an end-to-end item — unicode output ✔ in BasicPackage, whose id therefore ends in ✔ like the report's crash site — that prints a 100,000-character ✔ line so a pipe read really does split a character, and checks the output is complete, valid UTF-8 and attributed to the right item.

Verification

Full suite via the workspace test runner: 239/239 test items pass on Julia 1.12 / Windows.

🤖 Generated with Claude Code