Skip to content

Keep the error body of a failed streaming request, and fix the three framing defects the review found - #14

Merged
Sunrisepeak merged 2 commits into
masterfrom
fix/stream-error-body-and-framing
Aug 29, 2026
Merged

Keep the error body of a failed streaming request, and fix the three framing defects the review found#14
Sunrisepeak merged 2 commits into
masterfrom
fix/stream-error-body-and-framing

Conversation

@Sunrisepeak

Copy link
Copy Markdown
Member

Supersedes #12, which it carries as its first commit with @Cloud_Yun's authorship intact. Fixes #11.

It is one PR rather than two because #12's head is on a fork I cannot push to, and the second half is what makes the first half usable on the library's own defaults.

The reported defect, confirmed

One program, one source file, built against master and against this branch with target/ wiped between them:

master      status=418 events=0 body.size()=0
this branch status=418 events=0 body.size()=135

events=0 confirms the mechanism and not just the symptom: httpbin's teapot document does contain a blank line, so SseParser finds a boundary — the block it extracts has no data:, event: or id: field, so dispatch_event_ returns without pushing. The bytes reached the parser and died there.

The fix is placed correctly. dispatch is the single funnel for every body byte on both framing paths, and captureBody is decided after the headers, where statusCode is final (send_stream follows no redirects). Capturing there is also better than reaching into SseParser's buffer, which is not the whole body once any event has been dispatched.

1. A declared Content-Length — which is why the test had to close the connection

send_stream had no branch for it: a response that was not chunked was read until the connection closed, whatever its headers said. On the library's defaults (keepAlive = true) the server does not close, and the loop ran until readTimeoutMs expired. The evidence was in the test itself:

cfg.keepAlive = false;  // so the server closes and the read loop ends

Measured against /status/418:

keepAlive = false   status=418 body=135   elapsed  1370 ms
keepAlive = true    status=418 body=135   elapsed  9379 ms   (readTimeoutMs = 8000)
after this change:  status=418 body=135   elapsed  1192 ms

The body arrived either way; on the configuration a caller actually gets, it arrived a full read timeout late — a minute, at the shipped default of 60000. send() has had this branch throughout, which is the same asymmetry between the two entry points that #11 is about.

The live test now runs on the default configuration and asserts the elapsed time. A test that closes the connection to make the loop end is examining the one arrangement in which the defect does not appear.

2. A chunk size that does not parse is not a terminal chunk

parse_hex returns what it accumulated when it meets an unrecognised character, and 0 for an empty line — and read_line returns an empty line on a timeout or a closed connection. A stream that was cut short therefore read as a stream that ended cleanly, and the loop reported success.

#9 established parse_chunk_size_line for exactly this and it reached download_to_file alone; send and send_stream were left on the old parser.

3. Content-Length was parsed by keeping the digits

Measured, by compiling that parser on its own:

"135"                  -> 135
"abc"                  -> 0                     <- a refusal, read as a real zero
"12abc"                -> 12                    <- the reader stops twelve bytes in
"-1"                   -> 1                     <- the sign is discarded
"99999999999999999999" -> 7766279631452241919   <- wraps, in silence

The last two are the ones no care at the call site could recover from, because what it receives is a plausible number. parse_content_length is exported and shaped like parse_chunk_size_line, for the reason #9 gave — it is the half of the body framing that can be examined without a server — and both readers use it.

Criteria

Six unit tests over the two pure parsers, and two live ones: the failed stream on the default configuration with the elapsed time asserted, and a chunked 2xx asserted to leave body empty and return promptly, since the chunked branch is the one this change restructured around.

17 tests from 6 test suites pass, plus 3 in test_resolver.

And it works on openkal

tinyhttps builds and runs on openkal-musl: statically linked, zero PT_INTERP, TLS and DNS working, and this change behaves identically there.

One thing invisible from inside this library, recorded so nobody has to re-derive it. Socket::wait_readable is poll_fd, and on openkal-linux before 0.7.1 the bounded wait was performed on the descriptor below the one it then transferred on — so readTimeoutMs silently did nothing:

/delay/10 with readTimeoutMs = 2000
  host gnu (control)      status=0    elapsed  3051 ms
  openkal-linux 0.7.0     status=200  elapsed 11350 ms   <- waited out the server
  openkal-linux 0.7.1     status=0    elapsed  2994 ms

Nothing here needs to change for it; it is fixed beneath, in mcpplibs/openkal-linux#19.

yspbwx2010 and others added 2 commits August 30, 2026 03:52
send() fills HttpResponse::body on every path including failures; send_stream()
was the one entry point that dropped it. A non-2xx answer to a streaming request
is an error document, not an event stream: SseParser finds no event boundary in
it, emits nothing, and the bytes stay in its private buffer. Callers were left
with a status line and no reason.

Capture the body when the status is not 2xx. Events are still parsed and
dispatched exactly as before, and nothing is copied on a 2xx stream, so the
success path is byte-identical.

The copy is bounded by stream_error_body_limit (1 MiB) so a server answering 5xx
with an endless body cannot grow the buffer without limit. Truncation lives in an
exported append_within_limit, in the same spirit as parse_chunk_size_line, with
three unit tests for under, across and past the limit; a live test against
httpbin's /status/418 covers the wiring.
…ize rather than salvaging it

Review of the change this branch already carries. The defect it reports is real
and the fix is placed correctly --- `dispatch` is the single funnel for every
body byte on both framing paths, and `captureBody` is decided after the headers
are read, where the status is final. Measured against master, one program, one
source file:

    master   status=418 events=0 body.size()=0
    this     status=418 events=0 body.size()=135

What follows is what that fix could not do on its own.

--- 1. A DECLARED LENGTH, WHICH IS WHY THE TEST HAD TO CLOSE THE CONNECTION ----

`send_stream` had no branch for `Content-Length`: a response that was not
chunked was read until the connection closed, whatever its headers said. On this
library's own defaults --- `keepAlive = true`, so the request carries
`Connection: keep-alive` --- the server does not close, and the read loop ran
until `readTimeoutMs` expired. Measured against httpbin's `/status/418`:

    keepAlive = false   status=418 body=135   elapsed  1370 ms
    keepAlive = true    status=418 body=135   elapsed  9379 ms   (timeout 8000)

The error body arrived either way, and on the defaults it arrived a full read
timeout late --- sixty seconds, as the defaults stand. `send()` has had this
branch throughout, which is the same asymmetry between the two entry points that
this branch exists to remove.

The live test set `keepAlive = false`, "so the server closes and the read loop
ends". That comment was the defect, and the test was examining the one
arrangement in which it does not appear. It now runs on the defaults and asserts
the elapsed time.

    after: keepAlive = true    status=418 body=135   elapsed  1192 ms

--- 2. A CHUNK SIZE THAT DOES NOT PARSE IS NOT A TERMINAL CHUNK ---------------

`parse_hex` returns what it accumulated when it meets a character it does not
recognise, and zero for an empty line --- and `read_line` returns an empty line
on a timeout or a closed connection. So a stream that was cut short read as a
stream that ended cleanly and this loop reported success. #9 established
`parse_chunk_size_line` for exactly this and it reached `download_to_file`
alone; `send` and `send_stream` were left on the old one.

--- 3. Content-Length WAS PARSED BY KEEPING THE DIGITS ------------------------

Measured, by compiling that parser on its own:

    "135"                  -> 135
    "abc"                  -> 0                     <- a refusal read as a real zero
    "12abc"                -> 12                    <- stops twelve bytes in
    "-1"                   -> 1                     <- the sign is discarded
    "99999999999999999999" -> 7766279631452241919   <- wraps, in silence

The last two are the ones no care at the call site could recover from, because
what it receives is a plausible number. `parse_content_length` is exported and
shaped like `parse_chunk_size_line`, for the reason #9 gave: it is the half of
the body framing that can be examined without a server. Both readers use it.

--- criteria -----------------------------------------------------------------

Six unit tests over the two pure parsers, and two live ones: the failed stream
now runs on the DEFAULT configuration with the elapsed time asserted, and a
chunked 2xx is asserted to leave `body` empty and to return promptly --- the
success path is the one this change restructured around, so it is observed
rather than assumed.

17 tests from 6 suites pass, plus 3 in test_resolver.
@Sunrisepeak

Copy link
Copy Markdown
Member Author

@Sunrisepeak
Sunrisepeak merged commit 2cec1c1 into master Aug 29, 2026
1 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

send_stream never fills HttpResponse::body, so a failed streaming request carries no reason

2 participants