fix: Flush once per available batch of replayed events instead of per event#59
Draft
keelerm84 wants to merge 1 commit into
Draft
fix: Flush once per available batch of replayed events instead of per event#59keelerm84 wants to merge 1 commit into
keelerm84 wants to merge 1 commit into
Conversation
… event When a Repository replay delivers a large batch of events (for example, a proxy replaying a full data set to a newly connected SDK), the handler was calling flusher.Flush() after every event, producing one HTTP chunk write per event per connection. The handler now encodes the batch events that are immediately available and flushes once when the batch channel goes momentarily quiet, the batch ends, or maxEventsPerFlush (512) is reached. Timeliness is unchanged: a flush still happens before the handler blocks waiting for more data. Memory is bounded regardless of batch size -- the ResponseWriter's buffer writes through to the connection as it fills and a slow client backpressures the encode -- and the per-flush cap additionally guarantees the handler returns to its main select loop regularly, so connection close and MaxConnTime are honored even while an arbitrarily long batch is streaming. New BenchmarkReplayBatch (5000 events x 300B over a real HTTP connection, events preloaded in a buffered channel): 14.4ms -> 3.3ms per replay (-77%). Callers feeding the replay channel one unbuffered send at a time do not regress, but also do not benefit; the channel must be buffered for the drain to batch (BenchmarkReplayUnbuffered).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When a Repository replay delivers a large batch of events (for example, a
proxy replaying a full data set to a newly connected SDK), the handler was
calling flusher.Flush() after every event, producing one HTTP chunk write
per event per connection.
The handler now encodes the batch events that are immediately available and
flushes once when the batch channel goes momentarily quiet, the batch ends,
or maxEventsPerFlush (512) is reached. Timeliness is unchanged: a flush
still happens before the handler blocks waiting for more data. Memory is
bounded regardless of batch size -- the ResponseWriter's buffer writes
through to the connection as it fills and a slow client backpressures the
encode -- and the per-flush cap additionally guarantees the handler returns
to its main select loop regularly, so connection close and MaxConnTime are
honored even while an arbitrarily long batch is streaming.
New BenchmarkReplayBatch (5000 events x 300B over a real HTTP connection,
events preloaded in a buffered channel): 14.4ms -> 3.3ms per replay (-77%).
Callers feeding the replay channel one unbuffered send at a time do not
regress, but also do not benefit; the channel must be buffered for the
drain to batch (BenchmarkReplayUnbuffered).