Skip to content

fix: Flush once per available batch of replayed events instead of per event#59

Draft
keelerm84 wants to merge 1 commit into
mainfrom
mk/SDK-2699/sse-flush-batching
Draft

fix: Flush once per available batch of replayed events instead of per event#59
keelerm84 wants to merge 1 commit into
mainfrom
mk/SDK-2699/sse-flush-batching

Conversation

@keelerm84

Copy link
Copy Markdown
Member

When a Repository replay delivers a large batch of events (for example, a
proxy replaying a full data set to a newly connected SDK), the handler was
calling flusher.Flush() after every event, producing one HTTP chunk write
per event per connection.

The handler now encodes the batch events that are immediately available and
flushes once when the batch channel goes momentarily quiet, the batch ends,
or maxEventsPerFlush (512) is reached. Timeliness is unchanged: a flush
still happens before the handler blocks waiting for more data. Memory is
bounded regardless of batch size -- the ResponseWriter's buffer writes
through to the connection as it fills and a slow client backpressures the
encode -- and the per-flush cap additionally guarantees the handler returns
to its main select loop regularly, so connection close and MaxConnTime are
honored even while an arbitrarily long batch is streaming.

New BenchmarkReplayBatch (5000 events x 300B over a real HTTP connection,
events preloaded in a buffered channel): 14.4ms -> 3.3ms per replay (-77%).
Callers feeding the replay channel one unbuffered send at a time do not
regress, but also do not benefit; the channel must be buffered for the
drain to batch (BenchmarkReplayUnbuffered).

… event

When a Repository replay delivers a large batch of events (for example, a
proxy replaying a full data set to a newly connected SDK), the handler was
calling flusher.Flush() after every event, producing one HTTP chunk write
per event per connection.

The handler now encodes the batch events that are immediately available and
flushes once when the batch channel goes momentarily quiet, the batch ends,
or maxEventsPerFlush (512) is reached. Timeliness is unchanged: a flush
still happens before the handler blocks waiting for more data. Memory is
bounded regardless of batch size -- the ResponseWriter's buffer writes
through to the connection as it fills and a slow client backpressures the
encode -- and the per-flush cap additionally guarantees the handler returns
to its main select loop regularly, so connection close and MaxConnTime are
honored even while an arbitrarily long batch is streaming.

New BenchmarkReplayBatch (5000 events x 300B over a real HTTP connection,
events preloaded in a buffered channel): 14.4ms -> 3.3ms per replay (-77%).
Callers feeding the replay channel one unbuffered send at a time do not
regress, but also do not benefit; the channel must be buffered for the
drain to batch (BenchmarkReplayUnbuffered).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant