fix(session-recorder): prevent unbounded error loop from corrupted TextDecoder in emit - #305
Open
NickSolante wants to merge 3 commits into
Open
Conversation
🦋 Changeset detectedLatest commit: e42b015 The changes in this PR will be included in the next version bump. This PR includes changesets to release 2 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Greptile SummaryThis PR prevents persistent session-recorder emit failures caused by corrupted Safari
Confidence Score: 5/5The PR appears safe to merge. No blocking failure remains.
|
| Filename | Overview |
|---|---|
| packages/session-recorder/src/index.ts | Adds guarded emit processing, OpenTelemetry diagnostics, circuit-breaker teardown, and clean state reset across recorder lifetimes. |
| packages/session-recorder/src/sessionrecording-utils.ts | Adds lossless UTF-8-aware chunking with a single-chunk fast path and fresh-decoder retry. |
| packages/session-recorder/tests/emit.test.ts | Covers successful emits, swallowed failures, breaker activation, post-breaker suppression, and reinitialization. |
| packages/session-recorder/tests/sessionrecording-utils.test.ts | Covers chunk identity, lossless reassembly, multibyte boundaries, and decoder recovery. |
| .changeset/session-recorder-decoder-crash-loop.md | Records the session-recorder patch and accurately describes the decoder and circuit-breaker changes. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[rrweb emits event] --> B[Serialize and split into chunks]
B -->|Success| C[Convert all chunks]
C --> D[Queue logs]
D --> E[Reset consecutive failures]
B -->|Failure| F[Report through OpenTelemetry diagnostics]
F --> G{10 consecutive failures?}
G -->|No| H[Drop current event]
G -->|Yes| I[Stop recorder]
I --> J[init resets breaker state]
Reviews (5): Last reviewed commit: "Merge branch 'main' into fix/session-rec..." | Re-trigger Greptile
NickSolante
force-pushed
the
fix/session-recorder-decoder-crash-loop
branch
from
August 11, 2026 01:44
cb52d8d to
c43ff08
Compare
…xtDecoder in emit WebKit's TextDecoder can permanently corrupt after high cumulative decode volume and then throw `RangeError: Bad value` on valid input (https://bugs.webkit.org/show_bug.cgi?id=286266). Because emit() reused one module-level decoder with no error handling, a single corrupted instance turned every rrweb flush into an uncaught error (~40/sec) until page unload. - Skip the encode/decode round-trip entirely for single-chunk events, with a length*3 fast path that avoids encoding on the hot path at all - Use a fresh per-event TextDecoder for multi-chunk events, with { stream: true } so chunk boundaries no longer bisect multi-byte characters into U+FFFD - Retry a failed decode once with a new decoder - Stop the recorder after 10 consecutive emit failures instead of erroring on every event forever Circuit-breaker hardening (from review): - init() resets paused/consecutiveEmitFailures so a re-init after a breaker trip records again with the full failure tolerance - Every emit failure is surfaced via diag.error, not just in debug - All chunks are converted before any is sent, so a mid-loop failure can't export a partial (unreassemblable) chunk set - The breaker's deinit() can't throw back into rrweb, deinit() clears inited before calling stop, and init() finishes teardown if the trip happened during rrweb's synchronous initial snapshot - emit tests are now self-contained per it-block Fixes hyperdxio#303
NickSolante
force-pushed
the
fix/session-recorder-decoder-crash-loop
branch
from
August 11, 2026 01:46
c43ff08 to
ac43184
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #303
Problem
On Safari, the session recorder's
emit()could enter a permanent, unbounded error loop: WebKit'sTextDecodercan permanently corrupt after high cumulative decode volume and then throwRangeError: Bad valueon valid input (WebKit #286266, wasm-bindgen report).emit()reused one module-level decoder for the page lifetime with no error handling, so a single corrupted instance turned every rrweb mutation flush into an uncaught error (~40/sec, observed 64,606 errors over 26 minutes from one session) until page unload. Full telemetry in #303.Changes
TextDecoder. This also removes the cumulative decode volume that ages the decoder toward WebKit's corruption threshold.TextDecoderwith{ stream: true }, which also fixes a latent all-browser bug: byte-slice boundaries that bisected a multi-byte UTF-8 character previously produced U+FFFD replacement characters at chunk seams, silently corrupting reassembled replay JSON.emit()failures are caught (each dropped event is surfaced viaconsole.error), and after 10 consecutive failures the recorder stops instead of erroring on every event forever.init()resets the breaker state so a later recording lifetime starts clean.Tests
splitIntoChunksunit tests: single-chunk identity (no decode), lossless multi-chunk reassembly, a multi-byte character bisected at the chunk boundary (fails against the old non-streaming decode), and recovery via the fresh-decoder retry (first decoder instance always throwsRangeError: Bad value).emit()error-handling tests (mocked rrweb/exporter/tracer): failures don't throw into rrweb dispatch, the breaker trips on the 10th consecutive failure and stops recording, nothing is emitted after tripping, and recording works again after a re-init.compile:tscclean,jest10/10 passing.Notes
@hyperdx/otel-web-session-recorder).MutationRateLimiter's constructor starts asetIntervalthat is never cleared (it also survivesdeinit()); the emit tests use fake timers to work around it. Happy to fix in a follow-up.