Skip to content

feat: buffering/loading state for AudioFileSourceNode and <Audio> - #1243

Merged
mdydek merged 6 commits into
software-mansion:mainfrom
gavrichards:feat/audio-tag-buffering-state
Aug 20, 2026
Merged

feat: buffering/loading state for AudioFileSourceNode and <Audio>#1243
mdydek merged 6 commits into
software-mansion:mainfrom
gavrichards:feat/audio-tag-buffering-state

Conversation

@gavrichards

Copy link
Copy Markdown
Contributor

What

Adds a buffering/loading-state signal for AudioFileSourceNode and <Audio>. Today the audio-tag state model is only AudioTagPlaybackState = 'idle' | 'playing' | 'paused' — there's no way to tell a genuine network/decoder stall apart from normal playback, and no onWaiting/onStalled-style event. This surfaced while evaluating the library for a production radio app: both the buffering-spinner UI and a reconnect/stall-watchdog need this signal and currently have nothing to build on.

How

  • Native (shared C++ core, both platforms): detects render-thread frame starvation directly in AudioFileSourceNode::processDecodedOutput() — the point where the render thread already silently zeros the output buffer when no decoded chunk is available (!hasFreshChunk && pendingDecoderChunk_.size == 0). This is the single choke point both direct playback and MediaElementAudioSourceNode-routed (live-stream) playback go through, so both benefit without platform-specific code.
  • Starvation is debounced against normal decode-ahead jitter via a new ON_BUFFERING_STATE_DEBOUNCE_INTERVAL (150ms) threshold, accumulated across render quanta. Recovery is reported immediately with no symmetric debounce, since a UI wants to clear a spinner as soon as audio resumes. pause()/disable() reset the state so a deliberate pause never leaves a stale buffering reading behind.
  • New AudioEvent::BUFFERING_STATE_CHANGE + BoolValuePayload, wired through the existing EventCaller/HostObject/JsEnumParser plumbing exactly like the other AudioFileSourceNode events (onPositionChanged, onEnded) — no new architecture, same pattern throughout.
  • JS: AudioTagPlaybackState gains a 'buffering' sub-state. It's derived from the existing playbackState + a separate isBuffering flag rather than folded into one state variable, so the effect subscribing to buffering events doesn't tear itself down and re-subscribe every time a stall starts or ends.
  • <Audio> gains onWaiting/onPlaying props, mirroring the HTML <audio>/<video> spec events of the same name (waiting = stopped due to lack of data, playing = resumed after being paused/waiting) — chosen deliberately to fit the library's existing HTML-media-flavored event naming (onLoadStart, onLoad, onEnded, onPlay, onPause) rather than inventing new terms. Wired for both the native (Audio.tsx) and web (Audio.web.tsx) implementations — the web variant gets it almost for free since the underlying <audio> DOM element already fires waiting/playing natively, which was a useful sanity check that the naming choice lines up with existing browser semantics.

Testing

  • New AudioFileSourceNodeTest.cpp (7 tests) covers the debounce state machine directly — crossing/not crossing the threshold, immediate recovery, no-op when no listener is registered, intermittent jitter not tripping it, and pause() resetting stale state — rather than racing the real decoder daemon thread to force genuine starvation deterministically.
  • yarn typecheck, yarn lint:js, yarn lint:cpp, yarn format:check:common all clean.
  • Full C++ suite passes: 394 tests (387 existing + 7 new).

Not yet verified: on a real device against a genuine network stall. The 150ms debounce threshold is a reasoned starting point (long enough to skip normal decode-ahead jitter, short enough to feel responsive), not one tuned against real-world Icecast/HLS stall behavior — happy to adjust based on review or if it needs to be configurable.

gavrichards and others added 2 commits August 19, 2026 15:36
Adds the buffering/stall signal identified as missing during aiirmobile's
react-native-audio-api evaluation: the audio-tag state model previously
exposed only 'idle' | 'playing' | 'paused', with no way to tell a genuine
network/decoder stall apart from normal playback, and no onWaiting/onStalled-
style event.

- Native (shared C++ core, both platforms): detects render-thread frame
  starvation directly in AudioFileSourceNode::processDecodedOutput() (the
  single choke point both direct and MediaElementAudioSourceNode-routed
  playback go through), debounced against normal decode-ahead jitter via a
  new ON_BUFFERING_STATE_DEBOUNCE_INTERVAL (150ms) threshold. Recovery is
  reported immediately, with no symmetric debounce. pause()/disable() reset
  the state so a deliberate pause never leaves a stale 'buffering' reading.
- New AudioEvent::BUFFERING_STATE_CHANGE + BoolValuePayload, wired through
  the existing EventCaller/HostObject/JsEnumParser plumbing exactly like the
  other AudioFileSourceNode events (onPositionChanged, onEnded).
- JS: AudioTagPlaybackState gains a 'buffering' sub-state (derived, not a
  parallel piece of state, so a stall starting/ending doesn't tear down the
  effect watching it), and <Audio> gains onWaiting/onPlaying props mirroring
  the HTML <audio>/<video> spec events of the same name. Wired for both the
  native (Audio.tsx) and web (Audio.web.tsx) implementations.
- New AudioFileSourceNodeTest.cpp covers the debounce state machine directly
  (crossing/not crossing threshold, immediate recovery, no-op without a
  listener, intermittent-jitter not tripping it, pause() resetting stale
  state) rather than racing the real decoder daemon thread for genuine
  starvation.

Verified: yarn typecheck, yarn lint:js, yarn lint:cpp, yarn
format:check:common all clean; full C++ suite (394 tests, was 387) passes
including the 7 new tests.

Not yet verified: on a real device against real network stalls — the debounce
threshold (150ms) is a starting point, not tuned against real-world Icecast/
HLS stall behavior.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
android/.../AudioEvent.kt is a hand-maintained mirror of AudioEvent.h,
crossed via .ordinal at the JNI boundary (see PlaybackNotification.kt)
rather than referenced directly the way the iOS .mm files can. Missed
this when adding BUFFERING_STATE_CHANGE — caught by CI's
check-audio-enum-sync job.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Aug 19, 2026

Copy link
Copy Markdown

WPT non-regression comparison

PASS — no regressions · 0 improved section(s) · overall 2632 → 2632 (0)

Unchanged sections (28)
Spec section Base pass Head pass Delta
Processing model 0 0 0
Other 52 52 0
AnalyserNode 138 138 0
AudioBuffer 140 140 0
AudioBufferSourceNode 208 208 0
AudioContext 58 58 0
AudioNode 261 261 0
AudioParam 599 599 0
BiquadFilterNode 275 275 0
ChannelMergerNode 30 30 0
ChannelSplitterNode 7 7 0
ConstantSourceNode 59 59 0
ConvolverNode 170 170 0
DelayNode 104 104 0
DestinationNode 0 0 0
DynamicsCompressorNode 4 4 0
GainNode 15 15 0
IIRFilterNode 87 87 0
MediaElementAudioSourceNode 0 0 0
MediaStreamAudioDestinationNode 1 1 0
MediaStreamAudioSourceNode 0 0 0
OfflineAudioContext 35 35 0
OscillatorNode 104 104 0
PannerNode 75 75 0
PeriodicWave 33 33 0
ScriptProcessorNode 0 0 0
StereoPannerNode 102 102 0
WaveShaperNode 75 75 0

Baseline: ed9106e6c1763a03a2c5d76875e1ac71d6b73aed · Candidate: 8c4caeaf1badce813c708e2888dd26547b16cec8

Workflow run · this comment is updated on every push.

@gavrichards

Copy link
Copy Markdown
Contributor Author

The WPT run (head) / WPT compare failure looks like a CI flake, not something caused by this PR — wanted to leave the investigation here in case a re-run is easiest.

  • Both WPT run (base) and WPT run (head) reach the exact same last test (the-delaynode-interface/maxdelay-rounding.html) with identical (pre-existing, unrelated) assertion output.
  • base finishes and uploads its report ~25s later. head instead goes silent for ~5 minutes before the job exits with code 129 (SIGHUP) — the same code this harness's own signal handler uses for interrupted runs, consistent with the process being killed rather than crashing on its own.
  • Reproduced locally against this exact branch (built rnaudioapi_node_bindings via cmake-js, ran yarn wpt:only): the full the-audio-api smoke suite completes in ~3 minutes with no hang anywhere, including the exact maxdelay-rounding.htmlno-dezippering.html transition that stalled in CI (2588/3259 passed, matching the known-failure profile from both CI logs).
  • Nothing in this PR's diff (native or JS) touches thread lifecycle, process exit, or anything outside AudioFileSourceNode's own per-quantum buffering-state bookkeeping — and no test in the smoke suite exercises AudioFileSourceNode/MediaElementAudioSourceNode at all (those are excluded as crashtests), so there isn't an obvious mechanism for this diff to affect it either way.

Happy to dig further if a re-run reproduces it, but based on the above I'd guess a runner hiccup.

Comment thread packages/react-native-audio-api/src/Audio/AudioFileSourceNode.ts Outdated
Comment thread packages/react-native-audio-api/src/Audio/types.ts
Comment thread packages/react-native-audio-api/src/jsi-interfaces.ts Outdated
@mdydek

mdydek commented Aug 20, 2026

Copy link
Copy Markdown
Member

@gavrichards yes, the CI is certainly broken, we are aware of that, obviously your changes did not change any files connected with wpt tests

@mdydek mdydek added the feature New user-facing features or major capabilities label Aug 20, 2026
…sed poll getter

Per @mdydek's review on software-mansion#1243:

- Extract the buffering-state debounce/dispatch logic out of
  AudioFileSourceNode into its own BufferingStateDispatcher class, mirroring
  PositionChangedDispatcher — AudioFileSourceNode now just calls
  bufferingStateDispatcher_.advance(hasData, framesToProcess) at each call
  site and no longer owns the EventCaller/atomic-bool/counter/threshold
  fields or the decision logic itself.
- Remove the buffering readonly getter end-to-end (jsi-interfaces.ts's
  IAudioFileSourceNode, the AudioFileSourceNodeHostObject JSI getter, and
  AudioFileSourceNode.ts's isBuffering()) since nothing consumed it — the
  design is event-driven (onWaiting/onPlaying), not poll-based. Kept
  AudioFileSourceNode::isBuffering() (C++-only, not JS-exposed) for tests.
- Tightened the JSDoc comments added in Audio/types.ts (and matching ones in
  events/types.ts) down to one line each.
- Replaced AudioFileSourceNodeTest.cpp (which drove the debounce logic
  through a test-only protected hook on the node) with
  BufferingStateDispatcherTest.cpp, testing the extracted class directly —
  same 7 cases, now against the class that actually owns the logic.

Verified: yarn typecheck, lint:js, lint:cpp, format:check:common all clean;
full C++ suite passes (394 tests, same count as before — 7
BufferingStateDispatcherTest cases replacing the 7 removed
AudioFileSourceNodeTest cases).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@gavrichards

Copy link
Copy Markdown
Contributor Author

Addressed all four in f876ced:

  • Extracted the debounce/dispatch logic into BufferingStateDispatcher (mirrors PositionChangedDispatcher) — AudioFileSourceNode now just calls bufferingStateDispatcher_.advance(hasData, framesToProcess) at each call site and no longer owns the EventCaller/atomic/counter/threshold fields or the decision logic itself.
  • Removed the buffering poll-style getter end-to-end (jsi-interfaces.ts, the HostObject JSI getter, and AudioFileSourceNode.ts's isBuffering()) since nothing consumed it — this is event-driven via onWaiting/onPlaying, not polled. Kept a C++-only isBuffering() on the node (not JS-exposed) purely so BufferingStateDispatcherTest.cpp can assert on it.
  • Tightened the comments in Audio/types.ts (and the matching one in events/types.ts) down to one line each.
  • Replaced AudioFileSourceNodeTest.cpp (which drove the old logic through a test-only protected hook) with BufferingStateDispatcherTest.cpp, testing the extracted class directly with the same 7 cases.

Rebased onto the latest branch tip and re-verified: typecheck/lint/format-check/cpplint all clean, full C++ suite passes (394 tests).

@mdydek
mdydek merged commit d81131a into software-mansion:main Aug 20, 2026
12 checks passed
@gavrichards
gavrichards deleted the feat/audio-tag-buffering-state branch August 20, 2026 12:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature New user-facing features or major capabilities

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants