feat: buffering/loading state for AudioFileSourceNode and <Audio> - #1243
Conversation
Adds the buffering/stall signal identified as missing during aiirmobile's react-native-audio-api evaluation: the audio-tag state model previously exposed only 'idle' | 'playing' | 'paused', with no way to tell a genuine network/decoder stall apart from normal playback, and no onWaiting/onStalled- style event. - Native (shared C++ core, both platforms): detects render-thread frame starvation directly in AudioFileSourceNode::processDecodedOutput() (the single choke point both direct and MediaElementAudioSourceNode-routed playback go through), debounced against normal decode-ahead jitter via a new ON_BUFFERING_STATE_DEBOUNCE_INTERVAL (150ms) threshold. Recovery is reported immediately, with no symmetric debounce. pause()/disable() reset the state so a deliberate pause never leaves a stale 'buffering' reading. - New AudioEvent::BUFFERING_STATE_CHANGE + BoolValuePayload, wired through the existing EventCaller/HostObject/JsEnumParser plumbing exactly like the other AudioFileSourceNode events (onPositionChanged, onEnded). - JS: AudioTagPlaybackState gains a 'buffering' sub-state (derived, not a parallel piece of state, so a stall starting/ending doesn't tear down the effect watching it), and <Audio> gains onWaiting/onPlaying props mirroring the HTML <audio>/<video> spec events of the same name. Wired for both the native (Audio.tsx) and web (Audio.web.tsx) implementations. - New AudioFileSourceNodeTest.cpp covers the debounce state machine directly (crossing/not crossing threshold, immediate recovery, no-op without a listener, intermittent-jitter not tripping it, pause() resetting stale state) rather than racing the real decoder daemon thread for genuine starvation. Verified: yarn typecheck, yarn lint:js, yarn lint:cpp, yarn format:check:common all clean; full C++ suite (394 tests, was 387) passes including the 7 new tests. Not yet verified: on a real device against real network stalls — the debounce threshold (150ms) is a starting point, not tuned against real-world Icecast/ HLS stall behavior. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
android/.../AudioEvent.kt is a hand-maintained mirror of AudioEvent.h, crossed via .ordinal at the JNI boundary (see PlaybackNotification.kt) rather than referenced directly the way the iOS .mm files can. Missed this when adding BUFFERING_STATE_CHANGE — caught by CI's check-audio-enum-sync job. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
WPT non-regression comparisonPASS — no regressions · 0 improved section(s) · overall 2632 → 2632 (0) Unchanged sections (28)
Baseline: Workflow run · this comment is updated on every push. |
|
The
Happy to dig further if a re-run reproduces it, but based on the above I'd guess a runner hiccup. |
|
@gavrichards yes, the CI is certainly broken, we are aware of that, obviously your changes did not change any files connected with wpt tests |
…sed poll getter Per @mdydek's review on software-mansion#1243: - Extract the buffering-state debounce/dispatch logic out of AudioFileSourceNode into its own BufferingStateDispatcher class, mirroring PositionChangedDispatcher — AudioFileSourceNode now just calls bufferingStateDispatcher_.advance(hasData, framesToProcess) at each call site and no longer owns the EventCaller/atomic-bool/counter/threshold fields or the decision logic itself. - Remove the buffering readonly getter end-to-end (jsi-interfaces.ts's IAudioFileSourceNode, the AudioFileSourceNodeHostObject JSI getter, and AudioFileSourceNode.ts's isBuffering()) since nothing consumed it — the design is event-driven (onWaiting/onPlaying), not poll-based. Kept AudioFileSourceNode::isBuffering() (C++-only, not JS-exposed) for tests. - Tightened the JSDoc comments added in Audio/types.ts (and matching ones in events/types.ts) down to one line each. - Replaced AudioFileSourceNodeTest.cpp (which drove the debounce logic through a test-only protected hook on the node) with BufferingStateDispatcherTest.cpp, testing the extracted class directly — same 7 cases, now against the class that actually owns the logic. Verified: yarn typecheck, lint:js, lint:cpp, format:check:common all clean; full C++ suite passes (394 tests, same count as before — 7 BufferingStateDispatcherTest cases replacing the 7 removed AudioFileSourceNodeTest cases). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
Addressed all four in f876ced:
Rebased onto the latest branch tip and re-verified: typecheck/lint/format-check/cpplint all clean, full C++ suite passes (394 tests). |
What
Adds a buffering/loading-state signal for
AudioFileSourceNodeand<Audio>. Today the audio-tag state model is onlyAudioTagPlaybackState = 'idle' | 'playing' | 'paused'— there's no way to tell a genuine network/decoder stall apart from normal playback, and noonWaiting/onStalled-style event. This surfaced while evaluating the library for a production radio app: both the buffering-spinner UI and a reconnect/stall-watchdog need this signal and currently have nothing to build on.How
AudioFileSourceNode::processDecodedOutput()— the point where the render thread already silently zeros the output buffer when no decoded chunk is available (!hasFreshChunk && pendingDecoderChunk_.size == 0). This is the single choke point both direct playback andMediaElementAudioSourceNode-routed (live-stream) playback go through, so both benefit without platform-specific code.ON_BUFFERING_STATE_DEBOUNCE_INTERVAL(150ms) threshold, accumulated across render quanta. Recovery is reported immediately with no symmetric debounce, since a UI wants to clear a spinner as soon as audio resumes.pause()/disable()reset the state so a deliberate pause never leaves a stalebufferingreading behind.AudioEvent::BUFFERING_STATE_CHANGE+BoolValuePayload, wired through the existingEventCaller/HostObject/JsEnumParserplumbing exactly like the otherAudioFileSourceNodeevents (onPositionChanged,onEnded) — no new architecture, same pattern throughout.AudioTagPlaybackStategains a'buffering'sub-state. It's derived from the existingplaybackState+ a separateisBufferingflag rather than folded into one state variable, so the effect subscribing to buffering events doesn't tear itself down and re-subscribe every time a stall starts or ends.<Audio>gainsonWaiting/onPlayingprops, mirroring the HTML<audio>/<video>spec events of the same name (waiting= stopped due to lack of data,playing= resumed after being paused/waiting) — chosen deliberately to fit the library's existing HTML-media-flavored event naming (onLoadStart,onLoad,onEnded,onPlay,onPause) rather than inventing new terms. Wired for both the native (Audio.tsx) and web (Audio.web.tsx) implementations — the web variant gets it almost for free since the underlying<audio>DOM element already fireswaiting/playingnatively, which was a useful sanity check that the naming choice lines up with existing browser semantics.Testing
AudioFileSourceNodeTest.cpp(7 tests) covers the debounce state machine directly — crossing/not crossing the threshold, immediate recovery, no-op when no listener is registered, intermittent jitter not tripping it, andpause()resetting stale state — rather than racing the real decoder daemon thread to force genuine starvation deterministically.yarn typecheck,yarn lint:js,yarn lint:cpp,yarn format:check:commonall clean.Not yet verified: on a real device against a genuine network stall. The 150ms debounce threshold is a reasoned starting point (long enough to skip normal decode-ahead jitter, short enough to feel responsive), not one tuned against real-world Icecast/HLS stall behavior — happy to adjust based on review or if it needs to be configurable.