Skip to content

[WebRTC] Queue frames till Video Frame Observer is ready#1690

Open
ganeshprasads wants to merge 1 commit into
WebPlatformForEmbedded:wpe-2.46from
ganeshprasads:wpe-2.46-webrtc-blackscreen
Open

[WebRTC] Queue frames till Video Frame Observer is ready#1690
ganeshprasads wants to merge 1 commit into
WebPlatformForEmbedded:wpe-2.46from
ganeshprasads:wpe-2.46-webrtc-blackscreen

Conversation

@ganeshprasads

@ganeshprasads ganeshprasads commented Jun 15, 2026

Copy link
Copy Markdown
https://bugs.webkit.org/show_bug.cgi?id=317111
Reviewed by: E. Ocaña González

Problem: WebRTC playback doesnot recover from a black screen and the issue is seen Intermittently

 Cause: The first video frame(s) (containing SPS/PPS/IDR) being dropped
 because libwebrtc starts delivering decoded frames before the downstream
 GStreamer pipeline's InternalSource registers as a VideoFrameObserver.

 Without first having the SPS/PPS and the first sync frame, the decoder
 cannot decode any subsequent delta frames.

 In Following code snippet from mediastream/gstreamer/GStreamerMediaStreamSource.cpp,
 frames received while m_isObserving is false might be dropped.

void videoFrameAvailable(VideoFrame& videoFrame, VideoFrameTimeMetadata) final
{
    if (!m_parent || !m_isObserving)
       return;

    updateFirstVideoSampleSeenFlag();

Change: While this patch does not address the root cause of delayed start of
 VideoFrameObserver, add a small frame buffer to hold frames
 (currently max size set to 30 frames) that arrive before any observer registers.
 deliver the frames when observer is registered
Note: From tests it is observed that only the first frame or 2 is actually needed to be queued.

* Source/WebCore/platform/mediastream/RealtimeMediaSource.cpp: (RealtimeMediaSource::videoFrameAvailable): Queue frames till observer are ready
* Source/WebCore/platform/mediastream/RealtimeMediaSource.h: (struct PendingVideoFrame): Added a buffer to hold the frames<!--EWS-Status-Bubble-Start-->

617daa5

Build-Tests Layout-Tests
❌ 🛠 wpe-246-amd64-build ❌ 🧪 wpe-246-amd64-layout
❌ 🛠 wpe-246-arm32-build ❌ 🧪 wpe-246-arm32-layout

    https://bugs.webkit.org/show_bug.cgi?id=317111
    Reviewed by: E. Ocaña González

    Problem : WebRTC playback doesnot recover from a black screen and the issue is seen Intermittently

     Cause: The first video frame(s) (containing SPS/PPS/IDR) being dropped
     because libwebrtc starts delivering decoded frames before the downstream
     GStreamer pipeline's InternalSource registers as a VideoFrameObserver.

     Without first having the SPS/PPS and the first sync frame , the decoder
     can not decode any subsequent delta frames.

     In Following code snippet from mediastream/gstreamer/GStreamerMediaStreamSource.cpp,
     frames received while m_isObserving is false might be dropped.

    void videoFrameAvailable(VideoFrame& videoFrame, VideoFrameTimeMetadata) final
    {
        if (!m_parent || !m_isObserving)
           return;

        updateFirstVideoSampleSeenFlag();

    Change : While this patch doesnot address the root cause of delayed start of
     VideoFrameObserver, add a small frame buffer to hold frames
     (currently max size set to 30 frames) that arrive before any observer registers.
     deliver the frames when observer is registered
    Note: From tests it is observed that only the first frame or 2 is actually needed to be queued.

    * Source/WebCore/platform/mediastream/RealtimeMediaSource.cpp:
      (RealtimeMediaSource::videoFrameAvailable) : Queue frames till
    observer is ready
    * Source/WebCore/platform/mediastream/RealtimeMediaSource.h:
      (struct PendingVideoFrame) : Added a buffer to hold the frames
@modeveci modeveci requested a review from eocanha June 15, 2026 09:52
@eocanha eocanha self-assigned this Jun 17, 2026
@eocanha eocanha added the upstream Related to an upstream bug (or should be at some point) label Jun 17, 2026
@eocanha

eocanha commented Jun 18, 2026

Copy link
Copy Markdown
Member

Ported PR #1689 (from wpe-2.38) upstream, with some modifications to avoid 4 layour test regressions, and submitted it as WebKit/WebKit#67460 for review there (on the https://bugs.webkit.org/show_bug.cgi?id=317111 upstream bug created by Ganesh).

When landed there, will backport it to 2.46 as well instead of this PR #1690.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

upstream Related to an upstream bug (or should be at some point) wpe-2.46

Development

Successfully merging this pull request may close these issues.

3 participants