Add experimental AV1 WebRTC support

https://bugs.webkit.org/show_bug.cgi?id=248172
rdar://problem/102579632

Reviewed by Eric Carlson.

Add libaom to libwebrtc third party.
Make it compile on macOS/iOS.
Enable AV1 support in libwebrtc.
Expose API to turn on/off AV1 in libwebrtc.
Expose a new experimental AV1 flag and turn on/off aom in libwebrtc based on it..
Coverd by added test.

* LayoutTests/imported/w3c/web-platform-tests/media-capabilities/decodingInfo.webrtc-expected.txt:
* LayoutTests/imported/w3c/web-platform-tests/media-capabilities/encodingInfo.webrtc-expected.txt:
* LayoutTests/webrtc/video-av1-expected.txt: Added.
* LayoutTests/webrtc/video-av1.html: Added.
* Source/ThirdParty/libwebrtc/Configurations/libaom.xcconfig: Added.
* Source/ThirdParty/libwebrtc/Configurations/libwebrtc.iOS.exp:
* Source/ThirdParty/libwebrtc/Configurations/libwebrtc.iOSsim.exp:
* Source/ThirdParty/libwebrtc/Configurations/libwebrtc.mac.exp:
* Source/ThirdParty/libwebrtc/Configurations/libwebrtc.xcconfig:
* Source/ThirdParty/libwebrtc/Source/third_party/libaom: Added.
(WebCore::Layout::BlockLayoutState::floatingState):
(WebCore::Layout::BlockLayoutState::BlockLayoutState):
(WebCore::Layout::BlockLayoutState::setLineClamp): Deleted.
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitDecoder.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitDecoder.mm:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitEncoder.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitEncoder.mm:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitUtilities.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/api/video_codec/RTCVideoCodecConstants.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/api/video_codec/RTCVideoCodecConstants.mm:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/api/video_codec/RTCVideoDecoderAV1.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/api/video_codec/RTCVideoDecoderAV1.mm:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/api/video_codec/RTCVideoEncoderAV1.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/api/video_codec/RTCVideoEncoderAV1.mm:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCDefaultVideoDecoderFactory.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCDefaultVideoDecoderFactory.m:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCDefaultVideoEncoderFactory.h:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCDefaultVideoEncoderFactory.m:
* Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/native/src/objc_video_encoder_factory.mm:
* Source/ThirdParty/libwebrtc/libwebrtc.xcodeproj/project.pbxproj:
* Source/WTF/Scripts/Preferences/WebPreferencesExperimental.yaml:
* Source/WebCore/page/Page.cpp:
* Source/WebCore/platform/mediastream/WebRTCProvider.cpp:
* Source/WebCore/platform/mediastream/WebRTCProvider.h:
* Source/WebCore/platform/mediastream/libwebrtc/LibWebRTCProvider.cpp:
(WebCore::LibWebRTCProvider::videoDecodingCapabilitiesOverride):
(WebCore::LibWebRTCProvider::videoEncodingCapabilitiesOverride):
* Source/WebCore/platform/mediastream/libwebrtc/LibWebRTCProviderCocoa.cpp:

Canonical link: https://commits.webkit.org/257228@main

Author: youennf

LayoutTests/imported/w3c/web-platform-tests/media-capabilities/decodingInfo.webrtc-expected.txt

@@ -23,4 +23,5 @@ PASS Test that decodingInfo returns supported true for the codec video/H265 retu
 PASS Test that decodingInfo returns supported true for the codec video/VP8 returned by RTCRtpReceiver.getCapabilities()
 PASS Test that decodingInfo returns supported true for the codec video/VP9; profile-id=0 returned by RTCRtpReceiver.getCapabilities()
 PASS Test that decodingInfo returns supported true for the codec video/VP9; profile-id=2 returned by RTCRtpReceiver.getCapabilities()
+PASS Test that decodingInfo returns supported true for the codec video/AV1 returned by RTCRtpReceiver.getCapabilities()
 

LayoutTests/imported/w3c/web-platform-tests/media-capabilities/encodingInfo.webrtc-expected.txt

@@ -23,4 +23,5 @@ PASS Test that encodingInfo returns supported true for the codec video/H265 retu
 PASS Test that encodingInfo returns supported true for the codec video/VP8 returned by RTCRtpSender.getCapabilities()
 PASS Test that encodingInfo returns supported true for the codec video/VP9; profile-id=0 returned by RTCRtpSender.getCapabilities()
 PASS Test that encodingInfo returns supported true for the codec video/VP9; profile-id=2 returned by RTCRtpSender.getCapabilities()
+PASS Test that encodingInfo returns supported true for the codec video/AV1 returned by RTCRtpSender.getCapabilities()
 

LayoutTests/platform/glib/TestExpectations

@@ -1470,6 +1470,8 @@ webkit.org/b/235885 webrtc/datachannel/datachannel-stats.html [ Skip ]
 webkit.org/b/235885 webrtc/datachannel/getStats-no-prflx-remote-candidate.html [ Skip ]
 webkit.org/b/235885 fast/mediastream/RTCPeerConnection-statsSelector.html [ Skip ]
 
+webrtc/video-av1.html [ Skip ]
+
 # GStreamer's DTLS agent currently generates RSA certificates only. DTLS 1.2 is not supported yet (AFAIK).
 webrtc/datachannel/dtls10.html [ Failure ]
 

LayoutTests/webrtc/video-av1-expected.txt

@@ -0,0 +1,12 @@
+Video should be running, go to black and running.
+Following, should be a snapshot of the video, a black frame and a snapshot of the video.
+
+
+PASS Verify AV1 activation
+PASS Setting video exchange
+PASS Ensuring connection state is connected
+PASS Track is enabled, video should not be black
+PASS Track is disabled, video should be black
+PASS If disabled, black frames should still be coming
+PASS Track is enabled, video should not be black 2
+

LayoutTests/webrtc/video-av1.html

@@ -0,0 +1,127 @@
+<!doctype html>
+<html>
+    <head>
+        <meta charset="utf-8">
+        <title>Testing muting video</title>
+        <script src="../resources/testharness.js"></script>
+        <script src="../resources/testharnessreport.js"></script>
+    </head>
+    <body>
+        <div>Video should be running, go to black and running.</div>
+        <div>Following, should be a snapshot of the video, a black frame and a snapshot of the video.</div>
+        <video id="video" autoplay playsInline width="320" height="240"></video>
+        <canvas id="canvas1" width="320" height="240"></canvas>
+        <canvas id="canvas2" width="320" height="240"></canvas>
+        <canvas id="canvas3" width="320" height="240"></canvas>
+        <script src ="routines.js"></script>
+        <script>
+promise_test(async (test) => {
+    const codecs = RTCRtpSender.getCapabilities("video").codecs;
+    assert_true(codecs.some((codec) => { return codec.mimeType.indexOf("AV1") >= 0 }), "AV1 is listed as a codec");
+
+    codecs.forEach((codec) => {
+       if (codec.mimeType.indexOf("AV1") >=0)
+           assert_equals(codec.sdpFmtpLine, undefined, "AV1 codec.sdpFmtpLine is undefined");
+    });
+
+    const pc = new RTCPeerConnection();
+    pc.addTransceiver("video");
+    const description = await pc.createOffer();
+    pc.close();
+    assert_true(description.sdp.indexOf("AV1") !== -1, "AV1 codec is in the SDP");
+}, "Verify AV1 activation")
+
+var track;
+var remoteTrack;
+var receivingConnection;
+promise_test((test) => {
+    return navigator.mediaDevices.getUserMedia({video: {width: 320, height: 240, facingMode: "environment"}}).then((localStream) => {
+        return new Promise((resolve, reject) => {
+            track = localStream.getVideoTracks()[0];
+
+            createConnections((firstConnection) => {
+                firstConnection.addTrack(track, localStream);
+                firstConnection.getTransceivers()[0].setCodecPreferences([{mimeType: "video/AV1", clockRate: 90000}]);
+            }, (secondConnection) => {
+                receivingConnection = secondConnection;
+                secondConnection.ontrack = (trackEvent) => {
+                    remoteTrack = trackEvent.track;
+                    resolve(trackEvent.streams[0]);
+                };
+            }, { observeOffer : test.step_func((offer) => {
+                assert_false(offer.sdp.includes("H264"), "No H264");
+                assert_true(offer.sdp.includes("AV1"), "AV1");
+                return offer;
+            })
+            });
+            setTimeout(() => reject("Test timed out"), 5000);
+        });
+    }).then((remoteStream) => {
+        video.srcObject = remoteStream;
+        return video.play();
+    });
+}, "Setting video exchange");
+
+promise_test(() => {
+    if (receivingConnection.connectionState === "connected")
+        return Promise.resolve();
+    return new Promise((resolve, reject) => {
+        receivingConnection.onconnectionstatechange = () => {
+            if (receivingConnection.connectionState === "connected")
+                resolve();
+        };
+        setTimeout(() => reject("Test timed out"), 5000);
+    });
+}, "Ensuring connection state is connected");
+
+promise_test((test) => {
+    return checkVideoBlack(false, canvas1, video);
+}, "Track is enabled, video should not be black");
+
+promise_test((test) => {
+    track.enabled = false;
+    return checkVideoBlack(true, canvas2, video);
+}, "Track is disabled, video should be black");
+
+async function getInboundRTPStatsNumberOfDecodedFrames(connection)
+{
+    var report = await connection.getStats();
+    var framesDecoded;
+    report.forEach((statItem) => {
+        if (statItem.type === "inbound-rtp")
+            framesDecoded = statItem.framesDecoded;
+    });
+    return framesDecoded;
+}
+
+async function testFrameDecodedIncreased(connection, count, previousFramesNumber)
+{
+    if (previousFramesNumber === undefined) {
+        let number = await getInboundRTPStatsNumberOfDecodedFrames(connection);
+        await waitFor(1000);
+        return testFrameDecodedIncreased(connection, 0, number);
+    }
+
+    var number = await getInboundRTPStatsNumberOfDecodedFrames(connection);
+    if (previousFramesNumber && number > previousFramesNumber)
+        return;
+
+    if (count >= 20)
+        return Promise.reject("test increasing frame encoded timed out");
+
+    await waitFor(1000);
+    return testFrameDecodedIncreased(connection, ++count, previousFramesNumber);
+}
+
+promise_test((test) => {
+    return testFrameDecodedIncreased(receivingConnection);
+}, "If disabled, black frames should still be coming");
+
+promise_test((test) => {
+    track.enabled = true;
+    return checkVideoBlack(false, canvas2, video);
+}, "Track is enabled, video should not be black 2");
+
+        </script>
+    </body>
+</html>

Source/ThirdParty/libwebrtc/Configurations/libaom.xcconfig

@@ -0,0 +1,37 @@
+PRODUCT_NAME = aom;
+
+CLANG_WARN_BOOL_CONVERSION = YES;
+CLANG_WARN_ENUM_CONVERSION = YES;
+CLANG_WARN_INT_CONVERSION = YES;
+CLANG_WARN_STRICT_PROTOTYPES = NO;
+CLANG_WARN_IMPLICIT_FUNCTION_DECLARATION = NO;
+GCC_WARN_64_TO_32_BIT_CONVERSION = NO;
+COMBINE_HIDPI_IMAGES = NO;
+ENABLE_STRICT_OBJC_MSGSEND = YES;
+GCC_WARN_UNUSED_FUNCTION = NO;
+
+WARNING_CFLAGS = -Wno-implicit-function-declaration -Wno-strict-prototypes;
+
+// FIXME: Enable x86 optimization
+HEADER_SEARCH_PATHS[arch=x86_64] = Source/third_party/libaom/source/config/linux/generic Source/third_party/libaom/source/libaom Source/third_party/libaom/source/config;
+HEADER_SEARCH_PATHS[arch=arm64*] = Source/third_party/libaom/source/config/ios/arm64 Source/third_party/libaom/source/libaom Source/third_party/libaom/source/config;
+
+INSTALL_PATH = $(INSTALL_PATH_PREFIX)$(WK_LIBRARY_INSTALL_PATH);
+PUBLIC_HEADERS_FOLDER_PATH = $(INSTALL_PATH_PREFIX)$(WK_LIBRARY_HEADERS_FOLDER_PATH)/libwebrtc;
+USE_HEADERMAP = NO;
+
+GCC_PREPROCESSOR_DEFINITIONS = $(inherited) CONFIG_REALTIME_ONLY=1 WEBRTC_WEBKIT_BUILD CONFIG_TUNE_VMAF=0
+
+GCC_PREPROCESSOR_DEFINITIONS[sdk=iphonesimulator*] = $(inherited);
+GCC_PREPROCESSOR_DEFINITIONS[sdk=macosx*] = $(inherited) $(GCC_PREPROCESSOR_DEFINITIONS_$(WK_IS_CATALYST))
+GCC_PREPROCESSOR_DEFINITIONS_YES = WEBRTC_WEBKIT_MAC_CATALIST
+
+ARM_FILES = *_neon.c arm_cpudetect.c *_arm.c
+X86_FILES = *_sse2.c *_ssse3.c *_sse4.c *_avx2.c *_avx2.cc *_avx.c *.asm
+
+EXCLUDED_SOURCE_FILE_NAMES[arch=x86_64] = $(ARM_FILES) $(EXCLUDED_SOURCE_FILE_NAMES_$(WK_IS_CATALYST));
+EXCLUDED_SOURCE_FILE_NAMES_YES = *_sse4.c *_avx.c;
+EXCLUDED_SOURCE_FILE_NAMES[arch=arm64*] = $(X86_FILES) *_mmx.c
+EXCLUDED_SOURCE_FILE_NAMES[sdk=iphonesimulator*][arch=x86_64] = $(ARM_FILES) $(X86_FILES)
+
+OTHER_LDFLAGS = $(inherited) $(SOURCE_VERSION_LDFLAGS);

Source/ThirdParty/libwebrtc/Configurations/libwebrtc.iOS.exp

@@ -82,8 +82,8 @@ __ZN6webrtc27SessionDescriptionInterface16RemoveCandidatesERKNSt3__16vectorIN7cr
 __ZNK6webrtc21IceCandidateInterface10server_urlEv
 __ZN6webrtc18pixelBufferToFrameEP10__CVBuffer
 __ZN6webrtc20setApplicationStatusEb
-__ZN6webrtc26createWebKitDecoderFactoryENS_10WebKitH265ENS_9WebKitVP9ENS_12WebKitVP9VTBE
-__ZN6webrtc26createWebKitEncoderFactoryENS_10WebKitH265ENS_9WebKitVP9ENS_20WebKitH264LowLatencyE
+__ZN6webrtc26createWebKitDecoderFactoryENS_10WebKitH265ENS_9WebKitVP9ENS_12WebKitVP9VTBENS_9WebKitAv1E
+__ZN6webrtc26createWebKitEncoderFactoryENS_10WebKitH265ENS_9WebKitVP9ENS_20WebKitH264LowLatencyENS_9WebKitAv1E
 __ZN6webrtc29setH264HardwareEncoderAllowedEb
 __ZN6webrtc24registerWebKitVP8DecoderEv
 __ZN6webrtc24registerWebKitVP9DecoderEv

Source/ThirdParty/libwebrtc/Configurations/libwebrtc.iOSsim.exp

@@ -82,8 +82,8 @@ __ZN6webrtc27SessionDescriptionInterface16RemoveCandidatesERKNSt3__16vectorIN7cr
 __ZNK6webrtc21IceCandidateInterface10server_urlEv
 __ZN6webrtc18pixelBufferToFrameEP10__CVBuffer
 __ZN6webrtc20setApplicationStatusEb
-__ZN6webrtc26createWebKitDecoderFactoryENS_10WebKitH265ENS_9WebKitVP9ENS_12WebKitVP9VTBE
-__ZN6webrtc26createWebKitEncoderFactoryENS_10WebKitH265ENS_9WebKitVP9ENS_20WebKitH264LowLatencyE
+__ZN6webrtc26createWebKitDecoderFactoryENS_10WebKitH265ENS_9WebKitVP9ENS_12WebKitVP9VTBENS_9WebKitAv1E
+__ZN6webrtc26createWebKitEncoderFactoryENS_10WebKitH265ENS_9WebKitVP9ENS_20WebKitH264LowLatencyENS_9WebKitAv1E
 __ZN6webrtc29setH264HardwareEncoderAllowedEb
 __ZN6webrtc24registerWebKitVP8DecoderEv
 __ZN6webrtc24registerWebKitVP9DecoderEv

Source/ThirdParty/libwebrtc/Configurations/libwebrtc.mac.exp

@@ -82,8 +82,8 @@ __ZN6webrtc27SessionDescriptionInterface16RemoveCandidatesERKNSt3__16vectorIN7cr
 __ZNK6webrtc21IceCandidateInterface10server_urlEv
 __ZN6webrtc18pixelBufferToFrameEP10__CVBuffer
 __ZN6webrtc20setApplicationStatusEb
-__ZN6webrtc26createWebKitDecoderFactoryENS_10WebKitH265ENS_9WebKitVP9ENS_12WebKitVP9VTBE
-__ZN6webrtc26createWebKitEncoderFactoryENS_10WebKitH265ENS_9WebKitVP9ENS_20WebKitH264LowLatencyE
+__ZN6webrtc26createWebKitDecoderFactoryENS_10WebKitH265ENS_9WebKitVP9ENS_12WebKitVP9VTBENS_9WebKitAv1E
+__ZN6webrtc26createWebKitEncoderFactoryENS_10WebKitH265ENS_9WebKitVP9ENS_20WebKitH264LowLatencyENS_9WebKitAv1E
 __ZN6webrtc29setH264HardwareEncoderAllowedEb
 __ZN6webrtc24registerWebKitVP8DecoderEv
 __ZN6webrtc24registerWebKitVP9DecoderEv