Skip to content

fix: unblock a real OpenPlay cast — discovery, encoder config, capture caps - #50

Open
Developer1010x wants to merge 3 commits into
masterfrom
fix/openplay-real-cast-blockers
Open

Developer1010x wants to merge 3 commits into
masterfrom
fix/openplay-real-cast-blockers

Conversation

@Developer1010x

Copy link
Copy Markdown
Owner

Found by running both binaries and actually casting. None of this is reachable from the test suite: the sender path needs a desktop portal and a human at the consent prompt, which is exactly why webrtc_loopback.rs uses a videotestsrc stand-in.

Three defects, each hidden behind the one before it.

1. The sender discarded every mDNS update

add_receiver was guarded by if !contains_key, so the first advertisement for a name won permanently. mDNS resolves addresses incrementally — the first event routinely carries only link-local addresses, and the routable ones arrive in later events for the same name.

The sender therefore held a set of fe80:: addresses, none dialable, because nothing advertises a zone index and one cannot be reconstructed (address.rs says so in its own doc comment):

Connecting to receiver url=wss://[fe80::3cec:9aff:fea6:cac0]:7290/
  ... IO error: Invalid argument (os error 22)
  ... and the same for all six link-local addresses
Could not reach the receiver on any advertised address

Worth noting this is the other half of #48. That change made the sender try every address instead of only the first; this one makes sure the reachable addresses are in the list to try.

Each event carries the full set ServiceInfo::get_addresses knows rather than a delta, so the newest is the most complete and replacing outright is right. The key is the service fullname and is stable across updates, so a row keeps its list position and a selection on it survives.

2. A hardcoded rate-control="cbr" brought the cast down

The va plugin builds that enum per driver — the type is literally GstVaEncoderRateControl_H264_LP_renderD128 — and populates it from what the hardware reports. A Tiger Lake iGPU offers only cqp on vah264lpenc:

rate-control : Enum "GstVaEncoderRateControl_H264_LP_renderD128" Default: 16, "cqp"
                  (16): cqp - Constant Quantizer

set_property_from_str panics on a nick the enum does not define, so the cast died immediately after the user pressed Allow:

property 'rate-control' of type 'GstVaH264LPEnc' can't be set from string 'cbr'

probe_best_encoder could not have caught this. It instantiates each candidate and instantiation succeeds — configuring the element fails one step later. set_enum_property now picks the first nick the driver actually defines, and keeps the element's default if none match, so an unusual driver costs some quality instead of the whole cast.

3. Nothing converted the portal's frames for the encoder

The capture source fed the hardware encoder directly, with a fixed framerate=30/1 imposed on a portal stream that advertises a maximum rather than a fixed rate. The intersection was empty:

pipewiresrc0: stream error: no more input formats
streaming stopped, reason not-negotiated (-4)

videoconvert widens the accepted formats to everything raw, and videorate is what actually produces the fixed rate the capsfilter asks for — without it that filter is a demand the source cannot meet.

Verified

Cast between the two binaries on one machine, driven by hand:

sender:   Sender pipeline started / Negotiation needed — creating offer
          Local description ready kind=Offer bytes=795
          WebRTC connection state state=Connected
          Media session connected — streaming
receiver: SDP offer accepted — answering
          Local description ready kind=Answer bytes=621
          WebRTC connection state state=Connected
          Using hardware decoder decoder="vah264dec"
          Video decoding chain linked (RGBA appsink)

This is the first OpenPlay cast confirmed working end to end — the project has carried a note that it never had been.

cargo test --all 323 passing / 0 failures; fmt and clippy -D warnings clean.

Known remaining defect

The displayed image is uniformly green. Connection, negotiation and decode are all correct, so this is a colour or buffer-format problem in the capture-to-encoder hop — most likely the portal handing out DMABuf that a CPU videoconvert cannot import, given pipewiresrc is built with always-copy=false. Being tracked separately; this PR is what gets a cast to run at all.

Also still unverified between two machines.

🤖 Generated with Claude Code

Developer1011x and others added 3 commits September 14, 2026 20:07
…e caps

Three separate defects, each hidden behind the one before it. Found by
running both binaries and casting, which no test covers: the sender path
needs a desktop portal and a human at the consent prompt.

**1. The sender discarded every mDNS update.** `add_receiver` was guarded
by `if !contains_key`, so the first advertisement for a name won forever.
mDNS resolves a service's addresses incrementally — the first event
routinely carries only link-local addresses, with the routable ones
arriving in later events for the same name. The sender therefore held a
set of `fe80::` addresses, none of which is dialable, because nothing
advertises a zone index and one cannot be reconstructed:

    Connecting to receiver url=wss://[fe80::3cec:9aff:fea6:cac0]:7290/
    ... IO error: Invalid argument (os error 22)   (all six of them)

Each event carries the full set `get_addresses` knows rather than a delta,
so the newest is the most complete and replacing outright is correct. The
key is the service fullname and does not change between updates, so a row
keeps its position and a selection on it survives.

**2. A hardcoded `rate-control="cbr"` brought the cast down.** The `va`
plugin builds that enum per driver — the type is literally named
`GstVaEncoderRateControl_H264_LP_renderD128` — and populates it from what
the hardware reports. A Tiger Lake iGPU offers only `cqp` on
`vah264lpenc`, and `set_property_from_str` panics on a nick the enum does
not define:

    property 'rate-control' of type 'GstVaH264LPEnc' can't be set from string 'cbr'

`probe_best_encoder` could not have caught this. It instantiates each
candidate and instantiation succeeds; configuring the element fails one
step later. `set_enum_property` now picks the first nick the driver
actually defines and keeps the element's default if none match, so an
unusual driver costs quality rather than the whole cast.

**3. Nothing converted the portal's frames for the encoder.** The capture
source fed the hardware encoder directly while a fixed `framerate=30/1`
was imposed on a portal stream that advertises a maximum rather than a
fixed rate. The intersection was empty and the source gave up:

    pipewiresrc0: stream error: no more input formats
    streaming stopped, reason not-negotiated (-4)

`videoconvert` widens the accepted formats to everything raw, and
`videorate` is what actually produces the fixed rate the capsfilter asks
for — without it that filter is a demand the source cannot meet.

Verified by casting between the two binaries on one machine: offer/answer,
trickle ICE, `Connected` on both peers, hardware `vah264dec` on the
receiver and frames reaching its RGBA appsink. That is the first OpenPlay
cast confirmed working end to end.

Still wrong: the displayed image is uniformly green, which is a colour or
buffer-format problem in the capture-to-encoder hop rather than a
connection one. Tracked separately — this commit is what gets a cast to
run at all.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…on macOS

Two gaps, both of which meant nothing about macOS was actually checked.

**`query_primary_display_size` had no macOS implementation.** The branch was
an empty block with a comment, so every Mac fell through to the hardcoded
`(1920, 1080)` and capture was configured for a resolution the machine may
not have. It now asks CoreGraphics.

It asks for *pixels*, not points. `CGDisplayPixelsWide` reports points,
which on any Retina display is about half the real width — encoding at that
size would quietly halve the resolution of every Mac cast while looking
like it worked. `CGDisplayModeGetPixelWidth` reports backing pixels and is
tried first; the points API remains as a fallback for when the mode cannot
be read at all.

The six symbols are declared directly rather than through a wrapper crate:
this is the only place the framework is needed, and six extern declarations
are less to carry than a dependency.

**Nothing compiled the casting crates for macOS.** `cross-platform-check`
excludes `openplay-pipeline`, `-sender` and `-receiver` because they need
GStreamer, which the runner did not have — so the three crates that a Mac
cast actually depends on were built on no platform but Linux. That is the
same shape as the Windows break in #25, which survived precisely because
nothing compiled the crate for Windows.

The macOS runner now installs GStreamer and checks those three as well.
Windows keeps the narrow check: nobody has claimed a working cast there,
so paying the install cost would buy less.

Compiling is not casting. This job cannot press a consent prompt or answer
a screen-capture dialog, and the macOS capture path — `screencapturesrc`
with an `avfvideosrc` fallback — remains untested against real hardware.
What it does catch is a crate silently ceasing to build for the platform.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A uniformly green picture at the receiver is either a blank capture or a
mangled conversion, and nothing downstream of the source can tell those
apart — the encode/decode chain is provably fine on a videotestsrc.

With OPENPLAY_DUMP_CAPTURE set, tap the capture source pad and log the
non-zero byte count, max and mean of the first five buffers, before any
other element touches them. Off by default and costs nothing when unset.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants