Repository navigation
Streams, every frame from outside #215
Description
Plan 2026-10-13, closed 2026-10-30, shipped.
Goal
Three pull requests by Gohnnyman (#138 a backlight effect and VideoService, #139 USB video capture, #140 backlight wiring) arrived the same week TouchMyLight announced an FSEQ player for a 20 × 20 × 20 cube. Read one by one they look like unrelated features; read from the architecture they are one thing, a stream of frames arriving from outside the device. The plan modeled that once, by the textbook, so cameras, video, files, the network and the browser become combinations of a few parts, with MoonLight building the groundwork and contributors adding sources in small pull requests.
Outcome
The groundwork (G1 to G7) and every contribution but the color trim shipped: light-list placement and Network Receive's mapped_by, the Picture service and Picture effect, sender mapping that skips the board's mapping, the channel-data sink, a contributor page, the FSEQ player, border sampling, USB capture on the P4 and the backlight wiring. Measuring Network Receive on real boards added its receive task and counters, Network Send's paced packets, two-board scenarios and the receive defaults.
Decisions
- Two families, not one stream module: channel data (the pixel protocols, mapped by the sender) and pictures (width × height, sampled per light), as TouchDesigner separates channel and texture operators. One module for every transport was the first draft, rejected since it flattens the protocols' universes, discovery, sync and priorities.
- A stream's consumer is an effect in a layer, so it composes with effects, blend modes and modifiers; WLED's realtime override, which replaces effects, was rejected.
- The FSEQ player is a sibling of Network Receive on one channel-data sink, not a decoder in a generic stream source (2026-10-14): FSEQ is recorded channel data.
- Sender mapping through the layout's positions, not through the inverse of the layer's mapping, which cancelled the static modifiers.
- Core builds the groundwork, contributors add sources on top, each a small pull request of one topic.
- The browser authors and bakes, the device runs: what must run with no browser open belongs on the device.
- Shapes left out: a backlight effect of its own, PPM as a user format, HDR curves, video decoding beyond MJPEG, FSEQ extras up front, FPP MultiSync before MoonLight's own clock sync; the reasons are in the design study.
Where it lives
| What | Home |
|---|---|
The two families, mapped_by, the effect as consumer, inverse mapping |
MoonLight, frames from outside |
| The three axes, live and recorded, browser and device, the tools, WLED's attempts, shapes left out | Design study: streams |
| Adding a source of either family | Adding a picture or channel-data source |
| The sources' cards | Network Receive, FSEQ player, Picture effect, Picture service, Network Send |
| The measurements | Performance, Network Receive |
| Scenarios over several boards | Testing, several boards |
Steps
- G1 light-list placement (
Layer::placeByLayout) - G2 Network Receive's
mapped_by - G3 the Picture service, from feature: Follow the screen with the lights around it #138's VideoService
- G4 the Picture effect, from feature: Follow the screen with the lights around it #138's sampler
- G5 sender mapping skips the board's mapping (
Layer::wiringSource) - G6 the channel-data sink
- G7 the contributor page
- Backlight wiring (feature: Say where a backlight strand enters and which way it runs #140, feature: Hyperion-like backlight strand #144), Gohnnyman
- FSEQ playback (Play FSEQ shows from any mount, flash or SD alike #148), TouchMyLight
- Border sampling (feature: edgeDepth, smoothing and snapAbove added to PictureEffect #149), Gohnnyman
- USB capture on the P4 (feature: Groundwork for USB capture as a Picture service source #151, feature: the hardware-free half of USB video capture #152, feature: the hardware half of USB video capture #153), Gohnnyman
- Color trim (feature: Capture video from a USB grabber on the ESP32-P4 #139's second commit): continues as feature: Color balance #212
- Recording the drivers' output to FSEQ on SD, with clock sync: its card
Shipped in
#143, #144, #148, #149, #150, #151, #152, #153, #154, #155, #157.
Still open
- Color trim: feature: Color balance #212, its design (gains in the fixture profile, folded into per-channel tables) in the review there.
- Record the drivers' output to FSEQ, with clock sync
- E1.31 multicast never arrives
- The live runner restores every board when one fails mid-run
- The rest of the plan's measuring backlog was one-time observations, left out of the Project.
- The picture sources still to come (GIF, the browser tab, HDMI out): the Video in, LED walls out plan.
Activity