Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/selkies/components/baseimages.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ graph TD
SELKIES -->|starts in process| PF[pixelflux Wayland compositor, socket wayland-1]
DE -->|waits for wayland-1| LABWC[labwc or a full DE, exposes wayland-0]
LABWC --> APP[autostart application]
NGINX -->|3000 / 3001| WEB[web client, /websocket proxy, /files, /pelorus]
NGINX -->|3000 / 3001| WEB[web client, /api proxy, /pelorus]
```

At startup a chain of one shot init scripts configures everything from environment variables: Nginx substitution (ports, auth, subfolder, title), Wayland or X11 mode selection, first run copy of the autostart and menu defaults into `/config`, hardening (the `HARDEN_*` and `DISABLE_*` family), GPU detection and permission fixes, and gamepad device setup. Then the long running services above come up in dependency order.
Expand Down
4 changes: 2 additions & 2 deletions docs/selkies/components/selkies.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Selkies is the heart of the platform: a ground up, web native remote desktop pro

`selkies` is a Python asyncio application (console script `selkies`, installed in the baseimages at `/lsiopy/bin/selkies`) that owns the session:

- **Video**: drives [pixelflux](pixelflux.md) capture and encoding, and broadcasts encoded frames to all connected clients over the WebSocket, with per client backpressure (frame acknowledgements, RTT smoothing, and stall detection) so one slow viewer does not degrade the rest.
- **Video**: drives [pixelflux](pixelflux.md) capture and encoding, and broadcasts encoded frames to all connected clients over the WebSocket, with per client backpressure (frame acknowledgements, RTT smoothing, and stall detection) so one slow viewer does not degrade the rest. An opt in [WebRTC transport](../user-guide/webrtc.md) carries the same H.264 and Opus streams over UDP with a priority pacer and optional congestion control; clients can switch transports at runtime.
- **Audio out**: drives pcmflux, which captures the PulseAudio `output.monitor` source and Opus encodes at up to 320kbps.
- **Microphone in**: receives PCM from the browser and plays it into a virtual PulseAudio source (`SelkiesVirtualMic`) that session apps consume as a normal mic.
- **Input**: injects keyboard, mouse, touch, and scroll. On Wayland, injection goes through pixelflux's compositor APIs with an xkbcommon keymap (plus [waylandtyper](https://github.com/linuxserver/waylandtyper), our maintained fork of `wtype`, for unicode text batches). On X11, through pynput, xdotool, and python-xlib. Gamepads are handled by per slot Unix socket servers feeding the joystick interposer (below).
Expand Down Expand Up @@ -40,7 +40,7 @@ Both are preloaded automatically in the baseimages, and `NO_GAMEPAD=true` turns

| Port | What |
| --- | --- |
| 8082 | The data WebSocket (`SELKIES_PORT`; upstream default is 8081, the baseimages set 8082), proxied by Nginx at `/websocket` |
| 8082 | The Selkies server (`SELKIES_PORT`, upstream default is 8080, the baseimages set 8082 via `CUSTOM_WS_PORT`). Nginx proxies everything under `/api` to it: the data WebSocket at `/api/websockets`, WebRTC signaling at `/api/webrtc/signaling`, the transport switch, and the file browser at `/api/files/` |
| 8083 | Token control plane for secure sharing mode, never expose it |
| 3000 / 3001 | Nginx HTTP and HTTPS in front of everything ([baseimage](baseimages.md) territory) |

Expand Down
2 changes: 1 addition & 1 deletion docs/selkies/developer-guide/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ Gamepads bypass the display server entirely: Selkies serves the Linux joystick a

## The web layer

Nginx inside the container is the single front door: it serves the static client (a React dashboard over the `selkies-web-core` engine), proxies `/websocket` to the Selkies server, serves `/files` downloads with fancyindex, optionally enforces basic auth, applies the `SUBFOLDER` prefix, and proxies `/pelorus/` when the agent layer is on. The dashboard and the engine communicate over a documented `postMessage` API, which is the extension point for custom frontends.
Nginx inside the container is the single front door: it serves the static client (a React dashboard over the `selkies-web-core` engine), proxies `/api` (the data WebSocket, WebRTC signaling, and the `/api/files/` browser) to the Selkies server, optionally enforces basic auth, applies the `SUBFOLDER` prefix, and proxies `/pelorus/` when the agent layer is on. The dashboard and the engine communicate over a documented `postMessage` API, which is the extension point for custom frontends.

## Sharing and multi user

Expand Down
4 changes: 2 additions & 2 deletions docs/selkies/developer-guide/baseimage-internals.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ The baseimage Dockerfile is a multi stage build assembling, onto a LinuxServer.i
- The **joystick interposer** (`/usr/lib/selkies_joystick_interposer.so`) and **fake udev** (`/opt/lib/libudev.so.1.0.0-fake`) compiled from the Selkies addons
- Nginx with fancyindex, PulseAudio, mesa and VA-API userspace, Vulkan loaders, all system locales, proot-apps, Docker in Docker machinery, and passwordless sudo for `abc`

Baked ENV defaults worth knowing: `HOME=/config`, `DISPLAY=:1`, `TITLE=Selkies`, `SELKIES_ENCODER="x264enc,jpeg"`, `START_DOCKER=true`, `DISABLE_ZINK=false`, `DISABLE_DRI3=false`, `NVIDIA_DRIVER_CAPABILITIES=all`, and the interposer path in `SELKIES_INTERPOSER`.
Baked ENV defaults worth knowing: `HOME=/config`, `DISPLAY=:1`, `TITLE=Selkies`, `SELKIES_ENCODER="h264enc,jpeg"`, `SELKIES_ENABLE_BASIC_AUTH=false`, `SELKIES_VIDEO_STREAMING_MODE=false`, `SELKIES_ALLOWED_ORIGINS="*"`, `START_DOCKER=true`, `DISABLE_ZINK=false`, `DISABLE_DRI3=false`, `NVIDIA_DRIVER_CAPABILITIES=all`, and the interposer path in `SELKIES_INTERPOSER`.

## Boot: the init chain

Expand All @@ -34,7 +34,7 @@ init-os-end

**`init-selkies-config`** is the big one:

- Chooses the mode: `PIXELFLUX_WAYLAND=true` selects labwc paths (`$HOME/.config/labwc`, `/defaults/autostart_wayland`, `/defaults/menu_wayland.xml`) and forces `SELKIES_SECOND_SCREEN=false`; otherwise Openbox paths.
- Chooses the mode: `PIXELFLUX_WAYLAND=true` selects labwc paths (`$HOME/.config/labwc`, `/defaults/autostart_wayland`, `/defaults/menu_wayland.xml`) otherwise Openbox paths. Multi monitor on Wayland is Selkies' call, not the init script's: at startup it probes the compositor, the labwc IPC socket or a KWin capability check, and enables the second screen only where it finds support. Both compositors carry LinuxServer patches for this, the labwc ones ship in the baseimage.
- First run copies of `autostart` and `menu.xml` into the config dir (persistent, user editable); `rc.xml` for labwc is regenerated from the template every boot.
- Recreates `$HOME/.XDG` as `XDG_RUNTIME_DIR` and clears stale PulseAudio state, so unclean shutdowns recover.
- Applies every hardening variable (permission stripping, sudoers corruption, menu and keybind editing, locking rc.xml and autostart when watchdog mode is on), the details are in the [Security guide](../user-guide/security.md).
Expand Down
2 changes: 1 addition & 1 deletion docs/selkies/developer-guide/protocol.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# The Streaming Protocol

The wire protocol between the Selkies server and the web client, for anyone implementing a client, embedding the engine, or debugging with a network inspector. Everything rides **one WebSocket** (proxied at `<base>/websocket`), carrying a mix of binary frames and terse text messages. There is no negotiation dance: connect, receive settings, start receiving media.
The wire protocol between the Selkies server and the web client, for anyone implementing a client, embedding the engine, or debugging with a network inspector. Everything rides **one WebSocket** (proxied at `<base>/api/websockets`), carrying a mix of binary frames and terse text messages. In the opt in [WebRTC mode](../user-guide/webrtc.md) the same text messages travel over a data channel and media over RTP, with signaling at `<base>/api/webrtc/signaling`. There is no negotiation dance: connect, receive settings, start receiving media.

## Connection and roles

Expand Down
2 changes: 1 addition & 1 deletion docs/selkies/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ Selkies is a ground up, web native remote desktop protocol designed to replace l

1. **Hybrid protocol.** Damage tracking like VNC, video codecs like a streaming service. The screen is divided into horizontal stripes, only changed stripes are captured and encoded, and each stripe can be processed on a separate CPU core in parallel.
2. **Paint over quality.** H.264 handles fluid motion, and once motion stops the server repaints the static screen at high quality so text stays crisp. With FullColor 4:4:4 H.264 the painted over result is visually indistinguishable from a lossless image. A JPEG encoder remains available for older browsers that cannot decode video frames at all.
3. **WebSockets, not WebRTC.** Frames are delivered over a WebSocket connection and decoded in the browser with WebCodecs. This avoids WebRTC negotiation complexity, works cleanly through reverse proxies, and gives the server precise control over pacing and backpressure.
3. **WebSockets by default, WebRTC when you need it.** Frames are delivered over a WebSocket connection and decoded in the browser with WebCodecs. This avoids WebRTC negotiation complexity, works cleanly through reverse proxies, and gives the server precise control over pacing and backpressure. For lossy or high latency links an opt in [WebRTC transport](user-guide/webrtc.md) carries the same stream over UDP with congestion control, and users can switch between the two at runtime.
4. **Zero copy on Wayland.** In the current generation the display server is a virtual Wayland compositor built on [Smithay](https://github.com/Smithay/smithay). The framebuffer can live directly on a GPU, and frames are passed as DMA-BUF handles straight to the hardware encoder (VAAPI or NVENC) without a round trip through system RAM.
5. **Everything in one container.** Compositor, application, streaming server, audio, and web server all run inside a single OCI container built on `docker-baseimage-selkies`, managed by the s6 init system.

Expand Down
1 change: 1 addition & 0 deletions docs/selkies/user-guide/.pages
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,5 @@ nav:
- Installing Applications: installing-apps.md
- Security and Hardening: security.md
- Reverse Proxy: reverse-proxy.md
- WebRTC Transport: webrtc.md
- Troubleshooting: troubleshooting.md
Loading