feat(transport): WebRTC DataChannel transport (cloudflare SFU) by spomichter · Pull Request #2048 · dimensionalOS/dimos

spomichter · 2026-05-11T22:26:09Z

#2411 (ruthwik/hosted-teleop) is merged; main is merged into this branch. This PR turns its session plumbing into a first-class transport and deprecates HostedTeleopModule for the data planes.

Problem

DimOS has LCM, SHM, DDS, and ROS transports for local/LAN communication but no transport for internet-scale real-time data (NAT traversal, global edge routing). Hosted teleop required a dedicated HostedTeleopModule wrapper to bridge WebRTC DataChannels to DimOS streams.

Solution

A WebRTC DataChannel transport backed by Cloudflare Realtime, following the same pubsub abstraction as LCMTransport / SHMTransport. Hosted teleop becomes a one-line transport swap on a normal blueprint — no wrapper module:

teleop_hosted_go2_transport = unitree_go2_basic.transports(
    {("cmd_vel", Twist): CloudflareTransport("cmd_unreliable", TwistStamped)}
).global_config(viewer="none")

How to run it against ruthwik's hosted stack

Log in at https://teleop.dimensionalos.com → dashboard → + New Key (set a robot id). Copy the dtk_live_… key.
On the robot (robot identity is derived from the key — no robot id needed; credentials flow through the blueprint transport config, dimos run teleop-hosted-go2-transport --help lists all transports.broker.* fields):
```
dimos run teleop-hosted-go2-transport -o transports.broker.api_key=dtk_live_...
# or via env: TRANSPORTS__BROKER__API_KEY=dtk_live_... dimos run teleop-hosted-go2-transport
```
(The old TELEOP_API_KEY env fallback was removed when transport config moved into the blueprint config flow.)
Back in the dashboard the robot shows up — Connect (keyboard view). WASD sends LCM TwistStamped on cmd_unreliable; the transport decodes it straight onto the go2 cmd_vel stream. No HostedTeleopModule anywhere in the path.

Verify without credentials (no key, no robot)

# Transport semantics, config-override flow, pickling, provider lifecycle
# (in-memory MockProvider loopback):
uv run pytest dimos/protocol/pubsub/impl/webrtc -q

# WebRTC pubsub through the shared pubsub contract grid (same parametrized
# suite LCM/SHM/Zenoh/ROS run through):
uv run pytest dimos/protocol/pubsub/test_spec.py -q -k webrtc

# Config surface exposed to the CLI:
uv run dimos run teleop-hosted-go2-transport --help   # shows transports.broker.*

# Full wiring proof against the live broker — a fake key must produce
# "Broker session create failed: 401 Invalid robot credentials" in the logs,
# which means CLI -> blueprint config -> transport -> worker -> broker all work:
uv run dimos --simulation run teleop-hosted-go2-transport -o transports.broker.api_key=dtk_fake

# Benchmark the webrtc pubsub stack, keyless (WebrtcLoopback row — in-memory
# provider, ~2.4M msg/s stack ceiling; -s is required to see the matrix):
uv run pytest -s dimos/protocol/pubsub/benchmark/tool_benchmark.py -k webrtc

The full pubsub benchmark (uv run pytest -s dimos/protocol/pubsub/benchmark/tool_benchmark.py) also runs keyless for all local transports. The live WebrtcCloudflare row additionally self-registers when CF_TELEOP_APP_ID/CF_TELEOP_APP_SECRET are set (the app secret must never leave server-side environments) — for that row, run with DIMOS_BENCH_RECEIVE_TIMEOUT_S=5 so the CF round-trip can drain; sizes >16K skip (CF DataChannel message cap).

Architecture (`dimos/protocol/pubsub/impl/webrtc/`)

providers/spec.py — Provider protocol, AsyncProviderBase (shared loop-thread lifecycle), and ProviderConfig: a picklable, hashable factory resolving to a per-process singleton provider. Transports survive pickling into module worker processes; all transports in a process share one PeerConnection.

providers/broker.py — bidirectional broker-mediated CF Realtime (hosted teleop). Topic == DataChannel name:

channel	direction	use
`cmd_unreliable`	operator → robot	commands (unordered, lossy)
`state_reliable`	operator → robot	control plane (reliable)
`state_reliable_back`	robot → operator	telemetry via `publish()` (reliable; drops while no operator is connected — normal pubsub)

Heartbeat acks carry all three SCTP ids; channels open/close/re-open per operator join/leave/rejoin.

providers/cloudflare.py — direct CF access (pub+sub session loopback pair); integration tests + benchmark only.
webrtcpubsub.py — WebRTCPubSub(AllPubSub[str, bytes]), passes the standard grid tests in pubsub/test_spec.py.
WebRTCTransport[M: DimosMsg] (core/transport.py) — typed LCM encode/decode + wire-fingerprint demux on a multiplexed channel; CloudflareTransport binds BrokerConfig (→ BrokerProvider) for blueprints. Fingerprints derive from the wire format, not _get_packed_fingerprint() — TwistStamped inherits Twist's fingerprint but encodes as LCM TwistStamped; class fingerprints would drop every real message (regression-tested).

`HostedTeleopModule` deprecation

The module now emits a DeprecationWarning. Commands and telemetry are covered by CloudflareTransport; video-track publishing is the one remaining capability and stays on the module until BrokerProvider grows media support, after which it gets deleted. (Web-UI note: the browser currently parses state_reliable_back as its JSON control plane, so consuming LCM telemetry in the web app is a small front-end follow-up; the robot side is done.)

End-to-end verification (live, teleop.dimensionalos.com)

test_broker_e2e.py — aiortc stands in for the browser operator, speaking the web client's exact protocol (join → wait connected → bridge-datachannel → negotiated channels); the robot side is the blueprint's exact CloudflareTransport objects. Env-gated (TELEOP_API_KEY, TELEOP_ROBOT_ID, TELEOP_OPERATOR_TOKEN), -m tool:

operator → cmd_unreliable → CloudflareTransport → typed TwistStamped: 40/40 delivered, values verified
CloudflareTransport("state_reliable_back").broadcast() → operator: delivered + LCM-decoded
10/10 fresh connect/disconnect cycles against the live broker: zero leaked threads
multi-hour CF soak at 50 Hz: 0% loss, stable RSS (results in PR comments)

Benchmark (standard harness, sustained 1s windows, this machine → CF edge)

CF_TELEOP_APP_ID=… CF_TELEOP_APP_SECRET=… DIMOS_BENCH_RECEIVE_TIMEOUT_S=5 \
  pytest -m tool dimos/protocol/pubsub/benchmark/test_benchmark.py -k webrtc

Size	Sustained rate	Loss
256 B	~995 msg/s	0.0%
1 KiB	~994 msg/s	0.1%
4 KiB (~4 MB/s)	~994 msg/s	0.2%
16 KiB flooded	—	high (unreliable channel, offered load ≫ path)
> 16 KiB	dropped by CF	100%

CF paces DataChannel forwarding at ~1k msg/s per channel — plenty for teleop command planes (50–100 Hz). Same-host loopback RTT through the CF edge: median ~12–16 ms. Benchmark knobs are env-overridable (DIMOS_BENCH_DURATION_S, DIMOS_BENCH_MAX_MESSAGES, DIMOS_BENCH_RECEIVE_TIMEOUT_S).

Breaking changes

None — new optional transport. HostedTeleopModule keeps working (with a DeprecationWarning); teleop-hosted-go2 (video path) is unchanged.

Future work

Video track moves into BrokerProvider (media-track support), the web app learns LCM telemetry on state_reliable_back, then HostedTeleopModule and the teleop-hosted-go2 wrapper blueprint are deleted.

Contributor License Agreement

I have read and approved the CLA

                                       Benchmark Results                                       
┏━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━┳━━━━━━━━━┳━━━━━━━┓
┃ Transport        ┃ Msg Size ┃  Sent ┃  Recv ┃    Msgs/s ┃    MiB/s ┃  CPU ┃ Latency ┃  Loss ┃
┡━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━╇━━━━━━━━━╇━━━━━━━┩
│ LCM              │     64 B │ 5,000 │ 5,000 │    56,858 │      3.5 │ 1.48 │     0ms │  0.0% │
│ LCM              │    256 B │ 5,000 │ 5,000 │    56,903 │       14 │ 1.48 │     0ms │  0.0% │
│ LCM              │    1 KiB │ 5,000 │ 5,000 │    52,034 │       51 │ 1.46 │     0ms │  0.0% │
│ LCM              │    4 KiB │ 5,000 │ 4,998 │     4,684 │       18 │ 1.50 │  1000ms │  0.0% │
│ LCM              │   16 KiB │ 5,000 │ 4,942 │     4,579 │       72 │ 1.90 │  1000ms │  1.2% │
│ LCM              │   64 KiB │ 5,000 │ 4,951 │     4,457 │      279 │ 1.90 │  1000ms │  1.0% │
│ LCM              │  256 KiB │ 5,000 │ 4,835 │     3,583 │      896 │ 1.89 │  1000ms │  3.3% │
│ LCM              │  512 KiB │ 5,000 │ 4,634 │     2,791 │     1395 │ 1.89 │  1000ms │  7.3% │
│ LCM              │    1 MiB │ 3,614 │ 3,472 │     1,735 │     1735 │ 1.88 │  1000ms │  3.9% │
│ LCM              │    2 MiB │   772 │   632 │       316 │      632 │ 1.44 │  1000ms │ 18.1% │
│ LCM              │    5 MiB │   252 │   187 │        93 │      467 │ 1.38 │  1000ms │ 25.8% │
│ LCM              │   10 MiB │   114 │    66 │        33 │      329 │ 1.22 │  1001ms │ 42.1% │
│ ShmBytes         │     64 B │ 5,000 │ 5,000 │   196,631 │       12 │ 1.97 │     0ms │  0.0% │
│ ShmBytes         │    256 B │ 5,000 │ 5,000 │   197,354 │       48 │ 1.58 │     0ms │  0.0% │
│ ShmBytes         │    1 KiB │ 5,000 │ 5,000 │   206,112 │      201 │ 2.07 │     0ms │  0.0% │
│ ShmBytes         │    4 KiB │ 5,000 │ 5,000 │   197,810 │      773 │ 1.59 │     0ms │  0.0% │
│ ShmBytes         │   16 KiB │ 5,000 │ 5,000 │   176,225 │     2754 │ 2.12 │     0ms │  0.0% │
│ ShmBytes         │   64 KiB │ 5,000 │ 5,000 │   135,849 │     8491 │ 1.91 │     0ms │  0.0% │
│ ShmBytes         │  256 KiB │ 5,000 │ 5,000 │    68,705 │    17176 │ 2.06 │     0ms │  0.0% │
│ ShmBytes         │  512 KiB │ 5,000 │ 5,000 │    32,092 │    16046 │ 1.86 │     0ms │  0.0% │
│ ShmBytes         │    1 MiB │ 5,000 │ 5,000 │    12,062 │    12062 │ 1.88 │     0ms │  0.0% │
│ ShmBytes         │    2 MiB │ 4,873 │ 4,873 │     4,872 │     9744 │ 1.82 │     0ms │  0.0% │
│ ShmBytes         │    5 MiB │ 1,458 │ 1,458 │     1,457 │     7285 │ 1.80 │     0ms │  0.0% │
│ ShmBytes         │   10 MiB │   710 │   710 │       709 │     7090 │ 1.81 │     0ms │  0.0% │
│ ShmLcm           │     64 B │ 5,000 │ 5,000 │    84,897 │      5.2 │ 1.36 │     0ms │  0.0% │
│ ShmLcm           │    256 B │ 5,000 │ 5,000 │    92,068 │       22 │ 1.11 │     0ms │  0.0% │
│ ShmLcm           │    1 KiB │ 5,000 │ 5,000 │    84,484 │       83 │ 1.86 │     0ms │  0.0% │
│ ShmLcm           │    4 KiB │ 5,000 │ 5,000 │    85,006 │      332 │ 2.04 │     0ms │  0.0% │
│ ShmLcm           │   16 KiB │ 5,000 │ 5,000 │    74,391 │     1162 │ 1.64 │     0ms │  0.0% │
│ ShmLcm           │   64 KiB │ 5,000 │ 5,000 │    60,561 │     3785 │ 1.94 │     0ms │  0.0% │
│ ShmLcm           │  256 KiB │ 5,000 │ 5,000 │    26,346 │     6587 │ 1.90 │     0ms │  0.0% │
│ ShmLcm           │  512 KiB │ 5,000 │ 5,000 │    11,624 │     5812 │ 1.91 │     0ms │  0.0% │
│ ShmLcm           │    1 MiB │ 4,207 │ 4,207 │     4,206 │     4206 │ 1.87 │     0ms │  0.0% │
│ ShmLcm           │    2 MiB │ 1,779 │ 1,779 │     1,779 │     3557 │ 1.74 │     0ms │  0.0% │
│ ShmLcm           │    5 MiB │   614 │   614 │       613 │     3066 │ 1.51 │     0ms │  0.0% │
│ ShmLcm           │   10 MiB │   318 │   318 │       318 │     3179 │ 1.38 │     0ms │  0.0% │
│ ShmPickle        │     64 B │ 5,000 │ 5,000 │    74,515 │      4.5 │ 1.05 │     0ms │  0.0% │
│ ShmPickle        │    256 B │ 5,000 │ 5,000 │    71,706 │       18 │ 1.58 │     0ms │  0.0% │
│ ShmPickle        │    1 KiB │ 5,000 │ 5,000 │    71,421 │       70 │ 2.00 │     0ms │  0.0% │
│ ShmPickle        │    4 KiB │ 5,000 │ 5,000 │    68,519 │      268 │ 1.92 │     0ms │  0.0% │
│ ShmPickle        │   16 KiB │ 5,000 │ 5,000 │    59,934 │      936 │ 1.44 │     0ms │  0.0% │
│ ShmPickle        │   64 KiB │ 5,000 │ 5,000 │    51,749 │     3234 │ 1.86 │     0ms │  0.0% │
│ ShmPickle        │  256 KiB │ 5,000 │ 5,000 │    25,391 │     6348 │ 1.88 │     0ms │  0.0% │
│ ShmPickle        │  512 KiB │ 5,000 │ 5,000 │    11,469 │     5735 │ 1.90 │     0ms │  0.0% │
│ ShmPickle        │    1 MiB │ 4,649 │ 4,649 │     4,649 │     4649 │ 1.86 │     0ms │  0.0% │
│ ShmPickle        │    2 MiB │ 1,843 │ 1,843 │     1,842 │     3684 │ 1.71 │     0ms │  0.0% │
│ ShmPickle        │    5 MiB │   597 │   597 │       597 │     2984 │ 1.49 │     0ms │  0.0% │
│ ShmPickle        │   10 MiB │   178 │   178 │       177 │     1772 │ 1.22 │     0ms │  0.0% │
│ UdpBytes         │     64 B │ 5,000 │ 5,000 │   212,393 │       13 │ 2.13 │     0ms │  0.0% │
│ UdpBytes         │    256 B │ 5,000 │ 5,000 │   196,242 │       48 │ 1.97 │     0ms │  0.0% │
│ UdpBytes         │    1 KiB │ 5,000 │ 4,928 │     4,803 │      4.7 │ 1.93 │  1000ms │  1.4% │
│ UdpBytes         │    4 KiB │ 5,000 │ 5,000 │   168,475 │      658 │ 2.03 │     0ms │  0.0% │
│ UdpBytes         │   16 KiB │ 5,000 │ 4,992 │     4,793 │       75 │ 1.94 │  1000ms │  0.2% │
│ UdpBytes         │   64 KiB │ 5,000 │ 3,642 │     3,351 │      209 │ 2.08 │  1000ms │ 27.2% │
│ UdpBytes         │  256 KiB │ 5,000 │ 4,768 │     3,731 │      933 │ 1.94 │  1000ms │  4.6% │
│ UdpBytes         │  512 KiB │ 5,000 │ 4,354 │     2,826 │     1413 │ 1.94 │  1000ms │ 12.9% │
│ UdpBytes         │    1 MiB │ 4,733 │ 4,583 │     2,291 │     2291 │ 1.95 │  1000ms │  3.2% │
│ UdpBytes         │    2 MiB │ 2,421 │ 2,358 │     1,179 │     2357 │ 2.05 │  1000ms │  2.6% │
│ UdpBytes         │    5 MiB │   947 │   844 │       422 │     2109 │ 2.00 │  1001ms │ 10.9% │
│ UdpBytes         │   10 MiB │   432 │   410 │       205 │     2047 │ 2.05 │  1001ms │  5.1% │
│ WebrtcCloudflare │     64 B │ 5,000 │ 4,971 │     4,867 │     0.30 │ 1.42 │  1000ms │  0.6% │
│ WebrtcCloudflare │    256 B │ 5,000 │ 4,989 │     4,890 │      1.2 │ 1.50 │  1000ms │  0.2% │
│ WebrtcCloudflare │    1 KiB │ 5,000 │   592 │       580 │     0.57 │ 0.98 │  1000ms │ 88.2% │
│ WebrtcCloudflare │    4 KiB │ 5,000 │   124 │       120 │     0.47 │ 1.25 │  1000ms │ 97.5% │
│ WebrtcCloudflare │   16 KiB │ 5,000 │    29 │        27 │     0.43 │ 1.23 │  1000ms │ 99.4% │
│ WebrtcLoopback   │     64 B │ 5,000 │ 5,000 │ 3,840,464 │      234 │ 0.00 │     0ms │  0.0% │
│ WebrtcLoopback   │    256 B │ 5,000 │ 5,000 │ 3,866,010 │      944 │ 0.00 │     0ms │  0.0% │
│ WebrtcLoopback   │    1 KiB │ 5,000 │ 5,000 │ 3,543,516 │     3460 │ 7.17 │     0ms │  0.0% │
│ WebrtcLoopback   │    4 KiB │ 5,000 │ 5,000 │ 3,720,011 │    14531 │ 0.00 │     0ms │  0.0% │
│ WebrtcLoopback   │   16 KiB │ 5,000 │ 5,000 │ 3,655,241 │    57113 │ 0.00 │     0ms │  0.0% │
│ WebrtcLoopback   │   64 KiB │ 5,000 │ 5,000 │ 3,752,272 │   234517 │ 0.00 │     0ms │  0.0% │
│ WebrtcLoopback   │  256 KiB │ 5,000 │ 5,000 │ 3,791,029 │   947757 │ 0.00 │     0ms │  0.0% │
│ WebrtcLoopback   │  512 KiB │ 5,000 │ 5,000 │ 3,574,227 │  1787113 │ 0.00 │     0ms │  0.0% │
│ WebrtcLoopback   │    1 MiB │ 5,000 │ 5,000 │ 3,552,668 │  3552668 │ 7.18 │     0ms │  0.0% │
│ WebrtcLoopback   │    2 MiB │ 5,000 │ 5,000 │ 3,851,840 │  7703680 │ 0.00 │     0ms │  0.0% │
│ WebrtcLoopback   │    5 MiB │ 5,000 │ 5,000 │ 2,971,397 │ 14856987 │ 6.00 │     0ms │  0.0% │
│ WebrtcLoopback   │   10 MiB │ 5,000 │ 5,000 │ 3,584,576 │ 35845763 │ 0.00 │     0ms │  0.0% │
│ Zenoh            │     64 B │ 5,000 │ 5,000 │   125,701 │      7.7 │ 1.29 │     1ms │  0.0% │
│ Zenoh            │    256 B │ 5,000 │ 5,000 │   129,692 │       32 │ 1.31 │     0ms │  0.0% │
│ Zenoh            │    1 KiB │ 5,000 │ 5,000 │   125,514 │      123 │ 1.01 │     0ms │  0.0% │
│ Zenoh            │    4 KiB │ 5,000 │ 5,000 │   108,348 │      423 │ 1.09 │     0ms │  0.0% │
│ Zenoh            │   16 KiB │ 5,000 │ 5,000 │    96,878 │     1514 │ 1.19 │     1ms │  0.0% │
│ Zenoh            │   64 KiB │ 5,000 │ 5,000 │    64,726 │     4045 │ 1.31 │     1ms │  0.0% │
│ Zenoh            │  256 KiB │ 5,000 │ 5,000 │    10,262 │     2565 │ 1.03 │     3ms │  0.0% │
│ Zenoh            │  512 KiB │ 4,972 │ 4,972 │     4,950 │     2475 │ 1.02 │     0ms │  0.0% │
│ Zenoh            │    1 MiB │ 2,440 │ 2,440 │     2,435 │     2435 │ 1.02 │     2ms │  0.0% │
│ Zenoh            │    2 MiB │ 1,273 │ 1,273 │     1,226 │     2451 │ 1.02 │     0ms │  0.0% │
│ Zenoh            │    5 MiB │   461 │   461 │       409 │     2045 │ 1.15 │   126ms │  0.0% │
│ Zenoh            │   10 MiB │   153 │   153 │       153 │     1526 │ 0.99 │     2ms │  0.0% │
│ ZenohPeers       │     64 B │ 5,000 │ 5,000 │   112,606 │      6.9 │ 1.86 │    23ms │  0.0% │
│ ZenohPeers       │    256 B │ 5,000 │ 5,000 │   123,557 │       30 │ 1.78 │     1ms │  0.0% │
│ ZenohPeers       │    1 KiB │ 5,000 │ 5,000 │   115,642 │      113 │ 1.87 │     0ms │  0.0% │
│ ZenohPeers       │    4 KiB │ 5,000 │ 5,000 │    88,271 │      345 │ 1.77 │    11ms │  0.0% │
│ ZenohPeers       │   16 KiB │ 5,000 │ 5,000 │    74,828 │     1169 │ 1.68 │     1ms │  0.0% │
│ ZenohPeers       │   64 KiB │ 5,000 │ 5,000 │    50,023 │     3126 │ 1.90 │     0ms │  0.0% │
│ ZenohPeers       │  256 KiB │ 5,000 │ 5,000 │    22,860 │     5715 │ 1.60 │     0ms │  0.0% │
│ ZenohPeers       │  512 KiB │ 5,000 │ 5,000 │    12,793 │     6396 │ 1.43 │     0ms │  0.0% │
│ ZenohPeers       │    1 MiB │ 5,000 │ 5,000 │     5,390 │     5390 │ 1.30 │     0ms │  0.0% │
│ ZenohPeers       │    2 MiB │ 2,125 │ 2,125 │     2,124 │     4249 │ 1.22 │     0ms │  0.0% │
│ ZenohPeers       │    5 MiB │   630 │   630 │       629 │     3146 │ 1.15 │     0ms │  0.0% │
│ ZenohPeers       │   10 MiB │   285 │   285 │       284 │     2841 │ 1.62 │     2ms │  0.0% │
└──────────────────┴──────────┴───────┴───────┴───────────┴──────────┴──────┴─────────┴───────┘

                                                    Msgs/sec                                                     

                   64B     256B     1K      4K     16K     64K     256K    512K     1M      2M      5M     10M   
LCM               56.9k   56.9k   52.0k    4.7k    4.6k    4.5k    3.6k    2.8k    1.7k    316      93      33   
ShmBytes          196.6k  197.4k  206.1k  197.8k  176.2k  135.8k  68.7k   32.1k   12.1k    4.9k    1.5k    709   
ShmLcm            84.9k   92.1k   84.5k   85.0k   74.4k   60.6k   26.3k   11.6k    4.2k    1.8k    613     318   
ShmPickle         74.5k   71.7k   71.4k   68.5k   59.9k   51.7k   25.4k   11.5k    4.6k    1.8k    597     177   
UdpBytes          212.4k  196.2k   4.8k   168.5k   4.8k    3.4k    3.7k    2.8k    2.3k    1.2k    422     205   
WebrtcCloudflare   4.9k    4.9k    580     120      27      -       -       -       -       -       -       -    
WebrtcLoopback     3.8M    3.9M    3.5M    3.7M    3.7M    3.8M    3.8M    3.6M    3.6M    3.9M    3.0M    3.6M  
Zenoh             125.7k  129.7k  125.5k  108.3k  96.9k   64.7k   10.3k    4.9k    2.4k    1.2k    409     153   
ZenohPeers        112.6k  123.6k  115.6k  88.3k   74.8k   50.0k   22.9k   12.8k    5.4k    2.1k    629     284   


                                                 Bandwidth (IEC)                                                 

                   64B     256B     1K      4K     16K     64K     256K    512K     1M      2M      5M     10M   
LCM                3.5M   13.9M   50.8M   18.3M   71.5M   278.6M  895.8M   1.4G    1.7G   631.9M  467.2M  328.8M 
ShmBytes          12.0M   48.2M   201.3M  772.7M   2.7G    8.3G   16.8G   15.7G   11.8G    9.5G    7.1G    6.9G  
ShmLcm             5.2M   22.5M   82.5M   332.1M   1.1G    3.7G    6.4G    5.7G    4.1G    3.5G    3.0G    3.1G  
ShmPickle          4.5M   17.5M   69.7M   267.7M  936.5M   3.2G    6.2G    5.6G    4.5G    3.6G    2.9G    1.7G  
UdpBytes          13.0M   47.9M    4.7M   658.1M  74.9M   209.4M  932.8M   1.4G    2.2G    2.3G    2.1G    2.0G  
WebrtcCloudflare  304.2K   1.2M   580.1K  480.5K  435.6K    -       -       -       -       -       -       -    
WebrtcLoopback    234.4M  943.9M   3.4G   14.2G   55.8G   229.0G  925.5G   1.7T    3.4T    7.3T   14.2T   34.2T  
Zenoh              7.7M   31.7M   122.6M  423.2M   1.5G    4.0G    2.5G    2.4G    2.4G    2.4G    2.0G    1.5G  
ZenohPeers         6.9M   30.2M   112.9M  344.8M   1.1G    3.1G    5.6G    6.2G    5.3G    4.1G    3.1G    2.8G  


                                                   CPU (cores)                                                   

                   64B     256B     1K      4K     16K     64K     256K    512K     1M      2M      5M     10M   
LCM                1.48    1.48    1.46    1.50    1.90    1.90    1.89    1.89    1.88    1.44    1.38    1.22  
ShmBytes           1.97    1.58    2.07    1.59    2.12    1.91    2.06    1.86    1.88    1.82    1.80    1.81  
ShmLcm             1.36    1.11    1.86    2.04    1.64    1.94    1.90    1.91    1.87    1.74    1.51    1.38  
ShmPickle          1.05    1.58    2.00    1.92    1.44    1.86    1.88    1.90    1.86    1.71    1.49    1.22  
UdpBytes           2.13    1.97    1.93    2.03    1.94    2.08    1.94    1.94    1.95    2.05    2.00    2.05  
WebrtcCloudflare   1.42    1.50    0.98    1.25    1.23     -       -       -       -       -       -       -    
WebrtcLoopback      -       -      7.17     -       -       -       -       -      7.18     -      6.00     -    
Zenoh              1.29    1.31    1.01    1.09    1.19    1.31    1.03    1.02    1.02    1.02    1.15    0.99  
ZenohPeers         1.86    1.78    1.87    1.77    1.68    1.90    1.60    1.43    1.30    1.22    1.15    1.62  


                                                     Latency                                                     

                   64B     256B     1K      4K     16K     64K     256K    512K     1M      2M      5M     10M   
LCM               0.2 ms  0.1 ms  0.1 ms  1.0 s   1.0 s   1.0 s   1.0 s   1.0 s   1.0 s   1.0 s   1.0 s   1.0 s  
ShmBytes          0.1 ms  0.1 ms  0.1 ms  0.1 ms  0.1 ms  0.1 ms  0.1 ms  0.1 ms  0.2 ms  0.1 ms  0.1 ms  0.1 ms 
ShmLcm            0.1 ms  0.2 ms  0.1 ms  0.1 ms  0.1 ms  0.1 ms  0.1 ms  0.1 ms  0.1 ms  0.1 ms  0.1 ms  0.1 ms 
ShmPickle         0.2 ms  0.1 ms  0.1 ms  0.1 ms  0.1 ms  0.1 ms  0.1 ms  0.1 ms  0.1 ms  0.2 ms  0.1 ms  0.1 ms 
UdpBytes          0.0 ms  0.0 ms  1.0 s   0.1 ms  1.0 s   1.0 s   1.0 s   1.0 s   1.0 s   1.0 s   1.0 s   1.0 s  
WebrtcCloudflare  1.0 s   1.0 s   1.0 s   1.0 s   1.0 s     -       -       -       -       -       -       -    
WebrtcLoopback    0.0 ms  0.0 ms  0.0 ms  0.0 ms  0.0 ms  0.0 ms  0.0 ms  0.0 ms  0.0 ms  0.0 ms  0.0 ms  0.0 ms 
Zenoh             0.9 ms  0.5 ms  0.2 ms  0.1 ms  1.1 ms  1.2 ms  2.7 ms  0.1 ms  1.7 ms  0.1 ms 126.3 ms 1.7 ms 
ZenohPeers       22.9 ms  1.1 ms  0.4 ms 11.3 ms  1.2 ms  0.1 ms  0.1 ms  0.1 ms  0.2 ms  0.3 ms  0.1 ms  1.6 ms 


                                                     Loss %                                                      

                   64B     256B     1K      4K     16K     64K     256K    512K     1M      2M      5M     10M   
LCM                 -       -       -      0.0%    1.2%    1.0%    3.3%    7.3%    3.9%   18.1%   25.8%   42.1%  
ShmBytes            -       -       -       -       -       -       -       -       -       -       -       -    
ShmLcm              -       -       -       -       -       -       -       -       -       -       -       -    
ShmPickle           -       -       -       -       -       -       -       -       -       -       -       -    
UdpBytes            -       -      1.4%     -      0.2%   27.2%    4.6%   12.9%    3.2%    2.6%   10.9%    5.1%  
WebrtcCloudflare   0.6%    0.2%   88.2%   97.5%   99.4%     -       -       -       -       -       -       -    
WebrtcLoopback      -       -       -       -       -       -       -       -       -       -       -       -    
Zenoh               -       -       -       -       -       -       -       -       -       -       -       -    
ZenohPeers          -       -       -       -       -       -       -       -       -       -       -       -

…e SFU Implements a new pubsub transport backed by WebRTC DataChannels over Cloudflare's Realtime SFU. Two new classes in dimos/protocol/pubsub/impl/webrtcpubsub.py: - CloudflareSession: manages the WebRTC PeerConnection lifecycle. Opens two CF sessions (publisher + subscriber) so a single process can do loopback pubsub. Runs aiortc on a dedicated background asyncio thread with its own ThreadPoolExecutor (so we don't leak asyncio_N worker threads). Uses negotiated=True placeholder DCs with id=100 during transport establishment to avoid stream-id collisions with CF-assigned ids. - WebRTCPubSub: bytes-on-the-wire pubsub facade matching the LCMPubSubBase / BytesSharedMemory interface (string topics, bytes payloads). Lazily creates pub/sub DataChannel pairs on first publish/subscribe per topic. Also adds: - WebRTCTransport in dimos/core/transport.py (mirrors LCMTransport pattern, no encoding - bytes only). - WebRTC benchmark testcase in dimos/protocol/pubsub/benchmark/testdata.py, gated on aiortc + CF_TELEOP_APP_ID / CF_TELEOP_APP_SECRET env vars. - Integration test in dimos/protocol/pubsub/impl/test_webrtcpubsub.py covering basic pub/sub, latency, and throughput (all live tests skip without CF credentials). - aiortc + httpx as new 'webrtc' optional extra in pyproject.toml. Live benchmark (us-east-2 -> CF edge): - 64-256B: ~10K msgs/s, 0% loss - 1KiB: ~7K msgs/s, 0% loss - >= 64KiB: dropped (above SCTP message size) - Median single-RTT: ~2.5 ms

…sport - Add BrokerProvider: DataChannelProvider that works through the hosted teleop broker (dimensional-teleop) instead of directly with CF credentials. Handles session registration, heartbeat loop, and DataChannel creation when an operator joins via the broker's bridge-datachannel API. - Extend WebRTCTransport with optional msg_type parameter for typed LCM encode/decode with fingerprint-based filtering. Multiple transports can share a single multiplexed DataChannel and each receives only its type. - Add hosted teleop blueprints (dimos/teleop/hosted/) demonstrating the module-free architecture: make_teleop_hosted_go2() uses pure transport (zero modules), make_teleop_hosted_go2_scaled() adds a thin TeleopScalerModule for speed scaling only. - Add unit tests for typed mode, fingerprint filtering, multiplexed dispatch, and BrokerProvider credential validation.

- Rebase on main and regenerate uv.lock (resolve conflict) - Add _LoopbackProvider (in-process, no network) to benchmark testdata - Enables local WebRTC transport benchmarking without CF credentials - All 12 message sizes pass locally (2.78s total)

codecov · 2026-05-19T23:37:44Z

Codecov Report

❌ Patch coverage is 66.05505% with 333 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
...os/protocol/pubsub/impl/webrtc/providers/broker.py	35.23%	116 Missing and 9 partials ⚠️
...rotocol/pubsub/impl/webrtc/providers/cloudflare.py	32.97%	119 Missing and 5 partials ⚠️
...imos/protocol/pubsub/impl/webrtc/providers/spec.py	69.82%	29 Missing and 6 partials ⚠️
...imos/protocol/pubsub/impl/webrtc/test_transport.py	92.56%	13 Missing and 7 partials ⚠️
dimos/core/transport.py	82.50%	12 Missing and 2 partials ⚠️
dimos/protocol/pubsub/impl/webrtc/webrtcpubsub.py	77.58%	10 Missing and 3 partials ⚠️
dimos/core/coordination/module_coordinator.py	96.29%	0 Missing and 1 partial ⚠️
dimos/teleop/quest_hosted/hosted_teleop_module.py	75.00%	1 Missing ⚠️

@@            Coverage Diff             @@
##             main    #2048      +/-   ##
==========================================
- Coverage   72.53%   72.07%   -0.47%     
==========================================
  Files         905      941      +36     
  Lines       82233    84147    +1914     
  Branches     7432     7703     +271     
==========================================
+ Hits        59648    60645     +997     
- Misses      20646    21464     +818     
- Partials     1939     2038      +99

Flag	Coverage Δ
OS-ubuntu-24.04-arm	`64.44% <65.67%> (-0.21%)`	⬇️
OS-ubuntu-latest	`67.01% <65.67%> (-0.26%)`	⬇️
Py-3.10	`66.99% <65.67%> (-0.27%)`	⬇️
Py-3.11	`67.00% <65.67%> (-0.26%)`	⬇️
Py-3.12	`67.00% <65.67%> (-0.26%)`	⬇️
Py-3.13	`67.01% <65.67%> (-0.26%)`	⬇️
Py-3.14	`67.01% <65.67%> (-0.27%)`	⬇️
Py-3.14t	`67.01% <65.67%> (-0.26%)`	⬇️
SelfHosted-Large	`30.23% <28.44%> (+0.01%)`	⬆️
SelfHosted-Linux	`38.05% <32.28%> (-0.01%)`	⬇️
SelfHosted-macOS	`36.35% <28.14%> (-0.02%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
dimos/core/coordination/blueprints.py	`93.52% <100.00%> (+1.01%)`	⬆️
dimos/core/stream.py	`81.57% <100.00%> (+0.24%)`	⬆️
dimos/protocol/pubsub/impl/webrtc/providers/sdp.py	`15.00% <ø> (ø)`
...otocol/pubsub/impl/webrtc/providers/video_track.py	`42.55% <ø> (ø)`
dimos/protocol/pubsub/test_spec.py	`90.47% <100.00%> (+0.53%)`	⬆️
dimos/robot/all_blueprints.py	`100.00% <ø> (ø)`
dimos/robot/cli/dimos.py	`63.34% <100.00%> (-3.49%)`	⬇️
dimos/robot/test_all_blueprints.py	`87.50% <ø> (ø)`
dimos/robot/test_all_blueprints_generation.py	`85.03% <ø> (ø)`
dimos/teleop/quest_hosted/blueprints.py	`100.00% <100.00%> (ø)`
... and 8 more

... and 61 files with indirect coverage changes

🚀 New features to boost your workflow:

📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

WebRTCPubSub over an in-memory loopback provider — measures the webrtc pubsub/dispatch stack with no network, SCTP, or credentials. Gives every contributor a webrtc row in the matrix (~2.4M msg/s stack ceiling) and an upper bound to read the credential-gated WebrtcCloudflare row against.

If createDataChannel raised inside the heartbeat, the id was already recorded, so every later heartbeat skipped the open and the channel stayed dead until the operator id changed. Extract _reconcile_channels and store the id only after a successful open so the next beat retries. Also drop needless forward refs in Blueprint.transports (review nit).

…pping Per Paul's review: a dev or CI env without the webrtc deps used to skip the entire suite invisibly. The guard asserts WEBRTC_AVAILABLE when CI is set; downstream/no-network installs (Sam's case) keep skipping.

Per review: the repo convention moved dev-only/live tests to tool_*.py filenames (never collected; run by explicit path) and removed the tool marker codebase-wide. Cred-gating skipifs stay for keyless path runs.

tool_*.py dev-only test files (renamed from test_*.py in ea4db73) fell out of the test_ filename exclusion, leaking TwistSource/TwistSink from the webrtc e2e tool into all_modules and failing the CI currency check.

…issing root cause of the arm failures: the tests job only apt-installed portaudio, x64 runners had libturbojpeg by accident from the runner image. install it explicitly and turn the silent skip into a CI assert (same treatment as the webrtc aiortc guard from #2048), skip stays for local no-syslib installs

…her backends supported(#2048) Co-authored-by: ruthwikdasyam <ruthwikdasyam@gmail.com> Co-authored-by: Sam Bull <git@sambull.org> Co-authored-by: Paul Nechifor <paul@nechifor.net>

ruthwikdasyam added 9 commits May 7, 2026 15:28

mem2 recording fixes

41412df

feat: hosted teleop module

3d6adae

misc: hosted tests

33c2571

misc: no-cloudflare broker

fb43817

fix: auth updates

7b59671

updated html + web address

4e31bae

working commit

ed2d275

feat/workingcommit/keyboard-teleop

b6914e9

feat/TwistStamped + recording msgs

027dae0

spomichter force-pushed the feat/webrtc-transport branch from e56fd1f to 212450b Compare May 11, 2026 22:44

ruthwikdasyam and others added 18 commits May 18, 2026 11:07

Merge branch 'main' into ruthwik/hosted-teleop

838a2c0

[autofix.ci] apply automated fixes

a560a77

feat: added seq to Timestamped msg

0ed9fcd

no seperate sim blueprints needed

d5ea14e

feat: increase workers

06516f5

feat: benchmarking

f88746f

fix: pre-commit errors

d18e495

fix: logging spam

b2b11d6

added seq to cal loss

e20bceb

benchmarks for twist

996be8d

fix: pre-commit fixes

1b0829d

[autofix.ci] apply automated fixes

38e51d4

fix: notebook file import

2043f13

fix: pre-commit

3a4cf68

[autofix.ci] apply automated fixes

b5dfe71

spomichter force-pushed the feat/webrtc-transport branch from 212450b to 2c263eb Compare May 19, 2026 23:30

greptile-apps Bot reviewed Jul 7, 2026

View reviewed changes

Comment thread dimos/protocol/pubsub/impl/webrtc/providers/cloudflare.py

fix(bench): type the loopback provider callbacks properly (mypy)

c967feb

Dreamsorcerer reviewed Jul 7, 2026

View reviewed changes

Comment thread dimos/core/coordination/blueprints.py Outdated

greptile-apps Bot reviewed Jul 7, 2026

View reviewed changes

Comment thread dimos/protocol/pubsub/impl/webrtc/providers/broker.py Outdated

Dreamsorcerer reviewed Jul 7, 2026

View reviewed changes

Comment thread dimos/core/stream.py

Dreamsorcerer reviewed Jul 7, 2026

View reviewed changes

Comment thread dimos/core/coordination/blueprints.py

Dreamsorcerer previously approved these changes Jul 7, 2026

View reviewed changes

github-actions Bot added the ready-to-merge Required CI checks have passed on this PR label Jul 7, 2026

paul-nechifor reviewed Jul 7, 2026

View reviewed changes

Comment thread dimos/protocol/pubsub/impl/webrtc/test_broker_e2e.py Outdated

paul-nechifor reviewed Jul 7, 2026

View reviewed changes

Comment thread dimos/protocol/pubsub/impl/webrtc/tool_broker_e2e.py

paul-nechifor previously approved these changes Jul 7, 2026

View reviewed changes

spomichter dismissed stale reviews from paul-nechifor and Dreamsorcerer via 12173ae July 7, 2026 22:59

github-actions Bot removed the ready-to-merge Required CI checks have passed on this PR label Jul 7, 2026

spomichter added 3 commits July 7, 2026 16:01

chore: untrack accidentally committed local nav_stack symlinks

aa56aa4

test(webrtc): move tool tests to tool_*.py files, drop @pytest.mark.tool

ea4db73

Per review: the repo convention moved dev-only/live tests to tool_*.py filenames (never collected; run by explicit path) and removed the tool marker codebase-wide. Cred-gating skipifs stay for keyless path runs.

paul-nechifor previously approved these changes Jul 7, 2026

View reviewed changes

fix(blueprints): exclude tool_*.py files from module registry scan

434a13f

tool_*.py dev-only test files (renamed from test_*.py in ea4db73) fell out of the test_ filename exclusion, leaking TwistSource/TwistSink from the webrtc e2e tool into all_modules and failing the CI currency check.

spomichter dismissed paul-nechifor’s stale review via 434a13f July 7, 2026 23:32

greptile-apps Bot reviewed Jul 7, 2026

View reviewed changes

Comment thread dimos/protocol/pubsub/impl/webrtc/providers/broker.py

mustafab0 approved these changes Jul 7, 2026

View reviewed changes

github-actions Bot added the ready-to-merge Required CI checks have passed on this PR label Jul 8, 2026

spomichter merged commit e176fde into main Jul 8, 2026
86 of 112 checks passed

spomichter deleted the feat/webrtc-transport branch July 8, 2026 02:09

dimos-bot mentioned this pull request Jul 15, 2026

Fix end-to-end Cloudflare transport test #2972

Open

Navigation Menu

Repository navigation

feat(transport): WebRTC DataChannel transport (cloudflare SFU) - #2048

feat(transport): WebRTC DataChannel transport (cloudflare SFU)#2048
spomichter merged 116 commits into
mainfrom
feat/webrtc-transport

spomichter commented May 11, 2026 •

edited

Loading

Uh oh!

codecov Bot commented May 19, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

spomichter commented May 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Solution

How to run it against ruthwik's hosted stack

Verify without credentials (no key, no robot)

Architecture (dimos/protocol/pubsub/impl/webrtc/)

HostedTeleopModule deprecation

End-to-end verification (live, teleop.dimensionalos.com)

Benchmark (standard harness, sustained 1s windows, this machine → CF edge)

Breaking changes

Future work

Contributor License Agreement

Uh oh!

codecov Bot commented May 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

spomichter commented May 11, 2026 •

edited

Loading

Architecture (`dimos/protocol/pubsub/impl/webrtc/`)

`HostedTeleopModule` deprecation

codecov Bot commented May 19, 2026 •

edited

Loading