Skip to content

feat(control): consolidate CDR manipulation and teleop boundaries - #4736

Draft
TomCC7 wants to merge 8 commits into
cc/feat/cdr-unit-mappingfrom
cc/feat/cdr-unit-control
Draft

TomCC7 wants to merge 8 commits into
cc/feat/cdr-unit-mappingfrom
cc/feat/cdr-unit-control

Conversation

@TomCC7

@TomCC7 TomCC7 commented Oct 20, 2026 •

Copy link
Copy Markdown
Member

What this adds

Migrate control, manipulation and teleop ports to generated CDR messages. Keep task behavior and driver conversions, and update WebXR, keyboard and hosted command paths.

Base: #4737. Existing transport specs and coordinator backend selection replace the old CdrTransport calls.

Validation and limits

Fresh cleanup checks: 239 control and teleop tests passed, including JavaScript encoder/Python interoperability and both backend payload cases. The single-process LCM test uses memq and closes its owned transports. Repository hooks passed.

WIP: standalone installed-layer, physical browser/headset and hardware behavior are unverified. Drivers were mocked; multicast networking was not certified.

Review report: file map, scope and verification

CDR dependencies and TF conventions

The stack consumes published dimos-cdrgen 0.1.2 and the immutable CDR library fix. SDK #4774 now uses standard ROS TFMessage, TransformStamped and integer Time, with no per-driver TF codec callbacks or parallel public transform type. The external transforms library, validated by its complete std/no_std/serde and ARM verification gate, owns the frame tree, common-time lookup, interpolation and static history. Its atomic batch patch is pinned to 5ae9563de0c02638c04e393aadfad28444a4398e.

Combined-stack local validation: SDK 121, macros 22, mapper 101, MLS 135, local planner 91, follower 63, V4L2 6 and depth2depth 12 Rust tests; all 24 native/DimSim/port-contract acceptance tests pass without skips. The mapper exchange includes Python CDR tf_static input; the follower exchange covers drive, arrival, a new goal and empty-path stop. Missing native binaries fail the dedicated job. RealSense passes cargo check --all-targets --features docs-only; device linking and hardware execution were not tested. Seven real source-staging layouts pass locked Cargo resolution. Native acceptance for tip cc463c163aed465f7c1d6808165bf640820b0182: workflow run. These are combined-tip results, with the SDK additionally tested standalone at its own layer.

Native Nix inputs use verified source archive f7723ad4d5b9aa5dfe63614b6b537d1ccc2d212e, SHA256 f90af8fa794a34d53ee9481c6d0bdb63138853b30f851d92960fd34224ff62b2. Full Nix builds remain unverified. SDK static edges persist after receipt; LCM/Zenoh do not acquire DDS transient-local durability from CDR. Existing Python mount publishers retain dynamic tf compatibility. Legacy modules remain explicitly pinned to SDK 0.1.0.

Stack #4776 preserves existing history and PR bases. Current main still has conflicts with the original stack; this change does not rebase onto main. No DimOS PR has been merged.

@TomCC7

TomCC7 commented Oct 21, 2026 •

Copy link
Copy Markdown
Member Author

Review report

Updated: 2026-10-21. Direct predecessor: #4737.

  • Head: 0a72a633dc4d5b1593cbe7457e9701d627895886
  • Named base branch: cc/feat/cdr-unit-mapping at f46ed9ce6b23a3c4ec1a48adaeeeef1eb4eca9b1
  • PR-recorded base: f46ed9ce6b23a3c4ec1a48adaeeeef1eb4eca9b1; diff merge base: f46ed9ce6b23a3c4ec1a48adaeeeef1eb4eca9b1

What changes and why the diff is large

Migrate coordinator/task/teleoperation wire ports to generated messages, keeping existing numerical control and IK values behind explicit adapters. Hosted, keyboard and WebXR producers are updated with their consumers; WebXR adds schema-based browser CDR command encoding.
The implementation remains control-task, coordinator and command producer/consumer adaptation. Helper imports follow the shared namespace. A transport-test fix formerly carried by the robot layer now lives here.

Review map

Suggested entry points: dimos/control/coordinator.py, dimos/control/tasks/pose_target_ik.py, dimos/teleop/hosted/arm_command.py, dimos/teleop/webxr/module.py, dimos/teleop/webxr/web/static/command_codec.mjs.
All 36 changed files are accounted for below. Grouping is by file role, not a claim that the implementation is minimal.

Complete file map

Implementation (20)

File Change Purpose
dimos/control/coordinator.py modified Generated control ports with legacy JointState/TF conversion on publish and driver-facing routing.
dimos/control/tasks/cartesian_ik_task/cartesian_ik_task.py modified Use generated Pose, PoseStamped in on_cartesian_command.
dimos/control/tasks/eef_twist_task/eef_twist_task.py modified Use generated TwistStamped in on_ee_twist_command.
dimos/control/tasks/gripper_task/gripper_task.py modified Use generated Float32 in on_gripper_command.
dimos/control/tasks/pose_target_ik.py modified Convert generated pose targets into the retained numerical IK PoseStamped value.
dimos/control/tasks/teleop_ik_task/teleop_ik_task.py modified Use generated Pose, PoseStamped, UInt32 in on_left_cartesian_command, on_right_cartesian_command.
dimos/control/tasks/trajectory_task/trajectory_task.py modified Use generated JointState in on_joint_command.
dimos/control/teleop_coordinator.py modified Declare/use generated PoseStamped, TwistStamped, Float32 at this boundary.
dimos/robot/manipulators/common/coordinators.py modified Declare/use generated PoseStamped, TwistStamped, TFMessage at this boundary.
dimos/teleop/hosted/arm_command.py modified Use generated PoseStamped, Twist, TwistStamped in __init__, _on_cmd_raw.
dimos/teleop/hosted/blueprints/cloudflare.py modified Declare/use generated Twist, TwistStamped, Image at this boundary.
dimos/teleop/hosted/go2_command.py modified Use generated Point, Pose, PoseStamped in _clamp, __init__.
dimos/teleop/hosted/map_compress.py modified Use generated PoseStamped, OccupancyGrid in _on_costmap, _on_odom.
dimos/teleop/keyboard/keyboard_teleop_module.py modified Use generated Twist, TwistStamped, Vector3 in _publish_twist.
dimos/teleop/webxr/blueprints.py modified Declare/use generated PoseStamped, Image at this boundary.
dimos/teleop/webxr/extensions.py modified Use generated Twist, TwistStamped, Vector3 in _push_jpeg, _publish_msg.
dimos/teleop/webxr/module.py modified Decode CDR-framed WebXR commands and publish generated pose/button/status ports.
dimos/teleop/webxr/web/static/command_codec.mjs added Foxglove schema-based CDR command writer inside the existing relay data-frame envelope.
dimos/teleop/webxr/web/static/index.html modified Adapt index browser/simulation command or sensor handling to this unit's message flow.
dimos/teleop/webxr/web/static/teleop.js modified Adapt teleop browser/simulation command or sensor handling to this unit's message flow.

Build and dependency metadata (2)

File Change Purpose
MANIFEST.in modified Declare or package the dependencies/assets used by this unit.
pyproject.toml modified Declare or package the dependencies/assets used by this unit.

Tests (13)

File Change Purpose
dimos/control/tasks/cartesian_ik_task/test_cartesian_ik_task.py modified Checks cartesian config requires exactly one target frame.
dimos/control/tasks/eef_twist_task/test_eef_twist_task.py modified Checks eef twist config requires exactly one target frame.
dimos/control/tasks/gripper_task/test_gripper_task.py modified Checks normalized and native commands share the joint path.
dimos/control/tasks/teleop_ik_task/test_teleop_ik_task.py modified Checks face buttons do not engage arm teleop.
dimos/control/test_coordinator_routing.py modified Checks cdr joint wire reaches existing trajectory task and publishes cdr state.
dimos/control/test_joint_commands.py modified Checks intended subset matches dispatch even on failure.
dimos/control/test_per_robot_joint_states.py modified Checks default is off.
dimos/teleop/hosted/test_arm_command.py modified Checks cmd raw pose routes to hand.
dimos/teleop/hosted/test_go2_command.py modified Checks allowed sport cmd calls driver rpc.
dimos/teleop/hosted/test_map_compress.py modified Checks costmap encodes and publishes map.
dimos/teleop/keyboard/test_keyboard_teleop_module.py modified Checks publish twist emits unaddressed twist stamped.
dimos/teleop/webxr/test_blueprints.py modified Checks single arm blueprint uses one frame binding and right stream.
dimos/teleop/webxr/test_module.py modified Checks body debug logging reports received snapshots.

Documentation and replay inputs (1)

File Change Purpose
docs/capabilities/navigation/deep_dive.md modified Update the deep dive guide or replay input for the migrated boundaries.

Review focus and scope reductions

  • Internal control values still require adapters at coordinator/IK edges. Existing TickLoop, trajectory and IK consumers were checked before retaining them; deleting these conversions would require a separate control model migration.
  • The test no longer imports deleted CdrTransport. It uses the existing LCM transport. WebXR's single-process payload test uses memq and explicitly stops its owned transports; production transport/concurrency behavior is unchanged.
  • Review joint ordering, routing, button/status semantics and float-seconds precision loss at internal boundaries.
  • JavaScript encoder/Python interoperability was exercised; physical browser headset behavior and UDP multicast delivery are separate unverified paths.

Validation

Fresh checks on 0a72a633dc4d5b1593cbe7457e9701d627895886: 239 coordinator/hosted/keyboard/WebXR tests passed, including the actual JavaScript command encoder through Deno and both backend payload cases. All-files hooks passed with cargo/native lint excluded. The first rerun exposed a stale deleted-module import and a local multicast self-test failure; the existing later-layer import fix was moved here and the single-process WebXR test now owns memq transports. This is a local encoding/lifecycle check, not a claim that multicast networking passed.

Reproduction commands, not executed here unless stated above

Run from this PR's own checkout root after installing its documented test dependencies. These are selected review entry points, not a claim of full-stack coverage.

uv run pytest dimos/control/test_coordinator_routing.py dimos/control/test_joint_commands.py dimos/control/test_per_robot_joint_states.py
uv run pytest dimos/teleop/hosted/test_arm_command.py dimos/teleop/hosted/test_go2_command.py dimos/teleop/keyboard/test_keyboard_teleop_module.py dimos/teleop/webxr/test_module.py dimos/teleop/webxr/test_blueprints.py

Remaining work

Internal legacy value adapters remain because their consumers still use those models. Standalone installed-layer, physical browser/headset and hardware behavior are unverified. Full stack type/graph migration remains WIP.

@TomCC7
TomCC7 force-pushed the cc/feat/cdr-unit-control branch from 77fa4af to 0a72a63 Compare October 22, 2026 02:34
@TomCC7
TomCC7 removed this pull request from stack #4743 October 22, 2026 18:51
@TomCC7
TomCC7 added this pull request to stack #4776 October 22, 2026 18:52

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant