Skip to content

feat: add gated Dreamlabs rollout for dual YAM - #4653

Draft
TomCC7 wants to merge 2 commits into
mainfrom
cc/feat/dreamlabs-dual-yam
Draft

TomCC7 wants to merge 2 commits into
mainfrom
cc/feat/dreamlabs-dual-yam

Conversation

@TomCC7

@TomCC7 TomCC7 commented Oct 20, 2026 •

Copy link
Copy Markdown
Member

Dual OpenYAM currently has no remote multi-camera Dreamlabs rollout. This adds an idle-by-default normal DimOS module using the existing coordinator and three RGB streams, with explicit preflight/start/stop RPCs.

The optional dreamlabs extra adds only dreamlabs-client==1.0.1. Existing locked package versions are unchanged. The client is imported lazily at an explicit start. DimOS's existing Pinocchio FK supplies arm-local EEF observations from the pinned SDK reference joint geometry and TCP convention. No full robot SDK, isolated runtime, or action IK is required for YAM joint chunks.

The adapter preserves radians, normalized grippers, and left/right chunk mapping, and executes complete 48-step chunks at 20 Hz. It rejects malformed, stale, out-of-bounds, discontinuous, or EEF-only answers, cancels on error or manual takeover, and blocks restart after uncertain cancellation. A small coordinator status RPC lets it wait for actual task completion. SDK background recording is disabled. Setup instructions and an explicitly invoked, one-request synthetic API probe are included.

Validation: 76 offline integration/client/FK tests and 81 coordinator/trajectory/blueprint-registry regression tests pass. The FK tests compare 38 known and seeded varied poses with fixed reference outputs generated offline from the official SDK. Tests also exercise the actual lightweight client with its network transport replaced, and import/help with the optional SDK unavailable. They pass using existing main base dependencies plus the lightweight client, with no dream_ik installed. Strict mypy passes for five production modules; applicable pre-commit checks and the main lock check pass.

Live validation remains pending. No live inference, credentials, camera, robot driver, or motion was used. Before a paid API call, confirm the credential, account model access, payload source, and visible cost limit. Before hardware rollout, verify standard YAM identity, zero/direction and linear-gripper calibration, camera roles, SDK TCP convention, workspace, operator, and physical E-stop. The observation reference is not a physical calibration, and DimOS's grasp frame is not the SDK TCP. Position/speed checks add no scene collision planner. This PR contains no CDR restructuring or #4646 changes.

Setup: Dreamlabs dual YAM guide. Contract verified against official dreamlabs-client 1.0.1.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

go2 replay realtime (arm64)

Details
Benchmark suite Current: 96d8073 Previous: 17fdb6b Ratio
peak memory 1607.754 MB 1589.477 MB 1.01
peak threads 353 threads 350 threads 1.01
network (transport) 2722.048 MB 2730.079 MB 1.00
disk write 1.688 MB 1.707 MB 0.99
instructions 115.401 G 115.497 G 1.00

This comment was automatically generated by workflow using github-action-benchmark.

@greptile-apps

greptile-apps Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 4/5

[High impact] The rollout reaches remote inference and physical dual-arm motion at approved OpenYAM stations.

Do not merge until cancellation respects trajectory ownership; an ended or stopping Dreamlabs episode can abort motion submitted by another caller.

Findings

  1. P1 Old rollout stops new motion ▶

Summary

Adds an idle-by-default Dreamlabs policy for dual OpenYAM, with explicit approval gates and preflight, three-camera observations, bounded action chunks, an isolated SDK project, offline tests, setup guidance, and an opt-in synthetic probe.

Cancellation must respect control handoff before merging. Repeated stop, module shutdown, and delayed inference cleanup each aborted another caller's newly accepted motion in offline execution using the real runtime, coordinator, and trajectory task.

The established interpolation check remains unchanged: straight-line trajectory interpolation stays between the checked positions.

TomCC7 explicitly acknowledges that live inference and hardware validation are pending because no credentials are available. The PR requires credential, model-access, payload, and cost checks before a paid request, and calibration, camera-role, TCP, workspace, operator, and physical E-stop checks before hardware rollout. TomCC7 also explicitly notes that position and speed checks do not provide scene collision planning.

  • Dual OpenYAM can run gated Dreamlabs action chunks through its coordinator.
  • Operators can test one synthetic Dreamlabs request without connecting a robot.

Reviews (1) · Last reviewed commit: "feat: add gated Dreamlabs rollout for du..." · Reviewed by Greptile

Comment on lines +199 to +201
if episode is not None:
episode.stop.set()
self._cancel_locked(episode)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Old rollout stops new motion

stop_rollout() cancels the coordinator whenever a saved episode exists, even after that episode has ended. If an operator or planner starts a new trajectory on the same coordinator, another stop call—or module shutdown—aborts that caller's motion. The delayed cancellation in _run() also aborts new motion when inference returns after manual takeover. All three paths were reproduced with the real runtime, coordinator, and trajectory task.

Track whether the episode still owns the current trajectory, and make both explicit cancellation and delayed cleanup respect ownership before merging.

Knowledge Base Used:

Artifacts

Full offline cancellation ownership reproduction

  • The executed pytest source exercises three cancellation triggers using real runtime, coordinator, and trajectory methods with offline fixtures, making the reproduction repeatable.

Control run without a saved rollout episode

  • Ran all three control cases without a saved episode and observed accepted motion continue after stop/shutdown calls, establishing the unaffected comparison.

Reproduction with an ended or stopping rollout episode

  • Ran repeated stop, module shutdown, and delayed cleanup after takeover and observed each abort newly accepted motion, confirming the cancellation ownership defect.

Scoped runtime and coordinator regression results

  • Executed the existing runtime, coordinator command, and trajectory tests and observed 40 passes, showing the existing suite does not prevent this reproduced defect.

Executed source and cancellation path locations

  • Captured the tested commit, unchanged tracked diff, numbered cancellation code, and full authored reproduction source, tying the observed behavior to exact code locations.

View artifacts

TREX Ran code and verified through TREX

@codecov

codecov Bot commented Oct 20, 2026

Copy link
Copy Markdown

❌ 1 Tests Failed:

Tests completed Failed Passed Skipped
6264 1 6263 46
View the top 1 failed test(s) by shortest run time
dimos.codebase_checks.test_inline_heavy_imports::test_heavy_imports_are_inline
Stack Traces | 2.43s run time
def test_heavy_imports_are_inline() -> None:
        """Fail if any file imports cv2/open3d/rerun at module level."""
        hits = find_eager_heavy_imports()
        if hits:
            listing = "\n".join(
                f"  - dimos/{f}:{line}: `{module}`"
                for f, lines in sorted(hits.items())
                for line, module in lines
            )
>           raise AssertionError(
                f"Found module-level import(s) of {'/'.join(HEAVY_MODULES)}:\n{listing}\n\n"
                "These libraries load large native extensions into every process that "
                "transitively imports the module. Import them inside the function or "
                "method that uses them; imports needed only for type annotations go "
                "under `if TYPE_CHECKING:`."
            )
E           AssertionError: Found module-level import(s) of cv2/open3d/rerun:
E             - .../policy/dreamlabs/conversion.py:21: `cv2`
E           
E           These libraries load large native extensions into every process that transitively imports the module. Import them inside the function or method that uses them; imports needed only for type annotations go under `if TYPE_CHECKING:`.

hits       = {'.../policy/dreamlabs/conversion.py': [(21, 'cv2')]}
listing    = '  - .../policy/dreamlabs/conversion.py:21: `cv2`'

dimos/codebase_checks/test_inline_heavy_imports.py:92: AssertionError

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

This branch was successfully deployed

1 active (outdated) deployment
cachix — 96d80731 Deployed Oct 20, 2026 by TomCC7 via cachix-build-macos #13360
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant