Skip to content

refactor(perception): migrate experimental object and tracker values to CDR - #4665

Closed
TomCC7 wants to merge 2 commits into
cc/feat/cdr-localize-consumersfrom
cc/feat/cdr-experimental-objects
Closed

TomCC7 wants to merge 2 commits into
cc/feat/cdr-localize-consumersfrom
cc/feat/cdr-experimental-objects

Conversation

@TomCC7

@TomCC7 TomCC7 commented Oct 20, 2026

Copy link
Copy Markdown
Member

Linear project: Message Encoding Refactor

Experimental Object/ObjectDB, scene registration, 2D/3D tracking and spatial-memory image/TF boundaries now use generated values from dimos.messages. EdgeTAM and YOLO-E segmentation use the shared pixel helpers. Public processing methods, ports, projection geometry and model choices remain unchanged; the tracker optical-to-robot conversion uses the same coordinate mapping and facing yaw with generated Point/Quaternion values.

Object pose/detection headers retain exact color source time, while projected clouds retain depth source time. ObjectDB's repeated-frame check compares source frame and integer nanoseconds, preserving zero stamps and distinguishing adjacent nanoseconds even when application seconds round together. Pending TTL still uses its separate wall clock. Source-coherent scene snapshots continue to be captured before slow detection; no fallback to an unstamped TF query is added.

Validation

At 2cbe71ea2b6489932787cbae38516dfb91c42bef:

  • 30 focused source checks and 30 installed-main-wheel checks pass; four existing heavyweight spatial-memory cases are explicitly deselected, not claimed passed.
  • Real Open3D RGB-D projection/filtering/box creation and coherent scene projection execute. Color and depth differ by one nanosecond in the projection fixture; the corresponding output headers both remain exact. Generated object values encode/decode through actual CDR, including independent returned geometry.
  • Actual Module ports publish generated detection/overlay/TF values through local LCM memq. Only the external OpenCV tracker boundary is mocked. Invalid depth does not produce a 3D result or TF.
  • Real ObjectDB instances verify repeated-frame exclusion, distinct adjacent-nanosecond frames, zero time, promotion, nearest lookup and wall-clock TTL. Existing request/scan concurrency coverage retains event-based cleanup.
  • Segmentation pixel/metadata preparation executes with mocked model boundaries, and spatial-memory RGB/TF inputs reach the mocked embedding/vector backend with checked BGR colors and world pose. No new model weights, complete neural model run or hardware acceptance claim.
  • Strict typing passes ten production sources; applicable repository hooks, normal main wheel build and default dependency pip check (198 packages) pass.
  • Wheel SHA256: 2776bbd52e5ce86d99fab81e3719806c0b286608942c0a3eca9ca3b7dc24ae2d.

CI has not been polled; benchmarks remain paused. No LAN, robot or camera activation.

Review check

Use Python 3.12 with this branch's normal package installation and existing perception test dependencies. From the checkout, run these offline regressions:

python -m pytest --noconftest --import-mode=importlib -o addopts='' \
  -o 'markers=self_hosted: optional heavyweight integration' --strict-markers \
  -m 'not self_hosted' -q \
  dimos/perception/experimental/test_object_generated.py \
  dimos/perception/experimental/test_object_scene_registration_temporal.py \
  dimos/perception/experimental/test_generated_tracker.py \
  dimos/perception/experimental/test_spatial_memory.py \
  dimos/models/segmentation/test_generated_edge_tam_images.py \
  dimos/models/segmentation/test_yoloe.py

The perception guide explains the message/time boundaries. This slice adds no new user-facing command or API.

File change summary

Parent-relative diff from #4664 at 9c7c0805c4758f60e0ac032942e5f61e868f4fb4: 18 files, 1172 additions, 618 deletions.

Category Files Scope
Experimental consumers 8 Generated object, scene, tracking and spatial boundaries
Segmentation inputs 2 Shared generated pixel conversion
Regression tests 6 Real values, Open3D/CDR acceptance and model boundary tests
Docs and CI 2 Boundary documentation and installed-wheel gates

Stack and limits

Depends on #4664. Temporal-memory consumers, depth native stack, ZED, mapping/navigation, controls/robots, default cutover and legacy retirement remain later work. The recent additive #4247 main merge will be propagated through the chain during final reconciliation, preserving all existing commits. No merge, history rewrite, package publication or permissions change requested.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

go2 replay realtime (arm64)

Details
Benchmark suite Current: 5ee0615 Previous: fbe126e Ratio
peak memory 1746.477 MB 1564.824 MB 1.12
peak threads 371 threads 357 threads 1.04
network (transport) 2719.047 MB 2727.328 MB 1.00
disk write 1.648 MB 1.488 MB 1.11
instructions 133.488 G 114.929 G 1.16

This comment was automatically generated by workflow using github-action-benchmark.

@codecov

codecov Bot commented Oct 20, 2026

Copy link
Copy Markdown

❌ 4 Tests Failed:

Tests completed Failed Passed Skipped
4 4 0 19
View the top 3 failed test(s) by shortest run time
::dimos.robot.unitree.go2.blueprints.navigation.test_go2_sim_nav
Stack Traces | 0s run time
.../blueprints/navigation/test_go2_sim_nav.py:22: in <module>
    from dimos.robot.unitree.go2.blueprints.basic.go2_sim import go2_sim
        Blueprint  = <class 'dimos.core.coordination.blueprints.Blueprint'>
        BlueprintAtom = <class 'dimos.core.coordination.blueprints.BlueprintAtom'>
        ModuleBase = <class 'dimos.core.module.ModuleBase'>
        MovementManager = <class 'dimos.navigation.movement_manager.movement_manager.MovementManager'>
        __builtins__ = <builtins>
        __cached__ = '.../navigation/__pycache__/test_go2_sim_nav.cpython-312.pyc'
        __doc__    = 'The simulator runs the navigation the robot runs, unmodified.'
        __file__   = '/__w/dimos/dimos/.../blueprints/navigation/test_go2_sim_nav.py'
        __loader__ = <_pytest.assertion.rewrite.AssertionRewritingHook object at 0x7dfa5a0c76e0>
        __name__   = 'dimos.robot.unitree.go2.blueprints.navigation.test_go2_sim_nav'
        __package__ = 'dimos.robot.unitree.go2.blueprints.navigation'
        __spec__   = ModuleSpec(name='dimos.robot.unitree.go2.blueprints.navigation.test_go2_sim_nav', loader=<_pytest.assertion.rewrite.As...object at 0x7dfa5a0c76e0>, origin='/__w/dimos/dimos/.../blueprints/navigation/test_go2_sim_nav.py')
        pytest     = <module 'pytest' from '.../dimos/dimos/.venv/lib/python3.12.../site-packages/pytest/__init__.py'>
.../blueprints/basic/go2_sim.py:28: in <module>
    from dimos.simulation.go2_sim.world import SimGo2World
        Any        = typing.Any
        Go2Mid360StaticTf = <class 'dimos.robot.unitree.go2.go2_mid360_static_transforms.Go2Mid360StaticTf'>
        MovementManager = <class 'dimos.navigation.movement_manager.movement_manager.MovementManager'>
        ROBOT_HEIGHT = 0.45
        ROBOT_LENGTH = 0.6858
        ROBOT_WIDTH = 0.3175
        TYPE_CHECKING = False
        __annotations__ = {}
        __builtins__ = <builtins>
        __cached__ = '.../basic/__pycache__/go2_sim.cpython-312.pyc'
        __doc__    = 'The legged Go2 with its Mid-360 in a generated office, driven from the rerun viewer.'
        __file__   = '/__w/dimos/dimos/.../blueprints/basic/go2_sim.py'
        __loader__ = <_frozen_importlib_external.SourceFileLoader object at 0x7df7d90362d0>
        __name__   = 'dimos.robot.unitree.go2.blueprints.basic.go2_sim'
        __package__ = 'dimos.robot.unitree.go2.blueprints.basic'
        __spec__   = ModuleSpec(name='dimos.robot.unitree.go2.blueprints.basic.go2_sim', loader=<_frozen_importlib_external.SourceFileLoader object at 0x7df7d90362d0>, origin='/__w/dimos/dimos/.../blueprints/basic/go2_sim.py')
        annotations = _Feature((3, 7, 0, 'beta', 1), None, 16777216)
        autoconnect = <function autoconnect at 0x7dfa44610fe0>
        body_on_base_link = <function body_on_base_link at 0x7df7ddf9f420>
        global_config = GlobalConfig(robot_ip=None, robot_ips=None, unitree_aes_128_key=None, xarm7_ip=None, xarm6_ip=None, lite6_ip=None, can...load_retries=2, dimos_upload_chunk_mb=None, dimos_upload_quiet_s=30.0, dimos_http_timeout=60.0, dimos_staging_dir=None)
        partial    = <class 'functools.partial'>
.../simulation/go2_sim/world.py:115: in <module>
    MOUNT_XYZ, MOUNT_R = _mount()
        BOX_EDGES  = array([[[         -1,          -1,          -1],
        [          1,          -1,          -1]],

       [[         ... 1,           1]],

       [[          1,           1,          -1],
        [          1,           1,           1]]])
        BOX_RGBA   = {'ceiling': (0.95, 0.95, 0.95, 1.0), 'clutter': (0.45, 0.55, 0.7, 1.0), 'wall': (0.86, 0.84, 0.78, 1.0)}
        CEILING_GROUP = 1
        CMD_VEL_TIMEOUT = 0.2
        COLLISION_GROUP = 3
        CONTROL_DT = 0.02
        Contact    = <class 'dimos.msgs.sim_msgs.Contacts.Contact'>
        Contacts   = <class 'dimos.msgs.sim_msgs.Contacts.Contacts'>
        DEFAULT_THREAD_JOIN_TIMEOUT = 2.0
        Disposable = <class 'reactivex.disposable.disposable.Disposable'>
        Event      = <class 'threading.Event'>
        FLOOR_CHECKER = ((0.62, 0.6, 0.56), (0.7, 0.68, 0.64))
        FRAMES     = [('base_link', None, (0.0, 0.0, 0.0), (0.0, 0.0, 0.0)), ('front_camera', 'base_link', (0.32715, -3e-05, 0.04297), (0.0...511965976, 0.0)), ('camera_optical', 'front_camera', (0.0, 0.0, 0.0), (-1.5707963267948966, 0.0, -1.5707963267948966))]
        FRAME_DT   = 0.1
        Family     = typing.Literal['office']
        Field      = <function Field at 0x7dfa565c8720>
        Go2Policy  = <class 'dimos.simulation.go2_legged.policy.Go2Policy'>
        In         = <class 'dimos.core.stream.In'>
        LIDAR_GROUPS = array([1, 1, 1, 0, 0, 0], dtype=uint8)
        LIDAR_HALF_EXTENTS = (0.0325, 0.0325, 0.03)
        LeggedGo2  = <class 'dimos.simulation.go2_legged.robot.LeggedGo2'>
        LineSegments3D = <class 'dimos.msgs.nav_msgs.LineSegments3D.LineSegments3D'>
        Module     = <class 'dimos.core.module.Module'>
        ModuleConfig = <class 'dimos.core.module.ModuleConfig'>
        MujocoRaycaster = <class 'dimos.simulation.sensors.mujoco_raycaster.MujocoRaycaster'>
        NDArray    = numpy.ndarray[tuple[typing.Any, ...], numpy.dtype[~_ScalarT]]
        ODOM_FRAME_ID = 'odom'
        Odometry   = <class 'dimos.msgs.nav_msgs.Odometry.Odometry'>
        Out        = <class 'dimos.core.stream.Out'>
        POINT_RATE = 200000.0
        Part       = typing.Literal['trunk', 'lidar', 'leg', 'foot']
        Path       = <class 'pathlib.Path'>
        PointCloud2 = <class 'dimos.msgs.sensor_msgs.PointCloud2.PointCloud2'>
        Pose       = <class 'dimos.msgs.geometry_msgs.Pose.Pose'>
        PoseStamped = <class 'dimos.msgs.geometry_msgs.PoseStamped.PoseStamped'>
        Quaternion = <class 'dimos.msgs.geometry_msgs.Quaternion.Quaternion'>
        ROBOT_VISUAL_GROUP = 2
        Rotation   = <class 'scipy.spatial.transform._rotation.Rotation'>
        SCENE_GROUP = 0
        SCENE_PUBLISH_DT = 2.0
        SENSOR_FRAME_ID = 'mid360_link'
        STILL      = array([          0,           0,           0])
        Scene      = <class 'dimos.simulation.scenes.procedural.Scene'>
        SimMid360  = <class 'dimos.simulation.sensors.mid360.lidar.SimMid360'>
        TFMessage  = <class 'dimos.msgs.tf2_msgs.TFMessage.TFMessage'>
        TICKS_PER_FRAME = 5
        Thread     = <class 'threading.Thread'>
        Transform  = <class 'dimos.msgs.geometry_msgs.Transform.Transform'>
        Twist      = <class 'dimos.msgs.geometry_msgs.Twist.Twist'>
        Vector3    = <class 'dimos.msgs.geometry_msgs.Vector3.Vector3'>
        __builtins__ = <builtins>
        __cached__ = '.../go2_sim/__pycache__/world.cpython-312.pyc'
        __doc__    = "The simulated world: a generated scene, the legged Go2 and its Mid-360.\n\nStands in for the Go2 driver and PointLio, publishing PointLio's output contract from\nground truth.\n"
        __file__   = '/__w/dimos/dimos/.../simulation/go2_sim/world.py'
        __loader__ = <_frozen_importlib_external.SourceFileLoader object at 0x7df7d9035f70>
        __name__   = 'dimos.simulation.go2_sim.world'
        __package__ = 'dimos.simulation.go2_sim'
        __spec__   = ModuleSpec(name='dimos.simulation.go2_sim.world', loader=<_frozen_importlib_external.SourceFileLoader object at 0x7df7d9035f70>, origin='/__w/dimos/dimos/.../simulation/go2_sim/world.py')
        _mount     = <function _mount at 0x7df7d96999e0>
        annotations = _Feature((3, 7, 0, 'beta', 1), None, 16777216)
        apply_fitted_physics = <function apply_fitted_physics at 0x7df7d9699b20>
        dataclass  = <function dataclass at 0x7dfa5a640040>
        frames_to_edge_transforms = <function frames_to_edge_transforms at 0x7df7de3b6840>
        generate   = <function generate at 0x7df7d969b4c0>
        go2_spec   = <function go2_spec at 0x7df7d9699a80>
        load_policy = <function load_policy at 0x7df7d96994e0>
        logger     = <BoundLoggerLazyProxy(logger=<_FixedFindCallerLogger .../simulation/go2_sim/world.py (INFO)>, wrapper_class=<class 'structlog.stdlib.BoundLogger'>, processors=None, context_class=None, initial_values={}, logger_factory_args=())>
        math       = <module 'math' (built-in)>
        mujoco     = <module 'mujoco' from '.../dimos/dimos/.venv/lib/python3.12.../site-packages/mujoco/__init__.py'>
        np         = <module 'numpy' from '.../dimos/dimos/.venv/lib/python3.12.../site-packages/numpy/__init__.py'>
        rpc        = <function rpc at 0x7dfa4e308ea0>
        setup_logger = <function setup_logger at 0x7dfa4e142520>
        time       = <module 'time' (built-in)>
.../simulation/go2_sim/world.py:111: in _mount
    matrix = edges["front_camera"].to_matrix() @ edges["mid360_link"].to_matrix()
E   AttributeError: 'geometry_msgs__msg__TransformStamped' object has no attribute 'to_matrix'
        edges      = {'camera_optical': geometry_msgs__msg__TransformStamped(header=std_msgs__msg__Header(stamp=builtin_interfaces__msg__Ti...y_msgs/msg/Quaternion'), __msgtype__='geometry_msgs/msg/Transform'), __msgtype__='geometry_msgs/msg/TransformStamped')}
::dimos.robot.unitree.go2.blueprints.test_tf_topology
Stack Traces | 0s run time
.../go2/blueprints/test_tf_topology.py:34: in <module>
    from dimos.robot.unitree.go2.blueprints.navigation.go2_sim_nav import go2_sim_nav
        Blueprint  = <class 'dimos.core.coordination.blueprints.Blueprint'>
        PointLio   = <class 'dimos.hardware.sensors.lidar.pointlio.module.PointLio'>
        PoseStamped = <class 'dimos.msgs.geometry_msgs.PoseStamped.PoseStamped'>
        RelocalizationModule = <class 'dimos.mapping.relocalization.module.RelocalizationModule'>
        __builtins__ = <builtins>
        __cached__ = '.../blueprints/__pycache__/test_tf_topology.cpython-312.pyc'
        __doc__    = 'Every Go2 blueprint publishes a tf tree, not a tf graph.\n\ntf gives each frame one parent. Two publishers writing th...mount tree owns base_link, and the static tree is rooted at\nmid360_link so it never writes the frame PointLio owns.\n'
        __file__   = '/__w/dimos/dimos/.../go2/blueprints/test_tf_topology.py'
        __loader__ = <_pytest.assertion.rewrite.AssertionRewritingHook object at 0x7dfa5a0c76e0>
        __name__   = 'dimos.robot.unitree.go2.blueprints.test_tf_topology'
        __package__ = 'dimos.robot.unitree.go2.blueprints'
        __spec__   = ModuleSpec(name='dimos.robot.unitree.go2.blueprints.test_tf_topology', loader=<_pytest.assertion.rewrite.AssertionRewritingHook object at 0x7dfa5a0c76e0>, origin='/__w/dimos/dimos/.../go2/blueprints/test_tf_topology.py')
        pytest     = <module 'pytest' from '.../dimos/dimos/.venv/lib/python3.12.../site-packages/pytest/__init__.py'>
        unitree_go2_mid360_record = Blueprint(blueprints=(BlueprintAtom(kwargs={}, module=<class 'dimos.navigation.movement_manager.movement_manager.Movem..., ('keyboardteleop', 'cmd_vel'): 'tele_cmd_vel'}), requirement_checks=(), configurator_checks=(), hosted_placements=())
.../blueprints/navigation/go2_sim_nav.py:24: in <module>
    from dimos.robot.unitree.go2.blueprints.basic.go2_sim import go2_sim, rerun_blueprint, rerun_config
        Any        = typing.Any
        __annotations__ = {}
        __builtins__ = <builtins>
        __cached__ = '.../navigation/__pycache__/go2_sim_nav.cpython-312.pyc'
        __doc__    = 'Go2 navigation driving the simulated legged Go2 to a clicked goal.'
        __file__   = '/__w/dimos/dimos/.../blueprints/navigation/go2_sim_nav.py'
        __loader__ = <_frozen_importlib_external.SourceFileLoader object at 0x7df7d904d1c0>
        __name__   = 'dimos.robot.unitree.go2.blueprints.navigation.go2_sim_nav'
        __package__ = 'dimos.robot.unitree.go2.blueprints.navigation'
        __spec__   = ModuleSpec(name='dimos.robot.unitree.go2.blueprints.navigation.go2_sim_nav', loader=<_frozen_importlib_external.Source...ader object at 0x7df7d904d1c0>, origin='/__w/dimos/dimos/.../blueprints/navigation/go2_sim_nav.py')
        annotations = _Feature((3, 7, 0, 'beta', 1), None, 16777216)
        autoconnect = <function autoconnect at 0x7dfa44610fe0>
        global_config = GlobalConfig(robot_ip=None, robot_ips=None, unitree_aes_128_key=None, xarm7_ip=None, xarm6_ip=None, lite6_ip=None, can...load_retries=2, dimos_upload_chunk_mb=None, dimos_upload_quiet_s=30.0, dimos_http_timeout=60.0, dimos_staging_dir=None)
        partial    = <class 'functools.partial'>
.../blueprints/basic/go2_sim.py:28: in <module>
    from dimos.simulation.go2_sim.world import SimGo2World
        Any        = typing.Any
        Go2Mid360StaticTf = <class 'dimos.robot.unitree.go2.go2_mid360_static_transforms.Go2Mid360StaticTf'>
        MovementManager = <class 'dimos.navigation.movement_manager.movement_manager.MovementManager'>
        ROBOT_HEIGHT = 0.45
        ROBOT_LENGTH = 0.6858
        ROBOT_WIDTH = 0.3175
        TYPE_CHECKING = False
        __annotations__ = {}
        __builtins__ = <builtins>
        __cached__ = '.../basic/__pycache__/go2_sim.cpython-312.pyc'
        __doc__    = 'The legged Go2 with its Mid-360 in a generated office, driven from the rerun viewer.'
        __file__   = '/__w/dimos/dimos/.../blueprints/basic/go2_sim.py'
        __loader__ = <_frozen_importlib_external.SourceFileLoader object at 0x7df7d904d910>
        __name__   = 'dimos.robot.unitree.go2.blueprints.basic.go2_sim'
        __package__ = 'dimos.robot.unitree.go2.blueprints.basic'
        __spec__   = ModuleSpec(name='dimos.robot.unitree.go2.blueprints.basic.go2_sim', loader=<_frozen_importlib_external.SourceFileLoader object at 0x7df7d904d910>, origin='/__w/dimos/dimos/.../blueprints/basic/go2_sim.py')
        annotations = _Feature((3, 7, 0, 'beta', 1), None, 16777216)
        autoconnect = <function autoconnect at 0x7dfa44610fe0>
        body_on_base_link = <function body_on_base_link at 0x7df7ddf9f420>
        global_config = GlobalConfig(robot_ip=None, robot_ips=None, unitree_aes_128_key=None, xarm7_ip=None, xarm6_ip=None, lite6_ip=None, can...load_retries=2, dimos_upload_chunk_mb=None, dimos_upload_quiet_s=30.0, dimos_http_timeout=60.0, dimos_staging_dir=None)
        partial    = <class 'functools.partial'>
.../simulation/go2_sim/world.py:115: in <module>
    MOUNT_XYZ, MOUNT_R = _mount()
        BOX_EDGES  = array([[[         -1,          -1,          -1],
        [          1,          -1,          -1]],

       [[         ... 1,           1]],

       [[          1,           1,          -1],
        [          1,           1,           1]]])
        BOX_RGBA   = {'ceiling': (0.95, 0.95, 0.95, 1.0), 'clutter': (0.45, 0.55, 0.7, 1.0), 'wall': (0.86, 0.84, 0.78, 1.0)}
        CEILING_GROUP = 1
        CMD_VEL_TIMEOUT = 0.2
        COLLISION_GROUP = 3
        CONTROL_DT = 0.02
        Contact    = <class 'dimos.msgs.sim_msgs.Contacts.Contact'>
        Contacts   = <class 'dimos.msgs.sim_msgs.Contacts.Contacts'>
        DEFAULT_THREAD_JOIN_TIMEOUT = 2.0
        Disposable = <class 'reactivex.disposable.disposable.Disposable'>
        Event      = <class 'threading.Event'>
        FLOOR_CHECKER = ((0.62, 0.6, 0.56), (0.7, 0.68, 0.64))
        FRAMES     = [('base_link', None, (0.0, 0.0, 0.0), (0.0, 0.0, 0.0)), ('front_camera', 'base_link', (0.32715, -3e-05, 0.04297), (0.0...511965976, 0.0)), ('camera_optical', 'front_camera', (0.0, 0.0, 0.0), (-1.5707963267948966, 0.0, -1.5707963267948966))]
        FRAME_DT   = 0.1
        Family     = typing.Literal['office']
        Field      = <function Field at 0x7dfa565c8720>
        Go2Policy  = <class 'dimos.simulation.go2_legged.policy.Go2Policy'>
        In         = <class 'dimos.core.stream.In'>
        LIDAR_GROUPS = array([1, 1, 1, 0, 0, 0], dtype=uint8)
        LIDAR_HALF_EXTENTS = (0.0325, 0.0325, 0.03)
        LeggedGo2  = <class 'dimos.simulation.go2_legged.robot.LeggedGo2'>
        LineSegments3D = <class 'dimos.msgs.nav_msgs.LineSegments3D.LineSegments3D'>
        Module     = <class 'dimos.core.module.Module'>
        ModuleConfig = <class 'dimos.core.module.ModuleConfig'>
        MujocoRaycaster = <class 'dimos.simulation.sensors.mujoco_raycaster.MujocoRaycaster'>
        NDArray    = numpy.ndarray[tuple[typing.Any, ...], numpy.dtype[~_ScalarT]]
        ODOM_FRAME_ID = 'odom'
        Odometry   = <class 'dimos.msgs.nav_msgs.Odometry.Odometry'>
        Out        = <class 'dimos.core.stream.Out'>
        POINT_RATE = 200000.0
        Part       = typing.Literal['trunk', 'lidar', 'leg', 'foot']
        Path       = <class 'pathlib.Path'>
        PointCloud2 = <class 'dimos.msgs.sensor_msgs.PointCloud2.PointCloud2'>
        Pose       = <class 'dimos.msgs.geometry_msgs.Pose.Pose'>
        PoseStamped = <class 'dimos.msgs.geometry_msgs.PoseStamped.PoseStamped'>
        Quaternion = <class 'dimos.msgs.geometry_msgs.Quaternion.Quaternion'>
        ROBOT_VISUAL_GROUP = 2
        Rotation   = <class 'scipy.spatial.transform._rotation.Rotation'>
        SCENE_GROUP = 0
        SCENE_PUBLISH_DT = 2.0
        SENSOR_FRAME_ID = 'mid360_link'
        STILL      = array([          0,           0,           0])
        Scene      = <class 'dimos.simulation.scenes.procedural.Scene'>
        SimMid360  = <class 'dimos.simulation.sensors.mid360.lidar.SimMid360'>
        TFMessage  = <class 'dimos.msgs.tf2_msgs.TFMessage.TFMessage'>
        TICKS_PER_FRAME = 5
        Thread     = <class 'threading.Thread'>
        Transform  = <class 'dimos.msgs.geometry_msgs.Transform.Transform'>
        Twist      = <class 'dimos.msgs.geometry_msgs.Twist.Twist'>
        Vector3    = <class 'dimos.msgs.geometry_msgs.Vector3.Vector3'>
        __builtins__ = <builtins>
        __cached__ = '.../go2_sim/__pycache__/world.cpython-312.pyc'
        __doc__    = "The simulated world: a generated scene, the legged Go2 and its Mid-360.\n\nStands in for the Go2 driver and PointLio, publishing PointLio's output contract from\nground truth.\n"
        __file__   = '/__w/dimos/dimos/.../simulation/go2_sim/world.py'
        __loader__ = <_frozen_importlib_external.SourceFileLoader object at 0x7df7d904dbe0>
        __name__   = 'dimos.simulation.go2_sim.world'
        __package__ = 'dimos.simulation.go2_sim'
        __spec__   = ModuleSpec(name='dimos.simulation.go2_sim.world', loader=<_frozen_importlib_external.SourceFileLoader object at 0x7df7d904dbe0>, origin='/__w/dimos/dimos/.../simulation/go2_sim/world.py')
        _mount     = <function _mount at 0x7df7d8da5300>
        annotations = _Feature((3, 7, 0, 'beta', 1), None, 16777216)
        apply_fitted_physics = <function apply_fitted_physics at 0x7df7d9699b20>
        dataclass  = <function dataclass at 0x7dfa5a640040>
        frames_to_edge_transforms = <function frames_to_edge_transforms at 0x7df7de3b6840>
        generate   = <function generate at 0x7df7d969b4c0>
        go2_spec   = <function go2_spec at 0x7df7d9699a80>
        load_policy = <function load_policy at 0x7df7d96994e0>
        logger     = <BoundLoggerLazyProxy(logger=<_FixedFindCallerLogger .../simulation/go2_sim/world.py (INFO)>, wrapper_class=<class 'structlog.stdlib.BoundLogger'>, processors=None, context_class=None, initial_values={}, logger_factory_args=())>
        math       = <module 'math' (built-in)>
        mujoco     = <module 'mujoco' from '.../dimos/dimos/.venv/lib/python3.12.../site-packages/mujoco/__init__.py'>
        np         = <module 'numpy' from '.../dimos/dimos/.venv/lib/python3.12.../site-packages/numpy/__init__.py'>
        rpc        = <function rpc at 0x7dfa4e308ea0>
        setup_logger = <function setup_logger at 0x7dfa4e142520>
        time       = <module 'time' (built-in)>
.../simulation/go2_sim/world.py:111: in _mount
    matrix = edges["front_camera"].to_matrix() @ edges["mid360_link"].to_matrix()
E   AttributeError: 'geometry_msgs__msg__TransformStamped' object has no attribute 'to_matrix'
        edges      = {'camera_optical': geometry_msgs__msg__TransformStamped(header=std_msgs__msg__Header(stamp=builtin_interfaces__msg__Ti...y_msgs/msg/Quaternion'), __msgtype__='geometry_msgs/msg/Transform'), __msgtype__='geometry_msgs/msg/TransformStamped')}
::dimos.simulation.go2_sim.test_command_hold
Stack Traces | 0s run time
.../simulation/go2_sim/test_command_hold.py:19: in <module>
    from dimos.simulation.go2_sim.world import STILL, CommandHold
        __builtins__ = <builtins>
        __cached__ = '.../go2_sim/__pycache__/test_command_hold.cpython-312.pyc'
        __doc__    = None
        __file__   = '/__w/dimos/dimos/.../simulation/go2_sim/test_command_hold.py'
        __loader__ = <_pytest.assertion.rewrite.AssertionRewritingHook object at 0x7dfa5a0c76e0>
        __name__   = 'dimos.simulation.go2_sim.test_command_hold'
        __package__ = 'dimos.simulation.go2_sim'
        __spec__   = ModuleSpec(name='dimos.simulation.go2_sim.test_command_hold', loader=<_pytest.assertion.rewrite.AssertionRewritingHook object at 0x7dfa5a0c76e0>, origin='/__w/dimos/dimos/.../simulation/go2_sim/test_command_hold.py')
        annotations = _Feature((3, 7, 0, 'beta', 1), None, 16777216)
        np         = <module 'numpy' from '.../dimos/dimos/.venv/lib/python3.12....../site-packages/numpy/__init__.py'>
.../simulation/go2_sim/world.py:115: in <module>
    MOUNT_XYZ, MOUNT_R = _mount()
        BOX_EDGES  = array([[[         -1,          -1,          -1],
        [          1,          -1,          -1]],

       [[         ... 1,           1]],

       [[          1,           1,          -1],
        [          1,           1,           1]]])
        BOX_RGBA   = {'ceiling': (0.95, 0.95, 0.95, 1.0), 'clutter': (0.45, 0.55, 0.7, 1.0), 'wall': (0.86, 0.84, 0.78, 1.0)}
        CEILING_GROUP = 1
        CMD_VEL_TIMEOUT = 0.2
        COLLISION_GROUP = 3
        CONTROL_DT = 0.02
        Contact    = <class 'dimos.msgs.sim_msgs.Contacts.Contact'>
        Contacts   = <class 'dimos.msgs.sim_msgs.Contacts.Contacts'>
        DEFAULT_THREAD_JOIN_TIMEOUT = 2.0
        Disposable = <class 'reactivex.disposable.disposable.Disposable'>
        Event      = <class 'threading.Event'>
        FLOOR_CHECKER = ((0.62, 0.6, 0.56), (0.7, 0.68, 0.64))
        FRAMES     = [('base_link', None, (0.0, 0.0, 0.0), (0.0, 0.0, 0.0)), ('front_camera', 'base_link', (0.32715, -3e-05, 0.04297), (0.0...511965976, 0.0)), ('camera_optical', 'front_camera', (0.0, 0.0, 0.0), (-1.5707963267948966, 0.0, -1.5707963267948966))]
        FRAME_DT   = 0.1
        Family     = typing.Literal['office']
        Field      = <function Field at 0x7dfa565c8720>
        Go2Policy  = <class 'dimos.simulation.go2_legged.policy.Go2Policy'>
        In         = <class 'dimos.core.stream.In'>
        LIDAR_GROUPS = array([1, 1, 1, 0, 0, 0], dtype=uint8)
        LIDAR_HALF_EXTENTS = (0.0325, 0.0325, 0.03)
        LeggedGo2  = <class 'dimos.simulation.go2_legged.robot.LeggedGo2'>
        LineSegments3D = <class 'dimos.msgs.nav_msgs.LineSegments3D.LineSegments3D'>
        Module     = <class 'dimos.core.module.Module'>
        ModuleConfig = <class 'dimos.core.module.ModuleConfig'>
        MujocoRaycaster = <class 'dimos.simulation.sensors.mujoco_raycaster.MujocoRaycaster'>
        NDArray    = numpy.ndarray[tuple[typing.Any, ...], numpy.dtype[~_ScalarT]]
        ODOM_FRAME_ID = 'odom'
        Odometry   = <class 'dimos.msgs.nav_msgs.Odometry.Odometry'>
        Out        = <class 'dimos.core.stream.Out'>
        POINT_RATE = 200000.0
        Part       = typing.Literal['trunk', 'lidar', 'leg', 'foot']
        Path       = <class 'pathlib.Path'>
        PointCloud2 = <class 'dimos.msgs.sensor_msgs.PointCloud2.PointCloud2'>
        Pose       = <class 'dimos.msgs.geometry_msgs.Pose.Pose'>
        PoseStamped = <class 'dimos.msgs.geometry_msgs.PoseStamped.PoseStamped'>
        Quaternion = <class 'dimos.msgs.geometry_msgs.Quaternion.Quaternion'>
        ROBOT_VISUAL_GROUP = 2
        Rotation   = <class 'scipy.spatial.transform._rotation.Rotation'>
        SCENE_GROUP = 0
        SCENE_PUBLISH_DT = 2.0
        SENSOR_FRAME_ID = 'mid360_link'
        STILL      = array([          0,           0,           0])
        Scene      = <class 'dimos.simulation.scenes.procedural.Scene'>
        SimMid360  = <class 'dimos.simulation.sensors.mid360.lidar.SimMid360'>
        TFMessage  = <class 'dimos.msgs.tf2_msgs.TFMessage.TFMessage'>
        TICKS_PER_FRAME = 5
        Thread     = <class 'threading.Thread'>
        Transform  = <class 'dimos.msgs.geometry_msgs.Transform.Transform'>
        Twist      = <class 'dimos.msgs.geometry_msgs.Twist.Twist'>
        Vector3    = <class 'dimos.msgs.geometry_msgs.Vector3.Vector3'>
        __builtins__ = <builtins>
        __cached__ = '.../go2_sim/__pycache__/world.cpython-312.pyc'
        __doc__    = "The simulated world: a generated scene, the legged Go2 and its Mid-360.\n\nStands in for the Go2 driver and PointLio, publishing PointLio's output contract from\nground truth.\n"
        __file__   = '/__w/dimos/dimos/.../simulation/go2_sim/world.py'
        __loader__ = <_frozen_importlib_external.SourceFileLoader object at 0x7df7d8e9a810>
        __name__   = 'dimos.simulation.go2_sim.world'
        __package__ = 'dimos.simulation.go2_sim'
        __spec__   = ModuleSpec(name='dimos.simulation.go2_sim.world', loader=<_frozen_importlib_external.SourceFileLoader object at 0x7df7d8e9a810>, origin='/__w/dimos/dimos/.../simulation/go2_sim/world.py')
        _mount     = <function _mount at 0x7df7d9352840>
        annotations = _Feature((3, 7, 0, 'beta', 1), None, 16777216)
        apply_fitted_physics = <function apply_fitted_physics at 0x7df7d9699b20>
        dataclass  = <function dataclass at 0x7dfa5a640040>
        frames_to_edge_transforms = <function frames_to_edge_transforms at 0x7df7de3b6840>
        generate   = <function generate at 0x7df7d969b4c0>
        go2_spec   = <function go2_spec at 0x7df7d9699a80>
        load_policy = <function load_policy at 0x7df7d96994e0>
        logger     = <BoundLoggerLazyProxy(logger=<_FixedFindCallerLogger .../simulation/go2_sim/world.py (INFO)>, wrapper_class=<class 'structlog.stdlib.BoundLogger'>, processors=None, context_class=None, initial_values={}, logger_factory_args=())>
        math       = <module 'math' (built-in)>
        mujoco     = <module 'mujoco' from '.../dimos/dimos/.venv/lib/python3.12.../site-packages/mujoco/__init__.py'>
        np         = <module 'numpy' from '.../dimos/dimos/.venv/lib/python3.12....../site-packages/numpy/__init__.py'>
        rpc        = <function rpc at 0x7dfa4e308ea0>
        setup_logger = <function setup_logger at 0x7dfa4e142520>
        time       = <module 'time' (built-in)>
.../simulation/go2_sim/world.py:111: in _mount
    matrix = edges["front_camera"].to_matrix() @ edges["mid360_link"].to_matrix()
E   AttributeError: 'geometry_msgs__msg__TransformStamped' object has no attribute 'to_matrix'
        edges      = {'camera_optical': geometry_msgs__msg__TransformStamped(header=std_msgs__msg__Header(stamp=builtin_interfaces__msg__Ti...y_msgs/msg/Quaternion'), __msgtype__='geometry_msgs/msg/Transform'), __msgtype__='geometry_msgs/msg/TransformStamped')}
::dimos.simulation.go2_sim.test_world
Stack Traces | 0s run time
.../simulation/go2_sim/test_world.py:25: in <module>
    from dimos.simulation.go2_sim.world import (
        CONTROL_DT = 0.02
        Contact    = <class 'dimos.msgs.sim_msgs.Contacts.Contact'>
        NDArray    = numpy.ndarray[tuple[typing.Any, ...], numpy.dtype[~_ScalarT]]
        OnnxGo2Policy = <class 'dimos.simulation.go2_legged.policy.OnnxGo2Policy'>
        __builtins__ = <builtins>
        __cached__ = '.../go2_sim/__pycache__/test_world.cpython-312.pyc'
        __doc__    = None
        __file__   = '/__w/dimos/dimos/.../simulation/go2_sim/test_world.py'
        __loader__ = <_pytest.assertion.rewrite.AssertionRewritingHook object at 0x7dfa5a0c76e0>
        __name__   = 'dimos.simulation.go2_sim.test_world'
        __package__ = 'dimos.simulation.go2_sim'
        __spec__   = ModuleSpec(name='dimos.simulation.go2_sim.test_world', loader=<_pytest.assertion.rewrite.AssertionRewritingHook object at 0x7dfa5a0c76e0>, origin='/__w/dimos/dimos/.../simulation/go2_sim/test_world.py')
        annotations = _Feature((3, 7, 0, 'beta', 1), None, 16777216)
        mujoco     = <module 'mujoco' from '.../dimos/dimos/.venv/lib/python3.12....../site-packages/mujoco/__init__.py'>
        np         = <module 'numpy' from '.../dimos/dimos/.venv/lib/python3.12....../site-packages/numpy/__init__.py'>
        pytest     = <module 'pytest' from '.../dimos/dimos/.venv/lib/python3.12.../site-packages/pytest/__init__.py'>
.../simulation/go2_sim/world.py:115: in <module>
    MOUNT_XYZ, MOUNT_R = _mount()
        BOX_EDGES  = array([[[         -1,          -1,          -1],
        [          1,          -1,          -1]],

       [[         ... 1,           1]],

       [[          1,           1,          -1],
        [          1,           1,           1]]])
        BOX_RGBA   = {'ceiling': (0.95, 0.95, 0.95, 1.0), 'clutter': (0.45, 0.55, 0.7, 1.0), 'wall': (0.86, 0.84, 0.78, 1.0)}
        CEILING_GROUP = 1
        CMD_VEL_TIMEOUT = 0.2
        COLLISION_GROUP = 3
        CONTROL_DT = 0.02
        Contact    = <class 'dimos.msgs.sim_msgs.Contacts.Contact'>
        Contacts   = <class 'dimos.msgs.sim_msgs.Contacts.Contacts'>
        DEFAULT_THREAD_JOIN_TIMEOUT = 2.0
        Disposable = <class 'reactivex.disposable.disposable.Disposable'>
        Event      = <class 'threading.Event'>
        FLOOR_CHECKER = ((0.62, 0.6, 0.56), (0.7, 0.68, 0.64))
        FRAMES     = [('base_link', None, (0.0, 0.0, 0.0), (0.0, 0.0, 0.0)), ('front_camera', 'base_link', (0.32715, -3e-05, 0.04297), (0.0...511965976, 0.0)), ('camera_optical', 'front_camera', (0.0, 0.0, 0.0), (-1.5707963267948966, 0.0, -1.5707963267948966))]
        FRAME_DT   = 0.1
        Family     = typing.Literal['office']
        Field      = <function Field at 0x7dfa565c8720>
        Go2Policy  = <class 'dimos.simulation.go2_legged.policy.Go2Policy'>
        In         = <class 'dimos.core.stream.In'>
        LIDAR_GROUPS = array([1, 1, 1, 0, 0, 0], dtype=uint8)
        LIDAR_HALF_EXTENTS = (0.0325, 0.0325, 0.03)
        LeggedGo2  = <class 'dimos.simulation.go2_legged.robot.LeggedGo2'>
        LineSegments3D = <class 'dimos.msgs.nav_msgs.LineSegments3D.LineSegments3D'>
        Module     = <class 'dimos.core.module.Module'>
        ModuleConfig = <class 'dimos.core.module.ModuleConfig'>
        MujocoRaycaster = <class 'dimos.simulation.sensors.mujoco_raycaster.MujocoRaycaster'>
        NDArray    = numpy.ndarray[tuple[typing.Any, ...], numpy.dtype[~_ScalarT]]
        ODOM_FRAME_ID = 'odom'
        Odometry   = <class 'dimos.msgs.nav_msgs.Odometry.Odometry'>
        Out        = <class 'dimos.core.stream.Out'>
        POINT_RATE = 200000.0
        Part       = typing.Literal['trunk', 'lidar', 'leg', 'foot']
        Path       = <class 'pathlib.Path'>
        PointCloud2 = <class 'dimos.msgs.sensor_msgs.PointCloud2.PointCloud2'>
        Pose       = <class 'dimos.msgs.geometry_msgs.Pose.Pose'>
        PoseStamped = <class 'dimos.msgs.geometry_msgs.PoseStamped.PoseStamped'>
        Quaternion = <class 'dimos.msgs.geometry_msgs.Quaternion.Quaternion'>
        ROBOT_VISUAL_GROUP = 2
        Rotation   = <class 'scipy.spatial.transform._rotation.Rotation'>
        SCENE_GROUP = 0
        SCENE_PUBLISH_DT = 2.0
        SENSOR_FRAME_ID = 'mid360_link'
        STILL      = array([          0,           0,           0])
        Scene      = <class 'dimos.simulation.scenes.procedural.Scene'>
        SimMid360  = <class 'dimos.simulation.sensors.mid360.lidar.SimMid360'>
        TFMessage  = <class 'dimos.msgs.tf2_msgs.TFMessage.TFMessage'>
        TICKS_PER_FRAME = 5
        Thread     = <class 'threading.Thread'>
        Transform  = <class 'dimos.msgs.geometry_msgs.Transform.Transform'>
        Twist      = <class 'dimos.msgs.geometry_msgs.Twist.Twist'>
        Vector3    = <class 'dimos.msgs.geometry_msgs.Vector3.Vector3'>
        __builtins__ = <builtins>
        __cached__ = '.../go2_sim/__pycache__/world.cpython-312.pyc'
        __doc__    = "The simulated world: a generated scene, the legged Go2 and its Mid-360.\n\nStands in for the Go2 driver and PointLio, publishing PointLio's output contract from\nground truth.\n"
        __file__   = '/__w/dimos/dimos/.../simulation/go2_sim/world.py'
        __loader__ = <_frozen_importlib_external.SourceFileLoader object at 0x7df7d8e97470>
        __name__   = 'dimos.simulation.go2_sim.world'
        __package__ = 'dimos.simulation.go2_sim'
        __spec__   = ModuleSpec(name='dimos.simulation.go2_sim.world', loader=<_frozen_importlib_external.SourceFileLoader object at 0x7df7d8e97470>, origin='/__w/dimos/dimos/.../simulation/go2_sim/world.py')
        _mount     = <function _mount at 0x7df7d9352840>
        annotations = _Feature((3, 7, 0, 'beta', 1), None, 16777216)
        apply_fitted_physics = <function apply_fitted_physics at 0x7df7d9699b20>
        dataclass  = <function dataclass at 0x7dfa5a640040>
        frames_to_edge_transforms = <function frames_to_edge_transforms at 0x7df7de3b6840>
        generate   = <function generate at 0x7df7d969b4c0>
        go2_spec   = <function go2_spec at 0x7df7d9699a80>
        load_policy = <function load_policy at 0x7df7d96994e0>
        logger     = <BoundLoggerLazyProxy(logger=<_FixedFindCallerLogger .../simulation/go2_sim/world.py (INFO)>, wrapper_class=<class 'structlog.stdlib.BoundLogger'>, processors=None, context_class=None, initial_values={}, logger_factory_args=())>
        math       = <module 'math' (built-in)>
        mujoco     = <module 'mujoco' from '.../dimos/dimos/.venv/lib/python3.12....../site-packages/mujoco/__init__.py'>
        np         = <module 'numpy' from '.../dimos/dimos/.venv/lib/python3.12....../site-packages/numpy/__init__.py'>
        rpc        = <function rpc at 0x7dfa4e308ea0>
        setup_logger = <function setup_logger at 0x7dfa4e142520>
        time       = <module 'time' (built-in)>
.../simulation/go2_sim/world.py:111: in _mount
    matrix = edges["front_camera"].to_matrix() @ edges["mid360_link"].to_matrix()
E   AttributeError: 'geometry_msgs__msg__TransformStamped' object has no attribute 'to_matrix'
        edges      = {'camera_optical': geometry_msgs__msg__TransformStamped(header=std_msgs__msg__Header(stamp=builtin_interfaces__msg__Ti...y_msgs/msg/Quaternion'), __msgtype__='geometry_msgs/msg/Transform'), __msgtype__='geometry_msgs/msg/TransformStamped')}

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

@TomCC7 TomCC7 closed this Oct 20, 2026
@TomCC7
TomCC7 deleted the cc/feat/cdr-experimental-objects branch October 20, 2026 18:10

This branch was successfully deployed

1 active deployment
cachix — 5ee06151 Deployed Oct 20, 2026 by autofix-ci[bot] via cachix-build-macos #13395
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant