Skip to content

refactor(perception): migrate fiducial detection and TF to generated CDR - #4663

Closed
TomCC7 wants to merge 1 commit into
cc/feat/cdr-tracking-consumersfrom
cc/feat/cdr-fiducial-consumers
Closed

refactor(perception): migrate fiducial detection and TF to generated CDR#4663
TomCC7 wants to merge 1 commit into
cc/feat/cdr-tracking-consumersfrom
cc/feat/cdr-fiducial-consumers

Conversation

@TomCC7

@TomCC7 TomCC7 commented Oct 20, 2026

Copy link
Copy Markdown
Member

Linear project: Message Encoding Refactor

Fiducial marker detection, per-frame arrays, TF publication and the desk marker blueprint now use generated CDR values from dimos.messages. Existing stream/method names, geometry, quality/speed gates and smoothing remain in place. The desk's explicit LCM backend remains LCM with CDR encoding; global default cutover is later work.

Frame grouping compares integer source nanoseconds rather than rounded Observation seconds. Marker arrays and TF preserve the original image stamp, including zero. OpenCV scalar and one-element marker ID representations are both accepted. Generated CameraInfo configuration preserves exact class identity with Pydantic InstanceOf.

Validation

At b330329940e201771f0a54467275a2aae17af6b0:

  • 40 source and 40 installed-main-wheel regression cases pass, including real OpenCV ArUco/AprilTag detection, twelve-marker layout, pose composition, smoothing, empty frames, zero source time and adjacent-nanosecond frames.
  • Actual generated Detection3DArray -> CDR over LCM memq -> MarkerTfModule -> CDR TFMessage -> TF lookup passes, with exact nanosecond output stamps. No LAN, camera or host-network changes. Event synchronization replaces the old fixed sleeps/repeated publication.
  • The exact documentation example below executes in CI; installed-wheel tests use explicit checkout test-file paths while asserting production code comes from site-packages.
  • Strict typing passes seven production files. All applicable repository hooks, normal main wheel build and default dependency pip check (198 packages) pass.
  • Wheel SHA256: bdfde26ac39dd0a5417a0672f6fb98680edb62c2e1e971736cf8b1a5f38e5e95.

CI has not been polled. No physical camera or hardware acceptance claim.

Manual check

From a fresh Python 3.12 environment installed from this branch using the normal main package install path, install the existing OpenCV contrib dependency if it is not already present. Run this complete CPU-only example; no weights, fixtures or camera are required:

python - <<'PY'
import cv2
import numpy as np

from dimos.messages.builtin_interfaces.msg import Time
from dimos.messages.geometry_msgs.msg import Quaternion, Transform, TransformStamped, Vector3
from dimos.messages.std_msgs.msg import Header
from dimos.msgs.camera_info import camera_info_from_intrinsics
from dimos.msgs.image import image_from_array
from dimos.perception.fiducial.marker_detect import detect_markers_in_image

pixels = np.full((400, 400), 255, dtype=np.uint8)
dictionary = cv2.aruco.getPredefinedDictionary(cv2.aruco.DICT_4X4_50)
pixels[100:300, 100:300] = cv2.aruco.generateImageMarker(dictionary, 7, 200)
header = Header(frame_id="camera_optical", stamp=Time(sec=0, nanosec=123456789))
image = image_from_array(pixels, encoding="mono8", header=header)
camera = camera_info_from_intrinsics(400, 400, 200, 200, 400, 400, header=header)
mount = TransformStamped(
    header=Header(frame_id="world", stamp=header.stamp),
    child_frame_id="camera_optical",
    transform=Transform(translation=Vector3(x=2, y=0, z=0), rotation=Quaternion(x=0, y=0, z=0, w=1)),
)
markers = detect_markers_in_image(
    image, camera_info=camera, world_T_optical=mount,
    marker_length_m=0.2, aruco_dictionary="DICT_4X4_50",
)
message = markers[0].to_detection3d_msg()
assert message.id == "7"
assert message.header == Header(frame_id="world", stamp=header.stamp)
assert abs(message.bbox.center.position.z - 0.4) < 0.01
PY

The same copyable example lives in the perception guide.

File change summary

Parent-relative diff from #4662 at 1c25485f6a3f10113e2f6f539bdcd9b4ebb98603: 18 files, 717 additions, 470 deletions.

Category Files Scope
Detection, transforms and stream consumers 5 Generated message boundaries and exact source metadata
Desk blueprint and fixture helper 2 Existing LCM selection, generated calibration/static TF and OpenCV ID normalization
Tests and shared test values 9 Existing numerical/stream assertions migrated; real generated/CDR and docs acceptance added
Docs and CI 2 Full embedded example, installed-wheel cases and scoped typing

Stack

Depends on #4662. Remaining perception consumers, default runtime encoding cutover and legacy retirement remain later slices. Final stack ancestry/descriptions will be reconciled after those changes; no merge or release is requested.

github-actions[bot]
github-actions Bot reviewed Oct 20, 2026
View reviewed changes

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

go2 replay realtime (arm64)

Details
Benchmark suite Current: b330329 Previous: fbe126e Ratio
peak memory 1729.332 MB 1564.824 MB 1.11
peak threads 370 threads 357 threads 1.04
network (transport) 2727.367 MB 2727.328 MB 1.00
disk write 1.719 MB 1.488 MB 1.16
instructions 133.118 G 114.929 G 1.16

This comment was automatically generated by workflow using github-action-benchmark.

@TomCC7 TomCC7 mentioned this pull request Oct 20, 2026
Closed
@codecov

codecov Bot commented Oct 20, 2026

Copy link
Copy Markdown

❌ 4 Tests Failed:

Tests completed Failed Passed Skipped
4 4 0 19
View the top 3 failed test(s) by shortest run time 
::dimos.robot.unitree.go2.blueprints.navigation.test_go2_sim_nav
Stack Traces | 0s run time 
.../blueprints/navigation/test_go2_sim_nav.py:22: in <module>
    from dimos.robot.unitree.go2.blueprints.basic.go2_sim import go2_sim
        Blueprint  = <class 'dimos.core.coordination.blueprints.Blueprint'>
        BlueprintAtom = <class 'dimos.core.coordination.blueprints.BlueprintAtom'>
        ModuleBase = <class 'dimos.core.module.ModuleBase'>
        MovementManager = <class 'dimos.navigation.movement_manager.movement_manager.MovementManager'>
        __builtins__ = <builtins>
        __cached__ = '.../navigation/__pycache__/test_go2_sim_nav.cpython-312.pyc'
        __doc__    = 'The simulator runs the navigation the robot runs, unmodified.'
        __file__   = '/__w/dimos/dimos/.../blueprints/navigation/test_go2_sim_nav.py'
        __loader__ = <_pytest.assertion.rewrite.AssertionRewritingHook object at 0x7765233bb6e0>
        __name__   = 'dimos.robot.unitree.go2.blueprints.navigation.test_go2_sim_nav'
        __package__ = 'dimos.robot.unitree.go2.blueprints.navigation'
        __spec__   = ModuleSpec(name='dimos.robot.unitree.go2.blueprints.navigation.test_go2_sim_nav', loader=<_pytest.assertion.rewrite.As...object at 0x7765233bb6e0>, origin='/__w/dimos/dimos/.../blueprints/navigation/test_go2_sim_nav.py')
        pytest     = <module 'pytest' from '.../dimos/dimos/.venv/lib/python3.12.../site-packages/pytest/__init__.py'>
.../blueprints/basic/go2_sim.py:28: in <module>
    from dimos.simulation.go2_sim.world import SimGo2World
        Any        = typing.Any
        Go2Mid360StaticTf = <class 'dimos.robot.unitree.go2.go2_mid360_static_transforms.Go2Mid360StaticTf'>
        MovementManager = <class 'dimos.navigation.movement_manager.movement_manager.MovementManager'>
        ROBOT_HEIGHT = 0.45
        ROBOT_LENGTH = 0.6858
        ROBOT_WIDTH = 0.3175
        TYPE_CHECKING = False
        __annotations__ = {}
        __builtins__ = <builtins>
        __cached__ = '.../basic/__pycache__/go2_sim.cpython-312.pyc'
        __doc__    = 'The legged Go2 with its Mid-360 in a generated office, driven from the rerun viewer.'
        __file__   = '/__w/dimos/dimos/.../blueprints/basic/go2_sim.py'
        __loader__ = <_frozen_importlib_external.SourceFileLoader object at 0x7762a23e9b50>
        __name__   = 'dimos.robot.unitree.go2.blueprints.basic.go2_sim'
        __package__ = 'dimos.robot.unitree.go2.blueprints.basic'
        __spec__   = ModuleSpec(name='dimos.robot.unitree.go2.blueprints.basic.go2_sim', loader=<_frozen_importlib_external.SourceFileLoader object at 0x7762a23e9b50>, origin='/__w/dimos/dimos/.../blueprints/basic/go2_sim.py')
        annotations = _Feature((3, 7, 0, 'beta', 1), None, 16777216)
        autoconnect = <function autoconnect at 0x77650dab8fe0>
        body_on_base_link = <function body_on_base_link at 0x7762a721ef20>
        global_config = GlobalConfig(robot_ip=None, robot_ips=None, unitree_aes_128_key=None, xarm7_ip=None, xarm6_ip=None, lite6_ip=None, can...load_retries=2, dimos_upload_chunk_mb=None, dimos_upload_quiet_s=30.0, dimos_http_timeout=60.0, dimos_staging_dir=None)
        partial    = <class 'functools.partial'>
.../simulation/go2_sim/world.py:115: in <module>
    MOUNT_XYZ, MOUNT_R = _mount()
        BOX_EDGES  = array([[[         -1,          -1,          -1],
        [          1,          -1,          -1]],

       [[         ... 1,           1]],

       [[          1,           1,          -1],
        [          1,           1,           1]]])
        BOX_RGBA   = {'ceiling': (0.95, 0.95, 0.95, 1.0), 'clutter': (0.45, 0.55, 0.7, 1.0), 'wall': (0.86, 0.84, 0.78, 1.0)}
        CEILING_GROUP = 1
        CMD_VEL_TIMEOUT = 0.2
        COLLISION_GROUP = 3
        CONTROL_DT = 0.02
        Contact    = <class 'dimos.msgs.sim_msgs.Contacts.Contact'>
        Contacts   = <class 'dimos.msgs.sim_msgs.Contacts.Contacts'>
        DEFAULT_THREAD_JOIN_TIMEOUT = 2.0
        Disposable = <class 'reactivex.disposable.disposable.Disposable'>
        Event      = <class 'threading.Event'>
        FLOOR_CHECKER = ((0.62, 0.6, 0.56), (0.7, 0.68, 0.64))
        FRAMES     = [('base_link', None, (0.0, 0.0, 0.0), (0.0, 0.0, 0.0)), ('front_camera', 'base_link', (0.32715, -3e-05, 0.04297), (0.0...511965976, 0.0)), ('camera_optical', 'front_camera', (0.0, 0.0, 0.0), (-1.5707963267948966, 0.0, -1.5707963267948966))]
        FRAME_DT   = 0.1
        Family     = typing.Literal['office']
        Field      = <function Field at 0x77651f844720>
        Go2Policy  = <class 'dimos.simulation.go2_legged.policy.Go2Policy'>
        In         = <class 'dimos.core.stream.In'>
        LIDAR_GROUPS = array([1, 1, 1, 0, 0, 0], dtype=uint8)
        LIDAR_HALF_EXTENTS = (0.0325, 0.0325, 0.03)
        LeggedGo2  = <class 'dimos.simulation.go2_legged.robot.LeggedGo2'>
        LineSegments3D = <class 'dimos.msgs.nav_msgs.LineSegments3D.LineSegments3D'>
        Module     = <class 'dimos.core.module.Module'>
        ModuleConfig = <class 'dimos.core.module.ModuleConfig'>
        MujocoRaycaster = <class 'dimos.simulation.sensors.mujoco_raycaster.MujocoRaycaster'>
        NDArray    = numpy.ndarray[tuple[typing.Any, ...], numpy.dtype[~_ScalarT]]
        ODOM_FRAME_ID = 'odom'
        Odometry   = <class 'dimos.msgs.nav_msgs.Odometry.Odometry'>
        Out        = <class 'dimos.core.stream.Out'>
        POINT_RATE = 200000.0
        Part       = typing.Literal['trunk', 'lidar', 'leg', 'foot']
        Path       = <class 'pathlib.Path'>
        PointCloud2 = <class 'dimos.msgs.sensor_msgs.PointCloud2.PointCloud2'>
        Pose       = <class 'dimos.msgs.geometry_msgs.Pose.Pose'>
        PoseStamped = <class 'dimos.msgs.geometry_msgs.PoseStamped.PoseStamped'>
        Quaternion = <class 'dimos.msgs.geometry_msgs.Quaternion.Quaternion'>
        ROBOT_VISUAL_GROUP = 2
        Rotation   = <class 'scipy.spatial.transform._rotation.Rotation'>
        SCENE_GROUP = 0
        SCENE_PUBLISH_DT = 2.0
        SENSOR_FRAME_ID = 'mid360_link'
        STILL      = array([          0,           0,           0])
        Scene      = <class 'dimos.simulation.scenes.procedural.Scene'>
        SimMid360  = <class 'dimos.simulation.sensors.mid360.lidar.SimMid360'>
        TFMessage  = <class 'dimos.msgs.tf2_msgs.TFMessage.TFMessage'>
        TICKS_PER_FRAME = 5
        Thread     = <class 'threading.Thread'>
        Transform  = <class 'dimos.msgs.geometry_msgs.Transform.Transform'>
        Twist      = <class 'dimos.msgs.geometry_msgs.Twist.Twist'>
        Vector3    = <class 'dimos.msgs.geometry_msgs.Vector3.Vector3'>
        __builtins__ = <builtins>
        __cached__ = '.../go2_sim/__pycache__/world.cpython-312.pyc'
        __doc__    = "The simulated world: a generated scene, the legged Go2 and its Mid-360.\n\nStands in for the Go2 driver and PointLio, publishing PointLio's output contract from\nground truth.\n"
        __file__   = '/__w/dimos/dimos/.../simulation/go2_sim/world.py'
        __loader__ = <_frozen_importlib_external.SourceFileLoader object at 0x7762a23e9610>
        __name__   = 'dimos.simulation.go2_sim.world'
        __package__ = 'dimos.simulation.go2_sim'
        __spec__   = ModuleSpec(name='dimos.simulation.go2_sim.world', loader=<_frozen_importlib_external.SourceFileLoader object at 0x7762a23e9610>, origin='/__w/dimos/dimos/.../simulation/go2_sim/world.py')
        _mount     = <function _mount at 0x7762a2d9e700>
        annotations = _Feature((3, 7, 0, 'beta', 1), None, 16777216)
        apply_fitted_physics = <function apply_fitted_physics at 0x7762a2d9dc60>
        dataclass  = <function dataclass at 0x776523a64040>
        frames_to_edge_transforms = <function frames_to_edge_transforms at 0x7762a746e340>
        generate   = <function generate at 0x7762a2d9f600>
        go2_spec   = <function go2_spec at 0x7762a2d9dbc0>
        load_policy = <function load_policy at 0x7762a2d9d620>
        logger     = <BoundLoggerLazyProxy(logger=<_FixedFindCallerLogger .../simulation/go2_sim/world.py (INFO)>, wrapper_class=<class 'structlog.stdlib.BoundLogger'>, processors=None, context_class=None, initial_values={}, logger_factory_args=())>
        math       = <module 'math' (built-in)>
        mujoco     = <module 'mujoco' from '.../dimos/dimos/.venv/lib/python3.12.../site-packages/mujoco/__init__.py'>
        np         = <module 'numpy' from '.../dimos/dimos/.venv/lib/python3.12.../site-packages/numpy/__init__.py'>
        rpc        = <function rpc at 0x77651ec24ea0>
        setup_logger = <function setup_logger at 0x776517562520>
        time       = <module 'time' (built-in)>
.../simulation/go2_sim/world.py:111: in _mount
    matrix = edges["front_camera"].to_matrix() @ edges["mid360_link"].to_matrix()
E   AttributeError: 'geometry_msgs__msg__TransformStamped' object has no attribute 'to_matrix'
        edges      = {'camera_optical': geometry_msgs__msg__TransformStamped(header=std_msgs__msg__Header(stamp=builtin_interfaces__msg__Ti...y_msgs/msg/Quaternion'), __msgtype__='geometry_msgs/msg/Transform'), __msgtype__='geometry_msgs/msg/TransformStamped')}
::dimos.robot.unitree.go2.blueprints.test_tf_topology
Stack Traces | 0s run time 
.../go2/blueprints/test_tf_topology.py:34: in <module>
    from dimos.robot.unitree.go2.blueprints.navigation.go2_sim_nav import go2_sim_nav
        Blueprint  = <class 'dimos.core.coordination.blueprints.Blueprint'>
        PointLio   = <class 'dimos.hardware.sensors.lidar.pointlio.module.PointLio'>
        PoseStamped = <class 'dimos.msgs.geometry_msgs.PoseStamped.PoseStamped'>
        RelocalizationModule = <class 'dimos.mapping.relocalization.module.RelocalizationModule'>
        __builtins__ = <builtins>
        __cached__ = '.../blueprints/__pycache__/test_tf_topology.cpython-312.pyc'
        __doc__    = 'Every Go2 blueprint publishes a tf tree, not a tf graph.\n\ntf gives each frame one parent. Two publishers writing th...mount tree owns base_link, and the static tree is rooted at\nmid360_link so it never writes the frame PointLio owns.\n'
        __file__   = '/__w/dimos/dimos/.../go2/blueprints/test_tf_topology.py'
        __loader__ = <_pytest.assertion.rewrite.AssertionRewritingHook object at 0x7765233bb6e0>
        __name__   = 'dimos.robot.unitree.go2.blueprints.test_tf_topology'
        __package__ = 'dimos.robot.unitree.go2.blueprints'
        __spec__   = ModuleSpec(name='dimos.robot.unitree.go2.blueprints.test_tf_topology', loader=<_pytest.assertion.rewrite.AssertionRewritingHook object at 0x7765233bb6e0>, origin='/__w/dimos/dimos/.../go2/blueprints/test_tf_topology.py')
        pytest     = <module 'pytest' from '.../dimos/dimos/.venv/lib/python3.12.../site-packages/pytest/__init__.py'>
        unitree_go2_mid360_record = Blueprint(blueprints=(BlueprintAtom(kwargs={}, module=<class 'dimos.navigation.movement_manager.movement_manager.Movem..., ('keyboardteleop', 'cmd_vel'): 'tele_cmd_vel'}), requirement_checks=(), configurator_checks=(), hosted_placements=())
.../blueprints/navigation/go2_sim_nav.py:24: in <module>
    from dimos.robot.unitree.go2.blueprints.basic.go2_sim import go2_sim, rerun_blueprint, rerun_config
        Any        = typing.Any
        __annotations__ = {}
        __builtins__ = <builtins>
        __cached__ = '.../navigation/__pycache__/go2_sim_nav.cpython-312.pyc'
        __doc__    = 'Go2 navigation driving the simulated legged Go2 to a clicked goal.'
        __file__   = '/__w/dimos/dimos/.../blueprints/navigation/go2_sim_nav.py'
        __loader__ = <_frozen_importlib_external.SourceFileLoader object at 0x7762a23f8980>
        __name__   = 'dimos.robot.unitree.go2.blueprints.navigation.go2_sim_nav'
        __package__ = 'dimos.robot.unitree.go2.blueprints.navigation'
        __spec__   = ModuleSpec(name='dimos.robot.unitree.go2.blueprints.navigation.go2_sim_nav', loader=<_frozen_importlib_external.Source...ader object at 0x7762a23f8980>, origin='/__w/dimos/dimos/.../blueprints/navigation/go2_sim_nav.py')
        annotations = _Feature((3, 7, 0, 'beta', 1), None, 16777216)
        autoconnect = <function autoconnect at 0x77650dab8fe0>
        global_config = GlobalConfig(robot_ip=None, robot_ips=None, unitree_aes_128_key=None, xarm7_ip=None, xarm6_ip=None, lite6_ip=None, can...load_retries=2, dimos_upload_chunk_mb=None, dimos_upload_quiet_s=30.0, dimos_http_timeout=60.0, dimos_staging_dir=None)
        partial    = <class 'functools.partial'>
.../blueprints/basic/go2_sim.py:28: in <module>
    from dimos.simulation.go2_sim.world import SimGo2World
        Any        = typing.Any
        Go2Mid360StaticTf = <class 'dimos.robot.unitree.go2.go2_mid360_static_transforms.Go2Mid360StaticTf'>
        MovementManager = <class 'dimos.navigation.movement_manager.movement_manager.MovementManager'>
        ROBOT_HEIGHT = 0.45
        ROBOT_LENGTH = 0.6858
        ROBOT_WIDTH = 0.3175
        TYPE_CHECKING = False
        __annotations__ = {}
        __builtins__ = <builtins>
        __cached__ = '.../basic/__pycache__/go2_sim.cpython-312.pyc'
        __doc__    = 'The legged Go2 with its Mid-360 in a generated office, driven from the rerun viewer.'
        __file__   = '/__w/dimos/dimos/.../blueprints/basic/go2_sim.py'
        __loader__ = <_frozen_importlib_external.SourceFileLoader object at 0x7762a23f9a30>
        __name__   = 'dimos.robot.unitree.go2.blueprints.basic.go2_sim'
        __package__ = 'dimos.robot.unitree.go2.blueprints.basic'
        __spec__   = ModuleSpec(name='dimos.robot.unitree.go2.blueprints.basic.go2_sim', loader=<_frozen_importlib_external.SourceFileLoader object at 0x7762a23f9a30>, origin='/__w/dimos/dimos/.../blueprints/basic/go2_sim.py')
        annotations = _Feature((3, 7, 0, 'beta', 1), None, 16777216)
        autoconnect = <function autoconnect at 0x77650dab8fe0>
        body_on_base_link = <function body_on_base_link at 0x7762a721ef20>
        global_config = GlobalConfig(robot_ip=None, robot_ips=None, unitree_aes_128_key=None, xarm7_ip=None, xarm6_ip=None, lite6_ip=None, can...load_retries=2, dimos_upload_chunk_mb=None, dimos_upload_quiet_s=30.0, dimos_http_timeout=60.0, dimos_staging_dir=None)
        partial    = <class 'functools.partial'>
.../simulation/go2_sim/world.py:115: in <module>
    MOUNT_XYZ, MOUNT_R = _mount()
        BOX_EDGES  = array([[[         -1,          -1,          -1],
        [          1,          -1,          -1]],

       [[         ... 1,           1]],

       [[          1,           1,          -1],
        [          1,           1,           1]]])
        BOX_RGBA   = {'ceiling': (0.95, 0.95, 0.95, 1.0), 'clutter': (0.45, 0.55, 0.7, 1.0), 'wall': (0.86, 0.84, 0.78, 1.0)}
        CEILING_GROUP = 1
        CMD_VEL_TIMEOUT = 0.2
        COLLISION_GROUP = 3
        CONTROL_DT = 0.02
        Contact    = <class 'dimos.msgs.sim_msgs.Contacts.Contact'>
        Contacts   = <class 'dimos.msgs.sim_msgs.Contacts.Contacts'>
        DEFAULT_THREAD_JOIN_TIMEOUT = 2.0
        Disposable = <class 'reactivex.disposable.disposable.Disposable'>
        Event      = <class 'threading.Event'>
        FLOOR_CHECKER = ((0.62, 0.6, 0.56), (0.7, 0.68, 0.64))
        FRAMES     = [('base_link', None, (0.0, 0.0, 0.0), (0.0, 0.0, 0.0)), ('front_camera', 'base_link', (0.32715, -3e-05, 0.04297), (0.0...511965976, 0.0)), ('camera_optical', 'front_camera', (0.0, 0.0, 0.0), (-1.5707963267948966, 0.0, -1.5707963267948966))]
        FRAME_DT   = 0.1
        Family     = typing.Literal['office']
        Field      = <function Field at 0x77651f844720>
        Go2Policy  = <class 'dimos.simulation.go2_legged.policy.Go2Policy'>
        In         = <class 'dimos.core.stream.In'>
        LIDAR_GROUPS = array([1, 1, 1, 0, 0, 0], dtype=uint8)
        LIDAR_HALF_EXTENTS = (0.0325, 0.0325, 0.03)
        LeggedGo2  = <class 'dimos.simulation.go2_legged.robot.LeggedGo2'>
        LineSegments3D = <class 'dimos.msgs.nav_msgs.LineSegments3D.LineSegments3D'>
        Module     = <class 'dimos.core.module.Module'>
        ModuleConfig = <class 'dimos.core.module.ModuleConfig'>
        MujocoRaycaster = <class 'dimos.simulation.sensors.mujoco_raycaster.MujocoRaycaster'>
        NDArray    = numpy.ndarray[tuple[typing.Any, ...], numpy.dtype[~_ScalarT]]
        ODOM_FRAME_ID = 'odom'
        Odometry   = <class 'dimos.msgs.nav_msgs.Odometry.Odometry'>
        Out        = <class 'dimos.core.stream.Out'>
        POINT_RATE = 200000.0
        Part       = typing.Literal['trunk', 'lidar', 'leg', 'foot']
        Path       = <class 'pathlib.Path'>
        PointCloud2 = <class 'dimos.msgs.sensor_msgs.PointCloud2.PointCloud2'>
        Pose       = <class 'dimos.msgs.geometry_msgs.Pose.Pose'>
        PoseStamped = <class 'dimos.msgs.geometry_msgs.PoseStamped.PoseStamped'>
        Quaternion = <class 'dimos.msgs.geometry_msgs.Quaternion.Quaternion'>
        ROBOT_VISUAL_GROUP = 2
        Rotation   = <class 'scipy.spatial.transform._rotation.Rotation'>
        SCENE_GROUP = 0
        SCENE_PUBLISH_DT = 2.0
        SENSOR_FRAME_ID = 'mid360_link'
        STILL      = array([          0,           0,           0])
        Scene      = <class 'dimos.simulation.scenes.procedural.Scene'>
        SimMid360  = <class 'dimos.simulation.sensors.mid360.lidar.SimMid360'>
        TFMessage  = <class 'dimos.msgs.tf2_msgs.TFMessage.TFMessage'>
        TICKS_PER_FRAME = 5
        Thread     = <class 'threading.Thread'>
        Transform  = <class 'dimos.msgs.geometry_msgs.Transform.Transform'>
        Twist      = <class 'dimos.msgs.geometry_msgs.Twist.Twist'>
        Vector3    = <class 'dimos.msgs.geometry_msgs.Vector3.Vector3'>
        __builtins__ = <builtins>
        __cached__ = '.../go2_sim/__pycache__/world.cpython-312.pyc'
        __doc__    = "The simulated world: a generated scene, the legged Go2 and its Mid-360.\n\nStands in for the Go2 driver and PointLio, publishing PointLio's output contract from\nground truth.\n"
        __file__   = '/__w/dimos/dimos/.../simulation/go2_sim/world.py'
        __loader__ = <_frozen_importlib_external.SourceFileLoader object at 0x7762a23f9a60>
        __name__   = 'dimos.simulation.go2_sim.world'
        __package__ = 'dimos.simulation.go2_sim'
        __spec__   = ModuleSpec(name='dimos.simulation.go2_sim.world', loader=<_frozen_importlib_external.SourceFileLoader object at 0x7762a23f9a60>, origin='/__w/dimos/dimos/.../simulation/go2_sim/world.py')
        _mount     = <function _mount at 0x7762a2a51bc0>
        annotations = _Feature((3, 7, 0, 'beta', 1), None, 16777216)
        apply_fitted_physics = <function apply_fitted_physics at 0x7762a2d9dc60>
        dataclass  = <function dataclass at 0x776523a64040>
        frames_to_edge_transforms = <function frames_to_edge_transforms at 0x7762a746e340>
        generate   = <function generate at 0x7762a2d9f600>
        go2_spec   = <function go2_spec at 0x7762a2d9dbc0>
        load_policy = <function load_policy at 0x7762a2d9d620>
        logger     = <BoundLoggerLazyProxy(logger=<_FixedFindCallerLogger .../simulation/go2_sim/world.py (INFO)>, wrapper_class=<class 'structlog.stdlib.BoundLogger'>, processors=None, context_class=None, initial_values={}, logger_factory_args=())>
        math       = <module 'math' (built-in)>
        mujoco     = <module 'mujoco' from '.../dimos/dimos/.venv/lib/python3.12.../site-packages/mujoco/__init__.py'>
        np         = <module 'numpy' from '.../dimos/dimos/.venv/lib/python3.12.../site-packages/numpy/__init__.py'>
        rpc        = <function rpc at 0x77651ec24ea0>
        setup_logger = <function setup_logger at 0x776517562520>
        time       = <module 'time' (built-in)>
.../simulation/go2_sim/world.py:111: in _mount
    matrix = edges["front_camera"].to_matrix() @ edges["mid360_link"].to_matrix()
E   AttributeError: 'geometry_msgs__msg__TransformStamped' object has no attribute 'to_matrix'
        edges      = {'camera_optical': geometry_msgs__msg__TransformStamped(header=std_msgs__msg__Header(stamp=builtin_interfaces__msg__Ti...y_msgs/msg/Quaternion'), __msgtype__='geometry_msgs/msg/Transform'), __msgtype__='geometry_msgs/msg/TransformStamped')}
::dimos.simulation.go2_sim.test_command_hold
Stack Traces | 0s run time 
.../simulation/go2_sim/test_command_hold.py:19: in <module>
    from dimos.simulation.go2_sim.world import STILL, CommandHold
        __builtins__ = <builtins>
        __cached__ = '.../go2_sim/__pycache__/test_command_hold.cpython-312.pyc'
        __doc__    = None
        __file__   = '/__w/dimos/dimos/.../simulation/go2_sim/test_command_hold.py'
        __loader__ = <_pytest.assertion.rewrite.AssertionRewritingHook object at 0x7765233bb6e0>
        __name__   = 'dimos.simulation.go2_sim.test_command_hold'
        __package__ = 'dimos.simulation.go2_sim'
        __spec__   = ModuleSpec(name='dimos.simulation.go2_sim.test_command_hold', loader=<_pytest.assertion.rewrite.AssertionRewritingHook object at 0x7765233bb6e0>, origin='/__w/dimos/dimos/.../simulation/go2_sim/test_command_hold.py')
        annotations = _Feature((3, 7, 0, 'beta', 1), None, 16777216)
        np         = <module 'numpy' from '.../dimos/dimos/.venv/lib/python3.12....../site-packages/numpy/__init__.py'>
.../simulation/go2_sim/world.py:115: in <module>
    MOUNT_XYZ, MOUNT_R = _mount()
        BOX_EDGES  = array([[[         -1,          -1,          -1],
        [          1,          -1,          -1]],

       [[         ... 1,           1]],

       [[          1,           1,          -1],
        [          1,           1,           1]]])
        BOX_RGBA   = {'ceiling': (0.95, 0.95, 0.95, 1.0), 'clutter': (0.45, 0.55, 0.7, 1.0), 'wall': (0.86, 0.84, 0.78, 1.0)}
        CEILING_GROUP = 1
        CMD_VEL_TIMEOUT = 0.2
        COLLISION_GROUP = 3
        CONTROL_DT = 0.02
        Contact    = <class 'dimos.msgs.sim_msgs.Contacts.Contact'>
        Contacts   = <class 'dimos.msgs.sim_msgs.Contacts.Contacts'>
        DEFAULT_THREAD_JOIN_TIMEOUT = 2.0
        Disposable = <class 'reactivex.disposable.disposable.Disposable'>
        Event      = <class 'threading.Event'>
        FLOOR_CHECKER = ((0.62, 0.6, 0.56), (0.7, 0.68, 0.64))
        FRAMES     = [('base_link', None, (0.0, 0.0, 0.0), (0.0, 0.0, 0.0)), ('front_camera', 'base_link', (0.32715, -3e-05, 0.04297), (0.0...511965976, 0.0)), ('camera_optical', 'front_camera', (0.0, 0.0, 0.0), (-1.5707963267948966, 0.0, -1.5707963267948966))]
        FRAME_DT   = 0.1
        Family     = typing.Literal['office']
        Field      = <function Field at 0x77651f844720>
        Go2Policy  = <class 'dimos.simulation.go2_legged.policy.Go2Policy'>
        In         = <class 'dimos.core.stream.In'>
        LIDAR_GROUPS = array([1, 1, 1, 0, 0, 0], dtype=uint8)
        LIDAR_HALF_EXTENTS = (0.0325, 0.0325, 0.03)
        LeggedGo2  = <class 'dimos.simulation.go2_legged.robot.LeggedGo2'>
        LineSegments3D = <class 'dimos.msgs.nav_msgs.LineSegments3D.LineSegments3D'>
        Module     = <class 'dimos.core.module.Module'>
        ModuleConfig = <class 'dimos.core.module.ModuleConfig'>
        MujocoRaycaster = <class 'dimos.simulation.sensors.mujoco_raycaster.MujocoRaycaster'>
        NDArray    = numpy.ndarray[tuple[typing.Any, ...], numpy.dtype[~_ScalarT]]
        ODOM_FRAME_ID = 'odom'
        Odometry   = <class 'dimos.msgs.nav_msgs.Odometry.Odometry'>
        Out        = <class 'dimos.core.stream.Out'>
        POINT_RATE = 200000.0
        Part       = typing.Literal['trunk', 'lidar', 'leg', 'foot']
        Path       = <class 'pathlib.Path'>
        PointCloud2 = <class 'dimos.msgs.sensor_msgs.PointCloud2.PointCloud2'>
        Pose       = <class 'dimos.msgs.geometry_msgs.Pose.Pose'>
        PoseStamped = <class 'dimos.msgs.geometry_msgs.PoseStamped.PoseStamped'>
        Quaternion = <class 'dimos.msgs.geometry_msgs.Quaternion.Quaternion'>
        ROBOT_VISUAL_GROUP = 2
        Rotation   = <class 'scipy.spatial.transform._rotation.Rotation'>
        SCENE_GROUP = 0
        SCENE_PUBLISH_DT = 2.0
        SENSOR_FRAME_ID = 'mid360_link'
        STILL      = array([          0,           0,           0])
        Scene      = <class 'dimos.simulation.scenes.procedural.Scene'>
        SimMid360  = <class 'dimos.simulation.sensors.mid360.lidar.SimMid360'>
        TFMessage  = <class 'dimos.msgs.tf2_msgs.TFMessage.TFMessage'>
        TICKS_PER_FRAME = 5
        Thread     = <class 'threading.Thread'>
        Transform  = <class 'dimos.msgs.geometry_msgs.Transform.Transform'>
        Twist      = <class 'dimos.msgs.geometry_msgs.Twist.Twist'>
        Vector3    = <class 'dimos.msgs.geometry_msgs.Vector3.Vector3'>
        __builtins__ = <builtins>
        __cached__ = '.../go2_sim/__pycache__/world.cpython-312.pyc'
        __doc__    = "The simulated world: a generated scene, the legged Go2 and its Mid-360.\n\nStands in for the Go2 driver and PointLio, publishing PointLio's output contract from\nground truth.\n"
        __file__   = '/__w/dimos/dimos/.../simulation/go2_sim/world.py'
        __loader__ = <_frozen_importlib_external.SourceFileLoader object at 0x7762a2451370>
        __name__   = 'dimos.simulation.go2_sim.world'
        __package__ = 'dimos.simulation.go2_sim'
        __spec__   = ModuleSpec(name='dimos.simulation.go2_sim.world', loader=<_frozen_importlib_external.SourceFileLoader object at 0x7762a2451370>, origin='/__w/dimos/dimos/.../simulation/go2_sim/world.py')
        _mount     = <function _mount at 0x7762a27279c0>
        annotations = _Feature((3, 7, 0, 'beta', 1), None, 16777216)
        apply_fitted_physics = <function apply_fitted_physics at 0x7762a2d9dc60>
        dataclass  = <function dataclass at 0x776523a64040>
        frames_to_edge_transforms = <function frames_to_edge_transforms at 0x7762a746e340>
        generate   = <function generate at 0x7762a2d9f600>
        go2_spec   = <function go2_spec at 0x7762a2d9dbc0>
        load_policy = <function load_policy at 0x7762a2d9d620>
        logger     = <BoundLoggerLazyProxy(logger=<_FixedFindCallerLogger .../simulation/go2_sim/world.py (INFO)>, wrapper_class=<class 'structlog.stdlib.BoundLogger'>, processors=None, context_class=None, initial_values={}, logger_factory_args=())>
        math       = <module 'math' (built-in)>
        mujoco     = <module 'mujoco' from '.../dimos/dimos/.venv/lib/python3.12.../site-packages/mujoco/__init__.py'>
        np         = <module 'numpy' from '.../dimos/dimos/.venv/lib/python3.12....../site-packages/numpy/__init__.py'>
        rpc        = <function rpc at 0x77651ec24ea0>
        setup_logger = <function setup_logger at 0x776517562520>
        time       = <module 'time' (built-in)>
.../simulation/go2_sim/world.py:111: in _mount
    matrix = edges["front_camera"].to_matrix() @ edges["mid360_link"].to_matrix()
E   AttributeError: 'geometry_msgs__msg__TransformStamped' object has no attribute 'to_matrix'
        edges      = {'camera_optical': geometry_msgs__msg__TransformStamped(header=std_msgs__msg__Header(stamp=builtin_interfaces__msg__Ti...y_msgs/msg/Quaternion'), __msgtype__='geometry_msgs/msg/Transform'), __msgtype__='geometry_msgs/msg/TransformStamped')}
::dimos.simulation.go2_sim.test_world
Stack Traces | 0s run time 
.../simulation/go2_sim/test_world.py:25: in <module>
    from dimos.simulation.go2_sim.world import (
        CONTROL_DT = 0.02
        Contact    = <class 'dimos.msgs.sim_msgs.Contacts.Contact'>
        NDArray    = numpy.ndarray[tuple[typing.Any, ...], numpy.dtype[~_ScalarT]]
        OnnxGo2Policy = <class 'dimos.simulation.go2_legged.policy.OnnxGo2Policy'>
        __builtins__ = <builtins>
        __cached__ = '.../go2_sim/__pycache__/test_world.cpython-312.pyc'
        __doc__    = None
        __file__   = '/__w/dimos/dimos/.../simulation/go2_sim/test_world.py'
        __loader__ = <_pytest.assertion.rewrite.AssertionRewritingHook object at 0x7765233bb6e0>
        __name__   = 'dimos.simulation.go2_sim.test_world'
        __package__ = 'dimos.simulation.go2_sim'
        __spec__   = ModuleSpec(name='dimos.simulation.go2_sim.test_world', loader=<_pytest.assertion.rewrite.AssertionRewritingHook object at 0x7765233bb6e0>, origin='/__w/dimos/dimos/.../simulation/go2_sim/test_world.py')
        annotations = _Feature((3, 7, 0, 'beta', 1), None, 16777216)
        mujoco     = <module 'mujoco' from '.../dimos/dimos/.venv/lib/python3.12....../site-packages/mujoco/__init__.py'>
        np         = <module 'numpy' from '.../dimos/dimos/.venv/lib/python3.12....../site-packages/numpy/__init__.py'>
        pytest     = <module 'pytest' from '.../dimos/dimos/.venv/lib/python3.12.../site-packages/pytest/__init__.py'>
.../simulation/go2_sim/world.py:115: in <module>
    MOUNT_XYZ, MOUNT_R = _mount()
        BOX_EDGES  = array([[[         -1,          -1,          -1],
        [          1,          -1,          -1]],

       [[         ... 1,           1]],

       [[          1,           1,          -1],
        [          1,           1,           1]]])
        BOX_RGBA   = {'ceiling': (0.95, 0.95, 0.95, 1.0), 'clutter': (0.45, 0.55, 0.7, 1.0), 'wall': (0.86, 0.84, 0.78, 1.0)}
        CEILING_GROUP = 1
        CMD_VEL_TIMEOUT = 0.2
        COLLISION_GROUP = 3
        CONTROL_DT = 0.02
        Contact    = <class 'dimos.msgs.sim_msgs.Contacts.Contact'>
        Contacts   = <class 'dimos.msgs.sim_msgs.Contacts.Contacts'>
        DEFAULT_THREAD_JOIN_TIMEOUT = 2.0
        Disposable = <class 'reactivex.disposable.disposable.Disposable'>
        Event      = <class 'threading.Event'>
        FLOOR_CHECKER = ((0.62, 0.6, 0.56), (0.7, 0.68, 0.64))
        FRAMES     = [('base_link', None, (0.0, 0.0, 0.0), (0.0, 0.0, 0.0)), ('front_camera', 'base_link', (0.32715, -3e-05, 0.04297), (0.0...511965976, 0.0)), ('camera_optical', 'front_camera', (0.0, 0.0, 0.0), (-1.5707963267948966, 0.0, -1.5707963267948966))]
        FRAME_DT   = 0.1
        Family     = typing.Literal['office']
        Field      = <function Field at 0x77651f844720>
        Go2Policy  = <class 'dimos.simulation.go2_legged.policy.Go2Policy'>
        In         = <class 'dimos.core.stream.In'>
        LIDAR_GROUPS = array([1, 1, 1, 0, 0, 0], dtype=uint8)
        LIDAR_HALF_EXTENTS = (0.0325, 0.0325, 0.03)
        LeggedGo2  = <class 'dimos.simulation.go2_legged.robot.LeggedGo2'>
        LineSegments3D = <class 'dimos.msgs.nav_msgs.LineSegments3D.LineSegments3D'>
        Module     = <class 'dimos.core.module.Module'>
        ModuleConfig = <class 'dimos.core.module.ModuleConfig'>
        MujocoRaycaster = <class 'dimos.simulation.sensors.mujoco_raycaster.MujocoRaycaster'>
        NDArray    = numpy.ndarray[tuple[typing.Any, ...], numpy.dtype[~_ScalarT]]
        ODOM_FRAME_ID = 'odom'
        Odometry   = <class 'dimos.msgs.nav_msgs.Odometry.Odometry'>
        Out        = <class 'dimos.core.stream.Out'>
        POINT_RATE = 200000.0
        Part       = typing.Literal['trunk', 'lidar', 'leg', 'foot']
        Path       = <class 'pathlib.Path'>
        PointCloud2 = <class 'dimos.msgs.sensor_msgs.PointCloud2.PointCloud2'>
        Pose       = <class 'dimos.msgs.geometry_msgs.Pose.Pose'>
        PoseStamped = <class 'dimos.msgs.geometry_msgs.PoseStamped.PoseStamped'>
        Quaternion = <class 'dimos.msgs.geometry_msgs.Quaternion.Quaternion'>
        ROBOT_VISUAL_GROUP = 2
        Rotation   = <class 'scipy.spatial.transform._rotation.Rotation'>
        SCENE_GROUP = 0
        SCENE_PUBLISH_DT = 2.0
        SENSOR_FRAME_ID = 'mid360_link'
        STILL      = array([          0,           0,           0])
        Scene      = <class 'dimos.simulation.scenes.procedural.Scene'>
        SimMid360  = <class 'dimos.simulation.sensors.mid360.lidar.SimMid360'>
        TFMessage  = <class 'dimos.msgs.tf2_msgs.TFMessage.TFMessage'>
        TICKS_PER_FRAME = 5
        Thread     = <class 'threading.Thread'>
        Transform  = <class 'dimos.msgs.geometry_msgs.Transform.Transform'>
        Twist      = <class 'dimos.msgs.geometry_msgs.Twist.Twist'>
        Vector3    = <class 'dimos.msgs.geometry_msgs.Vector3.Vector3'>
        __builtins__ = <builtins>
        __cached__ = '.../go2_sim/__pycache__/world.cpython-312.pyc'
        __doc__    = "The simulated world: a generated scene, the legged Go2 and its Mid-360.\n\nStands in for the Go2 driver and PointLio, publishing PointLio's output contract from\nground truth.\n"
        __file__   = '/__w/dimos/dimos/.../simulation/go2_sim/world.py'
        __loader__ = <_frozen_importlib_external.SourceFileLoader object at 0x7762a2453f50>
        __name__   = 'dimos.simulation.go2_sim.world'
        __package__ = 'dimos.simulation.go2_sim'
        __spec__   = ModuleSpec(name='dimos.simulation.go2_sim.world', loader=<_frozen_importlib_external.SourceFileLoader object at 0x7762a2453f50>, origin='/__w/dimos/dimos/.../simulation/go2_sim/world.py')
        _mount     = <function _mount at 0x7762a2727920>
        annotations = _Feature((3, 7, 0, 'beta', 1), None, 16777216)
        apply_fitted_physics = <function apply_fitted_physics at 0x7762a2d9dc60>
        dataclass  = <function dataclass at 0x776523a64040>
        frames_to_edge_transforms = <function frames_to_edge_transforms at 0x7762a746e340>
        generate   = <function generate at 0x7762a2d9f600>
        go2_spec   = <function go2_spec at 0x7762a2d9dbc0>
        load_policy = <function load_policy at 0x7762a2d9d620>
        logger     = <BoundLoggerLazyProxy(logger=<_FixedFindCallerLogger .../simulation/go2_sim/world.py (INFO)>, wrapper_class=<class 'structlog.stdlib.BoundLogger'>, processors=None, context_class=None, initial_values={}, logger_factory_args=())>
        math       = <module 'math' (built-in)>
        mujoco     = <module 'mujoco' from '.../dimos/dimos/.venv/lib/python3.12....../site-packages/mujoco/__init__.py'>
        np         = <module 'numpy' from '.../dimos/dimos/.venv/lib/python3.12....../site-packages/numpy/__init__.py'>
        rpc        = <function rpc at 0x77651ec24ea0>
        setup_logger = <function setup_logger at 0x776517562520>
        time       = <module 'time' (built-in)>
.../simulation/go2_sim/world.py:111: in _mount
    matrix = edges["front_camera"].to_matrix() @ edges["mid360_link"].to_matrix()
E   AttributeError: 'geometry_msgs__msg__TransformStamped' object has no attribute 'to_matrix'
        edges      = {'camera_optical': geometry_msgs__msg__TransformStamped(header=std_msgs__msg__Header(stamp=builtin_interfaces__msg__Ti...y_msgs/msg/Quaternion'), __msgtype__='geometry_msgs/msg/Transform'), __msgtype__='geometry_msgs/msg/TransformStamped')}

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

@TomCC7 TomCC7 closed this Oct 20, 2026
@TomCC7
TomCC7 deleted the cc/feat/cdr-fiducial-consumers branch October 20, 2026 18:10

This branch was successfully deployed

1 active deployment
cachix — b3303299 Deployed Oct 20, 2026 by TomCC7 via cachix-build #13389
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@github-actions github-actions[bot] github-actions[bot] left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@TomCC7