Skip to content

refactor(perception): migrate temporal memory and entity rendering to CDR - #4666

Closed
TomCC7 wants to merge 2 commits into
cc/feat/cdr-experimental-objectsfrom
cc/feat/cdr-temporal-memory
Closed

refactor(perception): migrate temporal memory and entity rendering to CDR#4666
TomCC7 wants to merge 2 commits into
cc/feat/cdr-experimental-objectsfrom
cc/feat/cdr-temporal-memory

Conversation

@TomCC7

@TomCC7 TomCC7 commented Oct 20, 2026

Copy link
Copy Markdown
Member

Linear project: Message Encoding Refactor

Temporal memory and PerceiveLoop now consume generated Image/PoseStamped values. Pixel comparison respects row stride and RGB/BGR encoding, and frame accumulation reads the source Header while retaining the existing seconds-based window API.

Temporal memory publishes generated dimos_msgs/EntityMarkers through its existing port. Rendering lives in the Rerun adapter, retaining existing labels/colors/radii and world-frame association; generated values remain plain data. PerceiveLoop's rolling source-time sharpest-frame selection and repeated timer outputs are preserved instead of replacing them with nonoverlapping barrier semantics. The source subscription and timer are now disposed together. Skill/query APIs remain unchanged.

Validation

At b7b2e28e3d22c2af34376f8c7d9b68a1c7db69af:

  • 50 source and 50 installed-main-wheel temporal/PerceiveLoop/Rerun checks pass.
  • Real temporal SQLite persistence and generated entity publication over local CDR/LCM memq execute; actual Rerun components retain points, labels, colors, frame and exact Header nanoseconds.
  • Virtual time verifies the rolling image selection, repeated emissions when no new frame arrives and upstream/timer disposal. Pixel-padding and BGR-vs-RGB regressions execute through actual generated images and CDR decode.
  • VLM/model factories are mocked; persistence tests explicitly disable unrelated CLIP loading. No new weights, API calls, hardware or full-model inference claim.
  • Strict typing passes eight production files; applicable hooks, normal main wheel and default dependency pip check (198 packages) pass.
  • Wheel SHA256: 6c9c9c1534589864e8c9abedcc787243b5f2f172b821468400d570ff4613d21b.

CI has not been polled. Benchmarks remain paused.

Manual example

After installing this branch through the normal main package path, this complete example needs no model or robot:

from dimos_cdrgen.registry import decode, encode
from dimos.messages.builtin_interfaces.msg import Time
from dimos.messages.dimos_msgs.msg import EntityMarker, EntityMarkers
from dimos.messages.geometry_msgs.msg import Point
from dimos.messages.std_msgs.msg import Header
from dimos.visualization.rerun.cdr import generated_archetypes

message = EntityMarkers(
    header=Header(frame_id="world", stamp=Time(sec=0, nanosec=1)),
    markers=[EntityMarker(entity_id="E1", label="cup", entity_type="object",
                          position=Point(x=1, y=2, z=0.3))],
)
received = decode(encode(message), EntityMarkers)
assert received.header == message.header
assert generated_archetypes(received, "memory/entities") is not None

The exact marker/adapter behavior is covered in the tests above. The perception guide describes the runtime boundary. Live blueprint execution still needs the later default encoding cutover.

File change summary

Parent-relative diff from #4665 at 2cbe71ea2b6489932787cbae38516dfb91c42bef: 13 files, 278 additions, 87 deletions.

Category Files Scope
Temporal/PerceiveLoop consumers 7 Generated input types, pixel helpers, rolling selection
Viewer adapter 1 Generated entity markers with unchanged appearance
Tests 3 Persistence, actual CDR, virtual time and exact viewer fields
Docs and CI 2 Boundary explanation and installed-wheel checks

Stack

Depends on #4665. Remaining native depth/camera and downstream mapping/control/robot consumers, default cutover and legacy retirement remain later slices; final chain reconciliation will include the latest additive #4247 main merge. No package publication, permissions change, merge or history rewrite requested.

@TomCC7 TomCC7 mentioned this pull request Oct 20, 2026
Closed
github-actions[bot]
github-actions Bot reviewed Oct 20, 2026
View reviewed changes

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

go2 replay realtime (arm64)

Details
Benchmark suite Current: 6b7c1d1 Previous: fbe126e Ratio
peak memory 1743.75 MB 1564.824 MB 1.11
peak threads 368 threads 357 threads 1.03
network (transport) 2713.722 MB 2727.328 MB 1.00
disk write 1.75 MB 1.488 MB 1.18
instructions 133.093 G 114.929 G 1.16

This comment was automatically generated by workflow using github-action-benchmark.

@codecov

codecov Bot commented Oct 20, 2026

Copy link
Copy Markdown

❌ 4 Tests Failed:

Tests completed Failed Passed Skipped
4 4 0 19
View the top 3 failed test(s) by shortest run time 
::dimos.robot.unitree.go2.blueprints.navigation.test_go2_sim_nav
Stack Traces | 0s run time 
.../blueprints/navigation/test_go2_sim_nav.py:22: in <module>
    from dimos.robot.unitree.go2.blueprints.basic.go2_sim import go2_sim
        Blueprint  = <class 'dimos.core.coordination.blueprints.Blueprint'>
        BlueprintAtom = <class 'dimos.core.coordination.blueprints.BlueprintAtom'>
        ModuleBase = <class 'dimos.core.module.ModuleBase'>
        MovementManager = <class 'dimos.navigation.movement_manager.movement_manager.MovementManager'>
        __builtins__ = <builtins>
        __cached__ = '.../navigation/__pycache__/test_go2_sim_nav.cpython-312.pyc'
        __doc__    = 'The simulator runs the navigation the robot runs, unmodified.'
        __file__   = '/__w/dimos/dimos/.../blueprints/navigation/test_go2_sim_nav.py'
        __loader__ = <_pytest.assertion.rewrite.AssertionRewritingHook object at 0x75615476e4e0>
        __name__   = 'dimos.robot.unitree.go2.blueprints.navigation.test_go2_sim_nav'
        __package__ = 'dimos.robot.unitree.go2.blueprints.navigation'
        __spec__   = ModuleSpec(name='dimos.robot.unitree.go2.blueprints.navigation.test_go2_sim_nav', loader=<_pytest.assertion.rewrite.As...object at 0x75615476e4e0>, origin='/__w/dimos/dimos/.../blueprints/navigation/test_go2_sim_nav.py')
        pytest     = <module 'pytest' from '.../dimos/dimos/.venv/lib/python3.12.../site-packages/pytest/__init__.py'>
.../blueprints/basic/go2_sim.py:28: in <module>
    from dimos.simulation.go2_sim.world import SimGo2World
        Any        = typing.Any
        Go2Mid360StaticTf = <class 'dimos.robot.unitree.go2.go2_mid360_static_transforms.Go2Mid360StaticTf'>
        MovementManager = <class 'dimos.navigation.movement_manager.movement_manager.MovementManager'>
        ROBOT_HEIGHT = 0.45
        ROBOT_LENGTH = 0.6858
        ROBOT_WIDTH = 0.3175
        TYPE_CHECKING = False
        __annotations__ = {}
        __builtins__ = <builtins>
        __cached__ = '.../basic/__pycache__/go2_sim.cpython-312.pyc'
        __doc__    = 'The legged Go2 with its Mid-360 in a generated office, driven from the rerun viewer.'
        __file__   = '/__w/dimos/dimos/.../blueprints/basic/go2_sim.py'
        __loader__ = <_frozen_importlib_external.SourceFileLoader object at 0x755ecf521910>
        __name__   = 'dimos.robot.unitree.go2.blueprints.basic.go2_sim'
        __package__ = 'dimos.robot.unitree.go2.blueprints.basic'
        __spec__   = ModuleSpec(name='dimos.robot.unitree.go2.blueprints.basic.go2_sim', loader=<_frozen_importlib_external.SourceFileLoader object at 0x755ecf521910>, origin='/__w/dimos/dimos/.../blueprints/basic/go2_sim.py')
        annotations = _Feature((3, 7, 0, 'beta', 1), None, 16777216)
        autoconnect = <function autoconnect at 0x75613ee0ccc0>
        body_on_base_link = <function body_on_base_link at 0x755ed825b060>
        global_config = GlobalConfig(robot_ip=None, robot_ips=None, unitree_aes_128_key=None, xarm7_ip=None, xarm6_ip=None, lite6_ip=None, can...load_retries=2, dimos_upload_chunk_mb=None, dimos_upload_quiet_s=30.0, dimos_http_timeout=60.0, dimos_staging_dir=None)
        partial    = <class 'functools.partial'>
.../simulation/go2_sim/world.py:115: in <module>
    MOUNT_XYZ, MOUNT_R = _mount()
        BOX_EDGES  = array([[[         -1,          -1,          -1],
        [          1,          -1,          -1]],

       [[         ... 1,           1]],

       [[          1,           1,          -1],
        [          1,           1,           1]]])
        BOX_RGBA   = {'ceiling': (0.95, 0.95, 0.95, 1.0), 'clutter': (0.45, 0.55, 0.7, 1.0), 'wall': (0.86, 0.84, 0.78, 1.0)}
        CEILING_GROUP = 1
        CMD_VEL_TIMEOUT = 0.2
        COLLISION_GROUP = 3
        CONTROL_DT = 0.02
        Contact    = <class 'dimos.msgs.sim_msgs.Contacts.Contact'>
        Contacts   = <class 'dimos.msgs.sim_msgs.Contacts.Contacts'>
        DEFAULT_THREAD_JOIN_TIMEOUT = 2.0
        Disposable = <class 'reactivex.disposable.disposable.Disposable'>
        Event      = <class 'threading.Event'>
        FLOOR_CHECKER = ((0.62, 0.6, 0.56), (0.7, 0.68, 0.64))
        FRAMES     = [('base_link', None, (0.0, 0.0, 0.0), (0.0, 0.0, 0.0)), ('front_camera', 'base_link', (0.32715, -3e-05, 0.04297), (0.0...511965976, 0.0)), ('camera_optical', 'front_camera', (0.0, 0.0, 0.0), (-1.5707963267948966, 0.0, -1.5707963267948966))]
        FRAME_DT   = 0.1
        Family     = typing.Literal['office']
        Field      = <function Field at 0x756150b58400>
        Go2Policy  = <class 'dimos.simulation.go2_legged.policy.Go2Policy'>
        In         = <class 'dimos.core.stream.In'>
        LIDAR_GROUPS = array([1, 1, 1, 0, 0, 0], dtype=uint8)
        LIDAR_HALF_EXTENTS = (0.0325, 0.0325, 0.03)
        LeggedGo2  = <class 'dimos.simulation.go2_legged.robot.LeggedGo2'>
        LineSegments3D = <class 'dimos.msgs.nav_msgs.LineSegments3D.LineSegments3D'>
        Module     = <class 'dimos.core.module.Module'>
        ModuleConfig = <class 'dimos.core.module.ModuleConfig'>
        MujocoRaycaster = <class 'dimos.simulation.sensors.mujoco_raycaster.MujocoRaycaster'>
        NDArray    = numpy.ndarray[tuple[typing.Any, ...], numpy.dtype[~_ScalarT]]
        ODOM_FRAME_ID = 'odom'
        Odometry   = <class 'dimos.msgs.nav_msgs.Odometry.Odometry'>
        Out        = <class 'dimos.core.stream.Out'>
        POINT_RATE = 200000.0
        Part       = typing.Literal['trunk', 'lidar', 'leg', 'foot']
        Path       = <class 'pathlib.Path'>
        PointCloud2 = <class 'dimos.msgs.sensor_msgs.PointCloud2.PointCloud2'>
        Pose       = <class 'dimos.msgs.geometry_msgs.Pose.Pose'>
        PoseStamped = <class 'dimos.msgs.geometry_msgs.PoseStamped.PoseStamped'>
        Quaternion = <class 'dimos.msgs.geometry_msgs.Quaternion.Quaternion'>
        ROBOT_VISUAL_GROUP = 2
        Rotation   = <class 'scipy.spatial.transform._rotation.Rotation'>
        SCENE_GROUP = 0
        SCENE_PUBLISH_DT = 2.0
        SENSOR_FRAME_ID = 'mid360_link'
        STILL      = array([          0,           0,           0])
        Scene      = <class 'dimos.simulation.scenes.procedural.Scene'>
        SimMid360  = <class 'dimos.simulation.sensors.mid360.lidar.SimMid360'>
        TFMessage  = <class 'dimos.msgs.tf2_msgs.TFMessage.TFMessage'>
        TICKS_PER_FRAME = 5
        Thread     = <class 'threading.Thread'>
        Transform  = <class 'dimos.msgs.geometry_msgs.Transform.Transform'>
        Twist      = <class 'dimos.msgs.geometry_msgs.Twist.Twist'>
        Vector3    = <class 'dimos.msgs.geometry_msgs.Vector3.Vector3'>
        __builtins__ = <builtins>
        __cached__ = '.../go2_sim/__pycache__/world.cpython-312.pyc'
        __doc__    = "The simulated world: a generated scene, the legged Go2 and its Mid-360.\n\nStands in for the Go2 driver and PointLio, publishing PointLio's output contract from\nground truth.\n"
        __file__   = '/__w/dimos/dimos/.../simulation/go2_sim/world.py'
        __loader__ = <_frozen_importlib_external.SourceFileLoader object at 0x755ecf5210a0>
        __name__   = 'dimos.simulation.go2_sim.world'
        __package__ = 'dimos.simulation.go2_sim'
        __spec__   = ModuleSpec(name='dimos.simulation.go2_sim.world', loader=<_frozen_importlib_external.SourceFileLoader object at 0x755ecf5210a0>, origin='/__w/dimos/dimos/.../simulation/go2_sim/world.py')
        _mount     = <function _mount at 0x755ecfb719e0>
        annotations = _Feature((3, 7, 0, 'beta', 1), None, 16777216)
        apply_fitted_physics = <function apply_fitted_physics at 0x755ecfb70fe0>
        dataclass  = <function dataclass at 0x756154ce8040>
        frames_to_edge_transforms = <function frames_to_edge_transforms at 0x755ee403e480>
        generate   = <function generate at 0x755ecfb72980>
        go2_spec   = <function go2_spec at 0x755ecfb70f40>
        load_policy = <function load_policy at 0x755ecfb709a0>
        logger     = <BoundLoggerLazyProxy(logger=<_FixedFindCallerLogger .../simulation/go2_sim/world.py (INFO)>, wrapper_class=<class 'structlog.stdlib.BoundLogger'>, processors=None, context_class=None, initial_values={}, logger_factory_args=())>
        math       = <module 'math' (built-in)>
        mujoco     = <module 'mujoco' from '.../dimos/dimos/.venv/lib/python3.12.../site-packages/mujoco/__init__.py'>
        np         = <module 'numpy' from '.../dimos/dimos/.venv/lib/python3.12.../site-packages/numpy/__init__.py'>
        rpc        = <function rpc at 0x75614d510b80>
        setup_logger = <function setup_logger at 0x7561488ae200>
        time       = <module 'time' (built-in)>
.../simulation/go2_sim/world.py:111: in _mount
    matrix = edges["front_camera"].to_matrix() @ edges["mid360_link"].to_matrix()
E   AttributeError: 'geometry_msgs__msg__TransformStamped' object has no attribute 'to_matrix'
        edges      = {'camera_optical': geometry_msgs__msg__TransformStamped(header=std_msgs__msg__Header(stamp=builtin_interfaces__msg__Ti...y_msgs/msg/Quaternion'), __msgtype__='geometry_msgs/msg/Transform'), __msgtype__='geometry_msgs/msg/TransformStamped')}
::dimos.robot.unitree.go2.blueprints.test_tf_topology
Stack Traces | 0s run time 
.../go2/blueprints/test_tf_topology.py:34: in <module>
    from dimos.robot.unitree.go2.blueprints.navigation.go2_sim_nav import go2_sim_nav
        Blueprint  = <class 'dimos.core.coordination.blueprints.Blueprint'>
        PointLio   = <class 'dimos.hardware.sensors.lidar.pointlio.module.PointLio'>
        PoseStamped = <class 'dimos.msgs.geometry_msgs.PoseStamped.PoseStamped'>
        RelocalizationModule = <class 'dimos.mapping.relocalization.module.RelocalizationModule'>
        __builtins__ = <builtins>
        __cached__ = '.../blueprints/__pycache__/test_tf_topology.cpython-312.pyc'
        __doc__    = 'Every Go2 blueprint publishes a tf tree, not a tf graph.\n\ntf gives each frame one parent. Two publishers writing th...mount tree owns base_link, and the static tree is rooted at\nmid360_link so it never writes the frame PointLio owns.\n'
        __file__   = '/__w/dimos/dimos/.../go2/blueprints/test_tf_topology.py'
        __loader__ = <_pytest.assertion.rewrite.AssertionRewritingHook object at 0x75615476e4e0>
        __name__   = 'dimos.robot.unitree.go2.blueprints.test_tf_topology'
        __package__ = 'dimos.robot.unitree.go2.blueprints'
        __spec__   = ModuleSpec(name='dimos.robot.unitree.go2.blueprints.test_tf_topology', loader=<_pytest.assertion.rewrite.AssertionRewritingHook object at 0x75615476e4e0>, origin='/__w/dimos/dimos/.../go2/blueprints/test_tf_topology.py')
        pytest     = <module 'pytest' from '.../dimos/dimos/.venv/lib/python3.12.../site-packages/pytest/__init__.py'>
        unitree_go2_mid360_record = Blueprint(blueprints=(BlueprintAtom(kwargs={}, module=<class 'dimos.navigation.movement_manager.movement_manager.Movem..., ('keyboardteleop', 'cmd_vel'): 'tele_cmd_vel'}), requirement_checks=(), configurator_checks=(), hosted_placements=())
.../blueprints/navigation/go2_sim_nav.py:24: in <module>
    from dimos.robot.unitree.go2.blueprints.basic.go2_sim import go2_sim, rerun_blueprint, rerun_config
        Any        = typing.Any
        __annotations__ = {}
        __builtins__ = <builtins>
        __cached__ = '.../navigation/__pycache__/go2_sim_nav.cpython-312.pyc'
        __doc__    = 'Go2 navigation driving the simulated legged Go2 to a clicked goal.'
        __file__   = '/__w/dimos/dimos/.../blueprints/navigation/go2_sim_nav.py'
        __loader__ = <_frozen_importlib_external.SourceFileLoader object at 0x755ecf534770>
        __name__   = 'dimos.robot.unitree.go2.blueprints.navigation.go2_sim_nav'
        __package__ = 'dimos.robot.unitree.go2.blueprints.navigation'
        __spec__   = ModuleSpec(name='dimos.robot.unitree.go2.blueprints.navigation.go2_sim_nav', loader=<_frozen_importlib_external.Source...ader object at 0x755ecf534770>, origin='/__w/dimos/dimos/.../blueprints/navigation/go2_sim_nav.py')
        annotations = _Feature((3, 7, 0, 'beta', 1), None, 16777216)
        autoconnect = <function autoconnect at 0x75613ee0ccc0>
        global_config = GlobalConfig(robot_ip=None, robot_ips=None, unitree_aes_128_key=None, xarm7_ip=None, xarm6_ip=None, lite6_ip=None, can...load_retries=2, dimos_upload_chunk_mb=None, dimos_upload_quiet_s=30.0, dimos_http_timeout=60.0, dimos_staging_dir=None)
        partial    = <class 'functools.partial'>
.../blueprints/basic/go2_sim.py:28: in <module>
    from dimos.simulation.go2_sim.world import SimGo2World
        Any        = typing.Any
        Go2Mid360StaticTf = <class 'dimos.robot.unitree.go2.go2_mid360_static_transforms.Go2Mid360StaticTf'>
        MovementManager = <class 'dimos.navigation.movement_manager.movement_manager.MovementManager'>
        ROBOT_HEIGHT = 0.45
        ROBOT_LENGTH = 0.6858
        ROBOT_WIDTH = 0.3175
        TYPE_CHECKING = False
        __annotations__ = {}
        __builtins__ = <builtins>
        __cached__ = '.../basic/__pycache__/go2_sim.cpython-312.pyc'
        __doc__    = 'The legged Go2 with its Mid-360 in a generated office, driven from the rerun viewer.'
        __file__   = '/__w/dimos/dimos/.../blueprints/basic/go2_sim.py'
        __loader__ = <_frozen_importlib_external.SourceFileLoader object at 0x755ecf5358b0>
        __name__   = 'dimos.robot.unitree.go2.blueprints.basic.go2_sim'
        __package__ = 'dimos.robot.unitree.go2.blueprints.basic'
        __spec__   = ModuleSpec(name='dimos.robot.unitree.go2.blueprints.basic.go2_sim', loader=<_frozen_importlib_external.SourceFileLoader object at 0x755ecf5358b0>, origin='/__w/dimos/dimos/.../blueprints/basic/go2_sim.py')
        annotations = _Feature((3, 7, 0, 'beta', 1), None, 16777216)
        autoconnect = <function autoconnect at 0x75613ee0ccc0>
        body_on_base_link = <function body_on_base_link at 0x755ed825b060>
        global_config = GlobalConfig(robot_ip=None, robot_ips=None, unitree_aes_128_key=None, xarm7_ip=None, xarm6_ip=None, lite6_ip=None, can...load_retries=2, dimos_upload_chunk_mb=None, dimos_upload_quiet_s=30.0, dimos_http_timeout=60.0, dimos_staging_dir=None)
        partial    = <class 'functools.partial'>
.../simulation/go2_sim/world.py:115: in <module>
    MOUNT_XYZ, MOUNT_R = _mount()
        BOX_EDGES  = array([[[         -1,          -1,          -1],
        [          1,          -1,          -1]],

       [[         ... 1,           1]],

       [[          1,           1,          -1],
        [          1,           1,           1]]])
        BOX_RGBA   = {'ceiling': (0.95, 0.95, 0.95, 1.0), 'clutter': (0.45, 0.55, 0.7, 1.0), 'wall': (0.86, 0.84, 0.78, 1.0)}
        CEILING_GROUP = 1
        CMD_VEL_TIMEOUT = 0.2
        COLLISION_GROUP = 3
        CONTROL_DT = 0.02
        Contact    = <class 'dimos.msgs.sim_msgs.Contacts.Contact'>
        Contacts   = <class 'dimos.msgs.sim_msgs.Contacts.Contacts'>
        DEFAULT_THREAD_JOIN_TIMEOUT = 2.0
        Disposable = <class 'reactivex.disposable.disposable.Disposable'>
        Event      = <class 'threading.Event'>
        FLOOR_CHECKER = ((0.62, 0.6, 0.56), (0.7, 0.68, 0.64))
        FRAMES     = [('base_link', None, (0.0, 0.0, 0.0), (0.0, 0.0, 0.0)), ('front_camera', 'base_link', (0.32715, -3e-05, 0.04297), (0.0...511965976, 0.0)), ('camera_optical', 'front_camera', (0.0, 0.0, 0.0), (-1.5707963267948966, 0.0, -1.5707963267948966))]
        FRAME_DT   = 0.1
        Family     = typing.Literal['office']
        Field      = <function Field at 0x756150b58400>
        Go2Policy  = <class 'dimos.simulation.go2_legged.policy.Go2Policy'>
        In         = <class 'dimos.core.stream.In'>
        LIDAR_GROUPS = array([1, 1, 1, 0, 0, 0], dtype=uint8)
        LIDAR_HALF_EXTENTS = (0.0325, 0.0325, 0.03)
        LeggedGo2  = <class 'dimos.simulation.go2_legged.robot.LeggedGo2'>
        LineSegments3D = <class 'dimos.msgs.nav_msgs.LineSegments3D.LineSegments3D'>
        Module     = <class 'dimos.core.module.Module'>
        ModuleConfig = <class 'dimos.core.module.ModuleConfig'>
        MujocoRaycaster = <class 'dimos.simulation.sensors.mujoco_raycaster.MujocoRaycaster'>
        NDArray    = numpy.ndarray[tuple[typing.Any, ...], numpy.dtype[~_ScalarT]]
        ODOM_FRAME_ID = 'odom'
        Odometry   = <class 'dimos.msgs.nav_msgs.Odometry.Odometry'>
        Out        = <class 'dimos.core.stream.Out'>
        POINT_RATE = 200000.0
        Part       = typing.Literal['trunk', 'lidar', 'leg', 'foot']
        Path       = <class 'pathlib.Path'>
        PointCloud2 = <class 'dimos.msgs.sensor_msgs.PointCloud2.PointCloud2'>
        Pose       = <class 'dimos.msgs.geometry_msgs.Pose.Pose'>
        PoseStamped = <class 'dimos.msgs.geometry_msgs.PoseStamped.PoseStamped'>
        Quaternion = <class 'dimos.msgs.geometry_msgs.Quaternion.Quaternion'>
        ROBOT_VISUAL_GROUP = 2
        Rotation   = <class 'scipy.spatial.transform._rotation.Rotation'>
        SCENE_GROUP = 0
        SCENE_PUBLISH_DT = 2.0
        SENSOR_FRAME_ID = 'mid360_link'
        STILL      = array([          0,           0,           0])
        Scene      = <class 'dimos.simulation.scenes.procedural.Scene'>
        SimMid360  = <class 'dimos.simulation.sensors.mid360.lidar.SimMid360'>
        TFMessage  = <class 'dimos.msgs.tf2_msgs.TFMessage.TFMessage'>
        TICKS_PER_FRAME = 5
        Thread     = <class 'threading.Thread'>
        Transform  = <class 'dimos.msgs.geometry_msgs.Transform.Transform'>
        Twist      = <class 'dimos.msgs.geometry_msgs.Twist.Twist'>
        Vector3    = <class 'dimos.msgs.geometry_msgs.Vector3.Vector3'>
        __builtins__ = <builtins>
        __cached__ = '.../go2_sim/__pycache__/world.cpython-312.pyc'
        __doc__    = "The simulated world: a generated scene, the legged Go2 and its Mid-360.\n\nStands in for the Go2 driver and PointLio, publishing PointLio's output contract from\nground truth.\n"
        __file__   = '/__w/dimos/dimos/.../simulation/go2_sim/world.py'
        __loader__ = <_frozen_importlib_external.SourceFileLoader object at 0x755ecf5358e0>
        __name__   = 'dimos.simulation.go2_sim.world'
        __package__ = 'dimos.simulation.go2_sim'
        __spec__   = ModuleSpec(name='dimos.simulation.go2_sim.world', loader=<_frozen_importlib_external.SourceFileLoader object at 0x755ecf5358e0>, origin='/__w/dimos/dimos/.../simulation/go2_sim/world.py')
        _mount     = <function _mount at 0x755ecf36cf40>
        annotations = _Feature((3, 7, 0, 'beta', 1), None, 16777216)
        apply_fitted_physics = <function apply_fitted_physics at 0x755ecfb70fe0>
        dataclass  = <function dataclass at 0x756154ce8040>
        frames_to_edge_transforms = <function frames_to_edge_transforms at 0x755ee403e480>
        generate   = <function generate at 0x755ecfb72980>
        go2_spec   = <function go2_spec at 0x755ecfb70f40>
        load_policy = <function load_policy at 0x755ecfb709a0>
        logger     = <BoundLoggerLazyProxy(logger=<_FixedFindCallerLogger .../simulation/go2_sim/world.py (INFO)>, wrapper_class=<class 'structlog.stdlib.BoundLogger'>, processors=None, context_class=None, initial_values={}, logger_factory_args=())>
        math       = <module 'math' (built-in)>
        mujoco     = <module 'mujoco' from '.../dimos/dimos/.venv/lib/python3.12.../site-packages/mujoco/__init__.py'>
        np         = <module 'numpy' from '.../dimos/dimos/.venv/lib/python3.12.../site-packages/numpy/__init__.py'>
        rpc        = <function rpc at 0x75614d510b80>
        setup_logger = <function setup_logger at 0x7561488ae200>
        time       = <module 'time' (built-in)>
.../simulation/go2_sim/world.py:111: in _mount
    matrix = edges["front_camera"].to_matrix() @ edges["mid360_link"].to_matrix()
E   AttributeError: 'geometry_msgs__msg__TransformStamped' object has no attribute 'to_matrix'
        edges      = {'camera_optical': geometry_msgs__msg__TransformStamped(header=std_msgs__msg__Header(stamp=builtin_interfaces__msg__Ti...y_msgs/msg/Quaternion'), __msgtype__='geometry_msgs/msg/Transform'), __msgtype__='geometry_msgs/msg/TransformStamped')}
::dimos.simulation.go2_sim.test_command_hold
Stack Traces | 0s run time 
.../simulation/go2_sim/test_command_hold.py:19: in <module>
    from dimos.simulation.go2_sim.world import STILL, CommandHold
        __builtins__ = <builtins>
        __cached__ = '.../go2_sim/__pycache__/test_command_hold.cpython-312.pyc'
        __doc__    = None
        __file__   = '/__w/dimos/dimos/.../simulation/go2_sim/test_command_hold.py'
        __loader__ = <_pytest.assertion.rewrite.AssertionRewritingHook object at 0x75615476e4e0>
        __name__   = 'dimos.simulation.go2_sim.test_command_hold'
        __package__ = 'dimos.simulation.go2_sim'
        __spec__   = ModuleSpec(name='dimos.simulation.go2_sim.test_command_hold', loader=<_pytest.assertion.rewrite.AssertionRewritingHook object at 0x75615476e4e0>, origin='/__w/dimos/dimos/.../simulation/go2_sim/test_command_hold.py')
        annotations = _Feature((3, 7, 0, 'beta', 1), None, 16777216)
        np         = <module 'numpy' from '.../dimos/dimos/.venv/lib/python3.12....../site-packages/numpy/__init__.py'>
.../simulation/go2_sim/world.py:115: in <module>
    MOUNT_XYZ, MOUNT_R = _mount()
        BOX_EDGES  = array([[[         -1,          -1,          -1],
        [          1,          -1,          -1]],

       [[         ... 1,           1]],

       [[          1,           1,          -1],
        [          1,           1,           1]]])
        BOX_RGBA   = {'ceiling': (0.95, 0.95, 0.95, 1.0), 'clutter': (0.45, 0.55, 0.7, 1.0), 'wall': (0.86, 0.84, 0.78, 1.0)}
        CEILING_GROUP = 1
        CMD_VEL_TIMEOUT = 0.2
        COLLISION_GROUP = 3
        CONTROL_DT = 0.02
        Contact    = <class 'dimos.msgs.sim_msgs.Contacts.Contact'>
        Contacts   = <class 'dimos.msgs.sim_msgs.Contacts.Contacts'>
        DEFAULT_THREAD_JOIN_TIMEOUT = 2.0
        Disposable = <class 'reactivex.disposable.disposable.Disposable'>
        Event      = <class 'threading.Event'>
        FLOOR_CHECKER = ((0.62, 0.6, 0.56), (0.7, 0.68, 0.64))
        FRAMES     = [('base_link', None, (0.0, 0.0, 0.0), (0.0, 0.0, 0.0)), ('front_camera', 'base_link', (0.32715, -3e-05, 0.04297), (0.0...511965976, 0.0)), ('camera_optical', 'front_camera', (0.0, 0.0, 0.0), (-1.5707963267948966, 0.0, -1.5707963267948966))]
        FRAME_DT   = 0.1
        Family     = typing.Literal['office']
        Field      = <function Field at 0x756150b58400>
        Go2Policy  = <class 'dimos.simulation.go2_legged.policy.Go2Policy'>
        In         = <class 'dimos.core.stream.In'>
        LIDAR_GROUPS = array([1, 1, 1, 0, 0, 0], dtype=uint8)
        LIDAR_HALF_EXTENTS = (0.0325, 0.0325, 0.03)
        LeggedGo2  = <class 'dimos.simulation.go2_legged.robot.LeggedGo2'>
        LineSegments3D = <class 'dimos.msgs.nav_msgs.LineSegments3D.LineSegments3D'>
        Module     = <class 'dimos.core.module.Module'>
        ModuleConfig = <class 'dimos.core.module.ModuleConfig'>
        MujocoRaycaster = <class 'dimos.simulation.sensors.mujoco_raycaster.MujocoRaycaster'>
        NDArray    = numpy.ndarray[tuple[typing.Any, ...], numpy.dtype[~_ScalarT]]
        ODOM_FRAME_ID = 'odom'
        Odometry   = <class 'dimos.msgs.nav_msgs.Odometry.Odometry'>
        Out        = <class 'dimos.core.stream.Out'>
        POINT_RATE = 200000.0
        Part       = typing.Literal['trunk', 'lidar', 'leg', 'foot']
        Path       = <class 'pathlib.Path'>
        PointCloud2 = <class 'dimos.msgs.sensor_msgs.PointCloud2.PointCloud2'>
        Pose       = <class 'dimos.msgs.geometry_msgs.Pose.Pose'>
        PoseStamped = <class 'dimos.msgs.geometry_msgs.PoseStamped.PoseStamped'>
        Quaternion = <class 'dimos.msgs.geometry_msgs.Quaternion.Quaternion'>
        ROBOT_VISUAL_GROUP = 2
        Rotation   = <class 'scipy.spatial.transform._rotation.Rotation'>
        SCENE_GROUP = 0
        SCENE_PUBLISH_DT = 2.0
        SENSOR_FRAME_ID = 'mid360_link'
        STILL      = array([          0,           0,           0])
        Scene      = <class 'dimos.simulation.scenes.procedural.Scene'>
        SimMid360  = <class 'dimos.simulation.sensors.mid360.lidar.SimMid360'>
        TFMessage  = <class 'dimos.msgs.tf2_msgs.TFMessage.TFMessage'>
        TICKS_PER_FRAME = 5
        Thread     = <class 'threading.Thread'>
        Transform  = <class 'dimos.msgs.geometry_msgs.Transform.Transform'>
        Twist      = <class 'dimos.msgs.geometry_msgs.Twist.Twist'>
        Vector3    = <class 'dimos.msgs.geometry_msgs.Vector3.Vector3'>
        __builtins__ = <builtins>
        __cached__ = '.../go2_sim/__pycache__/world.cpython-312.pyc'
        __doc__    = "The simulated world: a generated scene, the legged Go2 and its Mid-360.\n\nStands in for the Go2 driver and PointLio, publishing PointLio's output contract from\nground truth.\n"
        __file__   = '/__w/dimos/dimos/.../simulation/go2_sim/world.py'
        __loader__ = <_frozen_importlib_external.SourceFileLoader object at 0x755ecf38d940>
        __name__   = 'dimos.simulation.go2_sim.world'
        __package__ = 'dimos.simulation.go2_sim'
        __spec__   = ModuleSpec(name='dimos.simulation.go2_sim.world', loader=<_frozen_importlib_external.SourceFileLoader object at 0x755ecf38d940>, origin='/__w/dimos/dimos/.../simulation/go2_sim/world.py')
        _mount     = <function _mount at 0x755ecf6cad40>
        annotations = _Feature((3, 7, 0, 'beta', 1), None, 16777216)
        apply_fitted_physics = <function apply_fitted_physics at 0x755ecfb70fe0>
        dataclass  = <function dataclass at 0x756154ce8040>
        frames_to_edge_transforms = <function frames_to_edge_transforms at 0x755ee403e480>
        generate   = <function generate at 0x755ecfb72980>
        go2_spec   = <function go2_spec at 0x755ecfb70f40>
        load_policy = <function load_policy at 0x755ecfb709a0>
        logger     = <BoundLoggerLazyProxy(logger=<_FixedFindCallerLogger .../simulation/go2_sim/world.py (INFO)>, wrapper_class=<class 'structlog.stdlib.BoundLogger'>, processors=None, context_class=None, initial_values={}, logger_factory_args=())>
        math       = <module 'math' (built-in)>
        mujoco     = <module 'mujoco' from '.../dimos/dimos/.venv/lib/python3.12.../site-packages/mujoco/__init__.py'>
        np         = <module 'numpy' from '.../dimos/dimos/.venv/lib/python3.12....../site-packages/numpy/__init__.py'>
        rpc        = <function rpc at 0x75614d510b80>
        setup_logger = <function setup_logger at 0x7561488ae200>
        time       = <module 'time' (built-in)>
.../simulation/go2_sim/world.py:111: in _mount
    matrix = edges["front_camera"].to_matrix() @ edges["mid360_link"].to_matrix()
E   AttributeError: 'geometry_msgs__msg__TransformStamped' object has no attribute 'to_matrix'
        edges      = {'camera_optical': geometry_msgs__msg__TransformStamped(header=std_msgs__msg__Header(stamp=builtin_interfaces__msg__Ti...y_msgs/msg/Quaternion'), __msgtype__='geometry_msgs/msg/Transform'), __msgtype__='geometry_msgs/msg/TransformStamped')}
::dimos.simulation.go2_sim.test_world
Stack Traces | 0s run time 
.../simulation/go2_sim/test_world.py:25: in <module>
    from dimos.simulation.go2_sim.world import (
        CONTROL_DT = 0.02
        Contact    = <class 'dimos.msgs.sim_msgs.Contacts.Contact'>
        NDArray    = numpy.ndarray[tuple[typing.Any, ...], numpy.dtype[~_ScalarT]]
        OnnxGo2Policy = <class 'dimos.simulation.go2_legged.policy.OnnxGo2Policy'>
        __builtins__ = <builtins>
        __cached__ = '.../go2_sim/__pycache__/test_world.cpython-312.pyc'
        __doc__    = None
        __file__   = '/__w/dimos/dimos/.../simulation/go2_sim/test_world.py'
        __loader__ = <_pytest.assertion.rewrite.AssertionRewritingHook object at 0x75615476e4e0>
        __name__   = 'dimos.simulation.go2_sim.test_world'
        __package__ = 'dimos.simulation.go2_sim'
        __spec__   = ModuleSpec(name='dimos.simulation.go2_sim.test_world', loader=<_pytest.assertion.rewrite.AssertionRewritingHook object at 0x75615476e4e0>, origin='/__w/dimos/dimos/.../simulation/go2_sim/test_world.py')
        annotations = _Feature((3, 7, 0, 'beta', 1), None, 16777216)
        mujoco     = <module 'mujoco' from '.../dimos/dimos/.venv/lib/python3.12....../site-packages/mujoco/__init__.py'>
        np         = <module 'numpy' from '.../dimos/dimos/.venv/lib/python3.12....../site-packages/numpy/__init__.py'>
        pytest     = <module 'pytest' from '.../dimos/dimos/.venv/lib/python3.12.../site-packages/pytest/__init__.py'>
.../simulation/go2_sim/world.py:115: in <module>
    MOUNT_XYZ, MOUNT_R = _mount()
        BOX_EDGES  = array([[[         -1,          -1,          -1],
        [          1,          -1,          -1]],

       [[         ... 1,           1]],

       [[          1,           1,          -1],
        [          1,           1,           1]]])
        BOX_RGBA   = {'ceiling': (0.95, 0.95, 0.95, 1.0), 'clutter': (0.45, 0.55, 0.7, 1.0), 'wall': (0.86, 0.84, 0.78, 1.0)}
        CEILING_GROUP = 1
        CMD_VEL_TIMEOUT = 0.2
        COLLISION_GROUP = 3
        CONTROL_DT = 0.02
        Contact    = <class 'dimos.msgs.sim_msgs.Contacts.Contact'>
        Contacts   = <class 'dimos.msgs.sim_msgs.Contacts.Contacts'>
        DEFAULT_THREAD_JOIN_TIMEOUT = 2.0
        Disposable = <class 'reactivex.disposable.disposable.Disposable'>
        Event      = <class 'threading.Event'>
        FLOOR_CHECKER = ((0.62, 0.6, 0.56), (0.7, 0.68, 0.64))
        FRAMES     = [('base_link', None, (0.0, 0.0, 0.0), (0.0, 0.0, 0.0)), ('front_camera', 'base_link', (0.32715, -3e-05, 0.04297), (0.0...511965976, 0.0)), ('camera_optical', 'front_camera', (0.0, 0.0, 0.0), (-1.5707963267948966, 0.0, -1.5707963267948966))]
        FRAME_DT   = 0.1
        Family     = typing.Literal['office']
        Field      = <function Field at 0x756150b58400>
        Go2Policy  = <class 'dimos.simulation.go2_legged.policy.Go2Policy'>
        In         = <class 'dimos.core.stream.In'>
        LIDAR_GROUPS = array([1, 1, 1, 0, 0, 0], dtype=uint8)
        LIDAR_HALF_EXTENTS = (0.0325, 0.0325, 0.03)
        LeggedGo2  = <class 'dimos.simulation.go2_legged.robot.LeggedGo2'>
        LineSegments3D = <class 'dimos.msgs.nav_msgs.LineSegments3D.LineSegments3D'>
        Module     = <class 'dimos.core.module.Module'>
        ModuleConfig = <class 'dimos.core.module.ModuleConfig'>
        MujocoRaycaster = <class 'dimos.simulation.sensors.mujoco_raycaster.MujocoRaycaster'>
        NDArray    = numpy.ndarray[tuple[typing.Any, ...], numpy.dtype[~_ScalarT]]
        ODOM_FRAME_ID = 'odom'
        Odometry   = <class 'dimos.msgs.nav_msgs.Odometry.Odometry'>
        Out        = <class 'dimos.core.stream.Out'>
        POINT_RATE = 200000.0
        Part       = typing.Literal['trunk', 'lidar', 'leg', 'foot']
        Path       = <class 'pathlib.Path'>
        PointCloud2 = <class 'dimos.msgs.sensor_msgs.PointCloud2.PointCloud2'>
        Pose       = <class 'dimos.msgs.geometry_msgs.Pose.Pose'>
        PoseStamped = <class 'dimos.msgs.geometry_msgs.PoseStamped.PoseStamped'>
        Quaternion = <class 'dimos.msgs.geometry_msgs.Quaternion.Quaternion'>
        ROBOT_VISUAL_GROUP = 2
        Rotation   = <class 'scipy.spatial.transform._rotation.Rotation'>
        SCENE_GROUP = 0
        SCENE_PUBLISH_DT = 2.0
        SENSOR_FRAME_ID = 'mid360_link'
        STILL      = array([          0,           0,           0])
        Scene      = <class 'dimos.simulation.scenes.procedural.Scene'>
        SimMid360  = <class 'dimos.simulation.sensors.mid360.lidar.SimMid360'>
        TFMessage  = <class 'dimos.msgs.tf2_msgs.TFMessage.TFMessage'>
        TICKS_PER_FRAME = 5
        Thread     = <class 'threading.Thread'>
        Transform  = <class 'dimos.msgs.geometry_msgs.Transform.Transform'>
        Twist      = <class 'dimos.msgs.geometry_msgs.Twist.Twist'>
        Vector3    = <class 'dimos.msgs.geometry_msgs.Vector3.Vector3'>
        __builtins__ = <builtins>
        __cached__ = '.../go2_sim/__pycache__/world.cpython-312.pyc'
        __doc__    = "The simulated world: a generated scene, the legged Go2 and its Mid-360.\n\nStands in for the Go2 driver and PointLio, publishing PointLio's output contract from\nground truth.\n"
        __file__   = '/__w/dimos/dimos/.../simulation/go2_sim/world.py'
        __loader__ = <_frozen_importlib_external.SourceFileLoader object at 0x755ecf38fe00>
        __name__   = 'dimos.simulation.go2_sim.world'
        __package__ = 'dimos.simulation.go2_sim'
        __spec__   = ModuleSpec(name='dimos.simulation.go2_sim.world', loader=<_frozen_importlib_external.SourceFileLoader object at 0x755ecf38fe00>, origin='/__w/dimos/dimos/.../simulation/go2_sim/world.py')
        _mount     = <function _mount at 0x755ecf6caca0>
        annotations = _Feature((3, 7, 0, 'beta', 1), None, 16777216)
        apply_fitted_physics = <function apply_fitted_physics at 0x755ecfb70fe0>
        dataclass  = <function dataclass at 0x756154ce8040>
        frames_to_edge_transforms = <function frames_to_edge_transforms at 0x755ee403e480>
        generate   = <function generate at 0x755ecfb72980>
        go2_spec   = <function go2_spec at 0x755ecfb70f40>
        load_policy = <function load_policy at 0x755ecfb709a0>
        logger     = <BoundLoggerLazyProxy(logger=<_FixedFindCallerLogger .../simulation/go2_sim/world.py (INFO)>, wrapper_class=<class 'structlog.stdlib.BoundLogger'>, processors=None, context_class=None, initial_values={}, logger_factory_args=())>
        math       = <module 'math' (built-in)>
        mujoco     = <module 'mujoco' from '.../dimos/dimos/.venv/lib/python3.12....../site-packages/mujoco/__init__.py'>
        np         = <module 'numpy' from '.../dimos/dimos/.venv/lib/python3.12....../site-packages/numpy/__init__.py'>
        rpc        = <function rpc at 0x75614d510b80>
        setup_logger = <function setup_logger at 0x7561488ae200>
        time       = <module 'time' (built-in)>
.../simulation/go2_sim/world.py:111: in _mount
    matrix = edges["front_camera"].to_matrix() @ edges["mid360_link"].to_matrix()
E   AttributeError: 'geometry_msgs__msg__TransformStamped' object has no attribute 'to_matrix'
        edges      = {'camera_optical': geometry_msgs__msg__TransformStamped(header=std_msgs__msg__Header(stamp=builtin_interfaces__msg__Ti...y_msgs/msg/Quaternion'), __msgtype__='geometry_msgs/msg/Transform'), __msgtype__='geometry_msgs/msg/TransformStamped')}

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

@TomCC7 TomCC7 closed this Oct 20, 2026
@TomCC7
TomCC7 deleted the cc/feat/cdr-temporal-memory branch October 20, 2026 18:10

This branch was successfully deployed

1 active deployment
cachix — 6b7c1d17 Deployed Oct 20, 2026 by autofix-ci[bot] via cachix-build #13397
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@github-actions github-actions[bot] github-actions[bot] left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@TomCC7