[TomCC7]

The actual three-language blueprint is recorded to self-describing MCAP, producers are stopped, then independent decoding and DimOS replay verify every payload in per-stream order. A separately generated external message package is recorded by the unchanged native recorder and decoded from its embedded schema. This layer contains substantive integration demos, viewer-decoder checks and cumulative README/evidence; required production storage/schema code is already in the runtime layer.

Base: #4363. This is layer 6/7, replacing the combined draft #4257 through the user-approved split. Original PR1–3 and the old feat/cdr-runtime-cutover branch/history are preserved. Reassembly uses selected files/hunks and forward merges, not a claim that the original commits cleanly cherry-pick.

Published HEAD: 180109d0648f83c2c6c58d92dad20213ef7d1495.

Validation: Candidate-built native recorder/relays: 16 Rust tests passed; 3 SQLite/MCAP/TF interoperability cases passed. Nine blueprint rows independently decoded and byte-exact replayed after producers stopped. Three external Telemetry rows recorded over Zenoh and decoded with pinned Foxglove libraries without viewer-side custom package. Unknown external native stamp layouts use reception time; payload Header remains preserved. Hooks passed. Full LCM/manual viewer acceptance and GitHub CI remain pending.

This remains a proposal with a deliberate old-message API/wire break. CDR replaces encoding; LCM/Zenoh remain transports. No old-recording fallback is promised. No merge or package registry release is requested. External gates remain M20 SDK, host multicast, Foxglove sign-in and human/full-pose visual acceptance. Benchmarks have not been waived or weakened.

Published replacement chain: #4247 → #4252 → #4254 → #4362 → #4363 → #4364 → #4365. Preserved full reference: #4257 at cb982817bde248baa70a6fa69cc9ecc275c43707.

Tutorial follow-up: the checkout workflow is documented in docs/development/messages-in-repository.md; the installed-runtime/custom-project workflow is in docs/development/messages-external-project.md (runtime layer onward); both share message-reference.md. Matching review wheels are required; pip does not resolve uv-only local sources. Clean isolated canonical .msg addition→uv build→wheel install→value/codec checks passed in 112 seconds without installing the DimOS runtime. Existing isolated external source/wheel/sdist/editable acceptance and exact three-language coordinator evidence are retained; embedded C++ includes are now checked correctly. Required toolchains, SDK artifacts and harness acquisition are explicit.

Initial authoring CI exposed a premature root Rust workspace join (lock/Nix source mismatch) and a cold docs build missing Fast CDR. The generated crate remains standalone in authoring and joins the root workspace at runtime cutover; the existing Fast CDR setup now precedes package installation. Locks/assertions/gates were not relaxed. New exact-HEAD CI remains pending.

Authoring namespace test runner now invokes python -m pytest so a clean checkout imports the lightweight message namespace; 11 focused project tests passed. Self-hosted setup now explicitly installs CMake before building Fast CDR, and rosbags==0.11.0 and mcap>=1.2.0 are declared in the shared test group with its locked dependency; eight focused ownership/MCAP cases and hooks passed. Exact updated-head checks remain pending.

Superseded by naming-only replacement #4370. Exact commit 180109d0648f83c2c6c58d92dad20213ef7d1495, source tree and parent-relative diff are unchanged.

Verified native seven-layer stack #4372: #4247 → #4366 → #4367 → #4368 → #4369 → #4370 → #4371. Stack: #4247

All branches use cc/ and the ordered PR bases preserve the prior commit ancestry. Old branches/history are retained. Replacement PR checks are pending and prior CI retains its exact SHA/run/ref provenance. No merge, force push or release.

[codecov]

❌ 4 Tests Failed:

Tests completed	Failed	Passed	Skipped
6365	4	6361	96

View the top 3 failed test(s) by shortest run time

::dimos.navigation.nav_3d.mls_planner.utils.test_plan_rrd

Stack Traces | 0s run time

ImportError while importing test module '.../mls_planner/utils/test_plan_rrd.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
.../mapping/ray_tracing/voxel_map.py:20: in <module>
    from dimos_voxel_ray_tracing import VoxelRayMapper  # noqa: F401  (re-exported)
E   ModuleNotFoundError: No module named 'dimos_voxel_ray_tracing'

The above exception was the direct cause of the following exception:
.../mls_planner/utils/test_plan_rrd.py:24: in <module>
    from dimos.navigation.nav_3d.mls_planner.utils.plan_rrd import _log_odometry, _TfSync
.../mls_planner/utils/plan_rrd.py:34: in <module>
    from dimos.mapping.ray_tracing.transformer import RayTraceMap
.../mapping/ray_tracing/transformer.py:23: in <module>
    from dimos.mapping.ray_tracing.voxel_map import VoxelRayMapper
.../mapping/ray_tracing/voxel_map.py:22: in <module>
    raise ImportError(
E   ImportError: dimos_voxel_ray_tracing is not built. Run: uv run maturin develop --release --uv -m .../rust/py/Cargo.toml

dimos.mapping.dim_slam.test_dim_odom_replay::test_dim_odom_fusion_replay

Stack Traces | 0.02s run time

def test_dim_odom_fusion_replay():
        """IMU + wheel odometry through OdometryFusion, twice, identically."""
        db_path = str(get_data(FUSION_SNIPPET))
>       first = _replay_fusion(db_path)

db_path    = '.../dimos/data/alfred_fusion_short.db'

.../mapping/dim_slam/test_dim_odom_replay.py:213: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.../mapping/dim_slam/test_dim_odom_replay.py:147: in _replay_fusion
    base_from_imu = _tf_lookup(store)("base_link", IMU_FRAME)
        db_path    = '.../dimos/data/alfred_fusion_short.db'
        store      = <dimos.memory.store.sqlite.SqliteStore object at 0x739493c3dc40>
.../mapping/dim_slam/test_dim_odom_replay.py:73: in _tf_lookup
    tf = StreamTF.from_store(store)
        store      = <dimos.memory.store.sqlite.SqliteStore object at 0x739493c3dc40>
dimos/memory/tf.py:60: in from_store
    recorded = tf_stream(store, stream)
        cls        = <class 'dimos.memory.tf.StreamTF'>
        store      = <dimos.memory.store.sqlite.SqliteStore object at 0x739493c3dc40>
        stream     = 'tf'
dimos/memory/tf.py:38: in tf_stream
    return cast("Stream[TFMessage]", store.stream(stream, TFMessage))
        store      = <dimos.memory.store.sqlite.SqliteStore object at 0x739493c3dc40>
        stream     = 'tf'
.../memory/store/base.py:197: in stream
    backend = self._create_backend(name, payload_type, **resolved)
        name       = 'tf'
        overrides  = {}
        payload_type = <class 'dimos_generated.tf2_msgs.msg.TFMessage'>
        resolved   = {'eager_blobs': False, 'must_exist': True, 'page_size': 256, 'path': '.../dimos/data/alfred_fusion_short.db'}
        self       = <dimos.memory.store.sqlite.SqliteStore object at 0x739493c3dc40>
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <dimos.memory.store.sqlite.SqliteStore object at 0x739493c3dc40>
name = 'tf', payload_type = <class 'dimos_generated.tf2_msgs.msg.TFMessage'>
config = {'eager_blobs': False, 'must_exist': True, 'page_size': 256, 'path': '.../dimos/data/alfred_fusion_short.db'}
stored = {'blob_store': {'class': 'dimos.memory.blobstore.sqlite.SqliteBlobStore', 'config': {'path': None}}, 'codec_id': 'lcm', 'eager_blobs': False, 'notifier': {'class': 'dimos.memory.notifier.subject.SubjectNotifier', 'config': {}}, ...}
actual_module = 'dimos_generated.tf2_msgs.msg.TFMessage'

    def _create_backend(
        self, name: str, payload_type: type[Any] | None = None, **config: Any
    ) -> Backend[Any]:
        validate_identifier(name)
    
        stored = self._registry.get(name)
    
        if stored is not None:
            # Load path: validate type, assemble from stored config
            if payload_type is not None:
                actual_module = f"{payload_type.__module__}.{payload_type.__qualname__}"
                if actual_module != stored["payload_module"]:
>                   raise ValueError(
                        f"Stream {name!r} was created with type {stored['payload_module']}, "
                        f"but opened with {actual_module}"
                    )
E                   ValueError: Stream 'tf' was created with type dimos.msgs.tf2_msgs.TFMessage.TFMessage, but opened with dimos_generated.tf2_msgs.msg.TFMessage

__class__  = <class 'dimos.memory.store.sqlite.SqliteStore'>
actual_module = 'dimos_generated.tf2_msgs.msg.TFMessage'
config     = {'eager_blobs': False, 'must_exist': True, 'page_size': 256, 'path': '.../dimos/data/alfred_fusion_short.db'}
name       = 'tf'
payload_type = <class 'dimos_generated.tf2_msgs.msg.TFMessage'>
self       = <dimos.memory.store.sqlite.SqliteStore object at 0x739493c3dc40>
stored     = {'blob_store': {'class': 'dimos.memory.blobstore.sqlite.SqliteBlobStore', 'config': {'path': None}}, 'codec_id': 'lcm', 'eager_blobs': False, 'notifier': {'class': 'dimos.memory.notifier.subject.SubjectNotifier', 'config': {}}, ...}

.../memory/store/sqlite.py:161: ValueError

dimos.mapping.dim_slam.test_dim_odom_replay::test_dim_odom_stereo_replay

Stack Traces | 1.35s run time

def test_dim_odom_stereo_replay():
        """The same recording twice through dim_odom gives the same trajectory."""
        db_path = str(get_data(SNIPPET))
>       first = _replay_trajectory(db_path)

db_path    = '.../dimos/data/alfred_stereo_short.db'

.../mapping/dim_slam/test_dim_odom_replay.py:198: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.../mapping/dim_slam/test_dim_odom_replay.py:110: in _replay_trajectory
    tf=_tf_lookup(store),
        db_path    = '.../dimos/data/alfred_stereo_short.db'
        store      = <dimos.memory.store.sqlite.SqliteStore object at 0x739499897ef0>
.../mapping/dim_slam/test_dim_odom_replay.py:73: in _tf_lookup
    tf = StreamTF.from_store(store)
        store      = <dimos.memory.store.sqlite.SqliteStore object at 0x739499897ef0>
dimos/memory/tf.py:60: in from_store
    recorded = tf_stream(store, stream)
        cls        = <class 'dimos.memory.tf.StreamTF'>
        store      = <dimos.memory.store.sqlite.SqliteStore object at 0x739499897ef0>
        stream     = 'tf'
dimos/memory/tf.py:38: in tf_stream
    return cast("Stream[TFMessage]", store.stream(stream, TFMessage))
        store      = <dimos.memory.store.sqlite.SqliteStore object at 0x739499897ef0>
        stream     = 'tf'
.../memory/store/base.py:197: in stream
    backend = self._create_backend(name, payload_type, **resolved)
        name       = 'tf'
        overrides  = {}
        payload_type = <class 'dimos_generated.tf2_msgs.msg.TFMessage'>
        resolved   = {'eager_blobs': False, 'must_exist': True, 'page_size': 256, 'path': '.../dimos/data/alfred_stereo_short.db'}
        self       = <dimos.memory.store.sqlite.SqliteStore object at 0x739499897ef0>
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <dimos.memory.store.sqlite.SqliteStore object at 0x739499897ef0>
name = 'tf', payload_type = <class 'dimos_generated.tf2_msgs.msg.TFMessage'>
config = {'eager_blobs': False, 'must_exist': True, 'page_size': 256, 'path': '.../dimos/data/alfred_stereo_short.db'}
stored = {'blob_store': {'class': 'dimos.memory.blobstore.sqlite.SqliteBlobStore', 'config': {'path': None}}, 'codec_id': 'lcm', 'eager_blobs': False, 'notifier': {'class': 'dimos.memory.notifier.subject.SubjectNotifier', 'config': {}}, ...}
actual_module = 'dimos_generated.tf2_msgs.msg.TFMessage'

    def _create_backend(
        self, name: str, payload_type: type[Any] | None = None, **config: Any
    ) -> Backend[Any]:
        validate_identifier(name)
    
        stored = self._registry.get(name)
    
        if stored is not None:
            # Load path: validate type, assemble from stored config
            if payload_type is not None:
                actual_module = f"{payload_type.__module__}.{payload_type.__qualname__}"
                if actual_module != stored["payload_module"]:
>                   raise ValueError(
                        f"Stream {name!r} was created with type {stored['payload_module']}, "
                        f"but opened with {actual_module}"
                    )
E                   ValueError: Stream 'tf' was created with type dimos.msgs.tf2_msgs.TFMessage.TFMessage, but opened with dimos_generated.tf2_msgs.msg.TFMessage

__class__  = <class 'dimos.memory.store.sqlite.SqliteStore'>
actual_module = 'dimos_generated.tf2_msgs.msg.TFMessage'
config     = {'eager_blobs': False, 'must_exist': True, 'page_size': 256, 'path': '.../dimos/data/alfred_stereo_short.db'}
name       = 'tf'
payload_type = <class 'dimos_generated.tf2_msgs.msg.TFMessage'>
self       = <dimos.memory.store.sqlite.SqliteStore object at 0x739499897ef0>
stored     = {'blob_store': {'class': 'dimos.memory.blobstore.sqlite.SqliteBlobStore', 'config': {'path': None}}, 'codec_id': 'lcm', 'eager_blobs': False, 'notifier': {'class': 'dimos.memory.notifier.subject.SubjectNotifier', 'config': {}}, ...}

.../memory/store/sqlite.py:161: ValueError

dimos.core.test_async_module_rpc_sync_to_async::test_async_module_rpc_sync_to_async[zenoh]

Stack Traces | 3.75s run time

start_module = None
in_transport = <dimos.core.transport.pZenohTransport object at 0x7f3e84ffa770>
out_transport = <dimos.core.transport.pZenohTransport object at 0x7f3e84ffb880>

    def test_async_module_rpc_sync_to_async(start_module, in_transport, out_transport):
        """
        Test that you can call a synchronous RPC from an asynchronous RPC and vice versa.
        """
        queue = Queue()
        out_transport.subscribe(queue.put)
        in_transport.publish(4)
>       cubed = queue.get(timeout=0.1)

in_transport = <dimos.core.transport.pZenohTransport object at 0x7f3e84ffa770>
out_transport = <dimos.core.transport.pZenohTransport object at 0x7f3e84ffb880>
queue      = <queue.Queue object at 0x7f3e84ffbe80>
start_module = None

dimos/core/test_async_module_rpc_sync_to_async.py:111: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <queue.Queue object at 0x7f3e84ffbe80>, block = True, timeout = 0.1

    def get(self, block=True, timeout=None):
        '''Remove and return an item from the queue.
    
        If optional args 'block' is true and 'timeout' is None (the default),
        block if necessary until an item is available. If 'timeout' is
        a non-negative number, it blocks at most 'timeout' seconds and raises
        the Empty exception if no item was available within that time.
        Otherwise ('block' is false), return an item if one is immediately
        available, else raise the Empty exception ('timeout' is ignored
        in that case).
        '''
        with self.not_empty:
            if not block:
                if not self._qsize():
                    raise Empty
            elif timeout is None:
                while not self._qsize():
                    self.not_empty.wait()
            elif timeout < 0:
                raise ValueError("'timeout' must be a non-negative number")
            else:
                endtime = time() + timeout
                while not self._qsize():
                    remaining = endtime - time()
                    if remaining <= 0.0:
>                       raise Empty
E                       _queue.Empty

block      = True
endtime    = 1679.2067779899999
remaining  = -0.00011059800021939736
self       = <queue.Queue object at 0x7f3e84ffbe80>
timeout    = 0.1

../...../_temp/uv-python-dir/cpython-3.10-linux-x86_64-gnu/lib/python3.10/queue.py:179: Empty

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}