[TomCC7]

Add explicit CDR ports over existing raw LCM/Zenoh, plus an actual Python → C++ → Rust → Python SDK coordinator example that runs outside the checkout. Existing factories, native default codecs and handler APIs stay unchanged.

Project: https://linear.app/dimensional/project/message-encoding-refactor-05a3ddb4af3e/overview

Head: daa87cb57fbff1cea730cbfa4ce994a731609833. Base: external-authoring #4647 at 32b5e778de754dad4d31f80c63bd92d310ed1d0b. Python recording #4645 is independent; it is not a prerequisite of this example.

flowchart TD
    D[One DeviceReading.msg + project config] --> G[Normal pip / CMake / Cargo hooks]
    G --> T[Owned values + imported built-in Header + complete schemas]
    T --> P[Python Out port]
    P -->|Plain CDR over loopback Zenoh| C[C++ SDK input and output]
    C -->|Plain CDR| R[Rust SDK input and output]
    R -->|Plain CDR| I[Python In port: exact result verified]
    W[Installed main dimOS wheel] --> S[Owned SDK sources and build hooks]
    S --> C
    S --> R

Loading

Review by category

Category	Files	What changes
Encoding/runtime	3	Explicit Python CdrTransport; optional Rust cdr module/feature export. C++ uses the existing published-library codec hooks.
SDK source packaging/build integration	5	Carry owned C++/Rust SDK sources in main wheel/sdist, expose CMake target and Cargo source patches. Pip does not compile those SDKs.
Tests	3	Raw transport, lifecycle, malformed data, source packaging, actual installed native SDK/coordinator checks and non-interactive manual refusal.
Executable example	8	Existing message-only example gains optional SDK modules and coordinator demo. Separate module Cargo manifest keeps SDK compilation out of message-only Cargo builds.
Guide	1	Short port/config examples, native prerequisites and no-publication Git review path.
CI and dependency metadata	3	Linux actual-SDK acceptance; optional Rust codec dependency; existing transport Git dependency pinned to its already-locked revision.

The new 110,564-byte example Cargo.lock is dependency resolution metadata (4,485 lines), not generated message classes or copied upstream implementation. No upstream library code is added to Git.

Run the actual new example without publishing

Use a fresh Python 3.12 environment and a C++ compiler for dimOS's existing A* source extension. Native SDK compilation additionally needs C++20/CMake, Rust/Cargo, liblcm development files and Zenoh-C/C++ >=1.9 with unstable API enabled. Supply those libraries through normal system/toolchain prefixes. CMake prepares pinned PFR/JSON sources if not cached; it installs no OS packages.

From this PR checkout:

python3.12 -m venv build/message-review-venv
source build/message-review-venv/bin/activate
python -m pip install "dimos[message-build] @ git+https://github.com/dimensionalOS/dimos.git@daa87cb57fbff1cea730cbfa4ce994a731609833" 'setuptools>=77' wheel
cd examples/external-messages
python -m pip install --no-build-isolation .
export CMAKE_PREFIX_PATH="$(python -m dimos.message_codegen.native cmake-prefix):${CMAKE_PREFIX_PATH:-}"
python -m dimos.message_codegen.native configure-cargo
cmake -S . -B build/cpp -DDIMOS_BUILD_MODULE_EXAMPLE=ON
cmake --build build/cpp
cargo build --locked --manifest-path modules/Cargo.toml
python demo_modules.py

The fresh environment matters: pip can keep an installed same-version proposal even when the requested Git commit changes; --upgrade alone did not replace it in the verification environment.

The explicit review-only --no-build-isolation reuses the Git-installed backend: public dimOS 0.0.14 does not yet contain this proposal. Released matching wheels support ordinary isolated pip install .; that path is tested in #4647. A matching complete wheelhouse also supports isolated review builds. No release, registry publication or manual generator script is required. Built-in-only users can omit the message-build extra.

Run the manual coordinator in a terminal and decline optional host-memory setup recommendations from the existing coordinator. This demo refuses non-interactive manual startup; its pytest acceptance uses the repository's standard system-configuration skip. Tests perform no host changes.

Expected output:

PASS: coordinator Python -> C++ -> Rust -> Python over loopback Zenoh CDR

The exact result is value 4.5, label sensor/cpp/rust, sequence 7, stamp 17s + 23ns, frame map, with the same dependency-owned Header class. The example starts a temporary loopback router with discovery/multicast disabled, deploys real SDK processes through ModuleCoordinator, and disposes subscriptions/modules/router. This is actual SDK pub/sub, separate from the file-codec demo in #4647.

Self-contained walkthrough.

Validation and bounds

• 13 Python transport tests passed: actual LCM in-memory queue and discovery-disabled local Zenoh, owned class and wire bytes, malformed input, lifecycle and pickle configuration.
• 32 source/build tests passed; three production helpers pass strict typing.
• Actual installed C++/Rust SDK example passed outside checkout; repeated with its committed Cargo lock and final manual-startup guard. Both the normal coordinator result and refusal without interactive confirmation are asserted.
• Rust codec literal-byte/both-endian and malformed/truncated/trailing-payload tests passed. The feature-off default SDK check passed against existing locked dependency versions.
• Compiler-prohibited external wheel installation passed. A fresh full-dependency Git source install at 05bfad8c873db8f91971ea98c981cbffd44e8ae4 passed pip check, the 73-type installed-package demo and SDK-source/API checks. The documented external source install then passed using that Git backend with no wheelhouse or publication. Final HEAD daa87cb57fbff1cea730cbfa4ce994a731609833 then passed normal full-resolution Git source installation after removing the earlier same-version dimOS package; direct_url.json confirms that exact commit. Its pip check and installed-package/SDK checks passed.
• Configured non-Nix hooks passed. Repository-wide Nix native lint remains unverified locally because Nix is absent. No CI or benchmark result is claimed; no CI polling or gate waiver.

Native builds are host-only and need explicit toolchains/development libraries. Main Python wheels carry source/build inputs for native SDK consumers, not precompiled SDK/message libraries. ROS message/wire compatibility is not ROS node integration. CDR channel suffixes isolate this opt-in path from legacy encoding; they are not runtime schema-hash negotiation. Native Rust recording, browser/domain consumers, default cutover and legacy cleanup remain later work. No physical robot operation or default migration is included.

SDK export completeness

Published correction: 6065e638b117f3fac565afde4e27aeeed4d35a4d. Wheel/sdist exports now include unchanged SDK example sources declared by Cargo, so ordinary Rust consumers can parse the shipped SDK manifest. Eight focused source-build checks, strict build-helper typing and applicable hooks passed on this branch. The same export fix was verified through sdist → wheel → installed SDK/generated owner → compiled RealSense regression in #4657 at 9deac468ad978eaeba80bb29b384a19c6ab4ae10. Those later native tests are not a terminal CI result for this PR. No benchmark was executed or tuned.

[github-actions]

go2 replay realtime (arm64)

Details

Benchmark suite	Current: 6065e63	Previous: 17fdb6b	Ratio
peak memory	1590.086 MB	1589.477 MB	1.00
peak threads	358 threads	350 threads	1.02
network (transport)	2724.815 MB	2730.079 MB	1.00
disk write	1.688 MB	1.707 MB	0.99
instructions	115.459 G	115.497 G	1.00

This comment was automatically generated by workflow using github-action-benchmark.

[codecov]

Codecov Report

❌ Patch coverage is 82.81250% with 44 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
dimos/message_codegen/test_project_native.py	13.33%	25 Missing and 1 partial ⚠️
dimos/message_codegen/build.py	77.77%	5 Missing and 1 partial ⚠️
dimos/message_codegen/native.py	62.50%	5 Missing and 1 partial ⚠️
dimos/core/cdr_transport.py	94.66%	2 Missing and 2 partials ⚠️
dimos/message_codegen/test_bundled.py	0.00%	2 Missing ⚠️

@@                            Coverage Diff                            @@
##           cc/feat/cdr-external-message-projects    #4649      +/-   ##
=========================================================================
- Coverage                                  81.74%   81.67%   -0.08%     
=========================================================================
  Files                                       1652     1654       +2     
  Lines                                     154755   155008     +253     
  Branches                                   13058    13079      +21     
=========================================================================
+ Hits                                      126507   126600      +93     
- Misses                                     24607    24782     +175     
+ Partials                                    3641     3626      -15     

Components	Coverage Δ
Tests	96.52% <79.71%> (-0.04%)	⬇️

Flag	Coverage Δ
OS-macos-latest	75.67% <81.64%> (+<0.01%)	⬆️
OS-ubuntu-24.04-arm	75.44% <81.64%> (+0.01%)	⬆️
OS-ubuntu-latest	75.79% <81.64%> (+<0.01%)	⬆️
Py-3.10	75.79% <81.64%> (+<0.01%)	⬆️
Py-3.11	75.78% <81.64%> (+0.01%)	⬆️
Py-3.12	75.79% <81.64%> (+0.01%)	⬆️
SelfHosted-Large	31.85% <20.31%> (-0.17%)	⬇️
SelfHosted-Linux	40.34% <20.31%> (-0.04%)	⬇️
SelfHosted-macOS	40.65% <21.48%> (-0.04%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
dimos/core/test_cdr_transport.py	100.00% <100.00%> (ø)
dimos/message_codegen/test_build.py	100.00% <100.00%> (ø)
dimos/message_codegen/test_bundled.py	19.44% <0.00%> (-0.56%)	⬇️
dimos/core/cdr_transport.py	94.66% <94.66%> (ø)
dimos/message_codegen/build.py	81.81% <77.77%> (-1.52%)	⬇️
dimos/message_codegen/native.py	47.58% <62.50%> (+1.21%)	⬆️
dimos/message_codegen/test_project_native.py	16.23% <13.33%> (-1.01%)	⬇️

... and 13 files with indirect coverage changes

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.