Skip to content

build: integrate published CDR generator (1/8) - #4247

Merged
TomCC7 merged 49 commits into
mainfrom
cc/feat/remove-lcm-encoding
Oct 20, 2026
Merged

TomCC7 merged 49 commits into
mainfrom
cc/feat/remove-lcm-encoding

Conversation

@TomCC7

@TomCC7 TomCC7 commented Sep 21, 2026 •

Copy link
Copy Markdown
Member

Linear project: Message Encoding Refactor

dimOS uses the published dimos-cdrgen==0.1.0 directly. The library owns parsing, generation, codecs, ownership checks and native build recipes. This PR adds SDK discovery, a C++ codec header adapter, one consumer test file, a short guide and Linux/macOS CI. Runtime transport changes and built-in message catalogs remain in later PRs.

Validation

At ee4feff4a967ea49c4d965a0ffba86fe9b811db2:

  • Four Python/SDK consumer checks passed on Linux after merging main. The real C++ consumer separately compiled, ran and verified its CDR bytes offline (one case, 33.82 seconds), using pinned/hash-verified cached upstream sources. The parent test environment remains free of pip and catkin_pkg.
  • Merges main fbe126e741c44e14ee1c23bb17a7e68e2b6a00d6 with an additive merge commit. Resolved only pyproject.toml and uv.lock: preserves main's lcm package rename and retains the pinned generator. Every pre-existing locked package version/source is unchanged; six generator-related packages are added. Offline lock consistency and applicable conflict-file hooks pass.
  • Fixes the user-reported main macOS job, which failed because its C++ test environment lacked pip and catkin_pkg. The native test now creates its own build-tool environment with pinned dimos-cdrgen[native]==0.1.0; both main and dedicated CI use the same test setup. The parent test environment remains free of pip/catkin_pkg afterward.
  • Native build dependencies stay isolated: the generator's PyYAML pin does not alter the main dependency lock. DIMOS_OFFLINE=1 also applies to provisioning the build tools.
  • All applicable repository hooks passed. Earlier strict adapter typing and documentation branding checks remain applicable; this repair changes only the test and its workflow.

macOS was not run locally. The supplied failed job was inspected; replacement CI has not been polled or claimed green. This PR is open for review; merge approval is still required.

Ownership administration

The reviewer ownership requests remain open for maintainers. PyPI permission changes and package publication are outside this assistant's scope per the user's instruction; no administrative or publishing action was performed. Organization-source publication is also unchanged.

Downstream callers must migrate from dimos_message_build and generic dimos.message_codegen imports in their own PR scopes.

File change summary

Diff against merged main fbe126e741c44e14ee1c23bb17a7e68e2b6a00d6 at the head above:

Category Files Added Deleted
SDK and codec adapters 2 46 0
Consumer tests 1 194 0
Documentation 1 49 0
Dependencies and CI 3 45 1
Automated Nix lock normalization 3 binary binary

The dependency category includes binary-classified uv.lock, excluded from line counts. Total text changes: 334 additions and one deletion. The three Nix lock updates came from autofix.ci commit 9e52cd0a; they are not generator implementation. No upstream .msg snapshots or generated classes are checked in. Root conftest owns shared markers; the dedicated --noconftest commands declare their marker locally.

@codecov

codecov Bot commented Sep 21, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.75000% with 1 line in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
dimos/message_codegen/native_build.py 93.33% 0 Missing and 1 partial ⚠️
@@           Coverage Diff           @@
##             main    #4247   +/-   ##
=======================================
  Coverage   81.96%   81.97%           
=======================================
  Files        1657     1659    +2     
  Lines      154382   154462   +80     
  Branches    12992    12996    +4     
=======================================
+ Hits       126537   126614   +77     
- Misses      24244    24247    +3     
  Partials     3601     3601           
Components Coverage Δ
Tests 96.84% <100.00%> (+<0.01%) ⬆️
Flag Coverage Δ
OS-macos-latest 75.85% <72.50%> (-0.01%) ⬇️
OS-ubuntu-24.04-arm 75.62% <72.50%> (-0.01%) ⬇️
OS-ubuntu-latest 75.97% <72.50%> (-0.01%) ⬇️
Py-3.10 75.97% <72.50%> (+<0.01%) ⬆️
Py-3.11 75.96% <72.50%> (+<0.01%) ⬆️
Py-3.12 75.96% <72.50%> (-0.01%) ⬇️
SelfHosted-Large 32.15% <27.50%> (-0.01%) ⬇️
SelfHosted-Linux 40.61% <27.50%> (-0.02%) ⬇️
SelfHosted-macOS 40.93% <61.25%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
dimos/message_codegen/test_integration.py 100.00% <100.00%> (ø)
dimos/message_codegen/native_build.py 93.33% <93.33%> (ø)

... and 5 files with indirect coverage changes

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@TomCC7 TomCC7 changed the title cc/feat/remove lcm encoding feat: generate native CDR messages without ROS (1/6) Sep 21, 2026
Comment thread dimos/message_codegen/_vendor/rosidl_parser.py Fixed
Comment thread examples/message-codegen/demo_transport.py Fixed
@TomCC7
TomCC7 added this pull request to stack #4253 September 21, 2026 17:24
@TomCC7
TomCC7 removed this pull request from stack #4253 October 1, 2026 07:57
@TomCC7
TomCC7 added this pull request to stack #4372 October 1, 2026 07:58

@github-actions github-actions Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

go2 replay realtime

Details
Benchmark suite Current: 8ab9ab3 Previous: 13e4a21 Ratio
first frame wall 9.499 s 9.978 s 0.95
first frame cpu 20.307 s 20.862 s 0.97
run cpu 32.042 s 32.466 s 0.99
run cpu (user) 24.21 s 25.213 s 0.96
run cpu (system) 7.832 s 7.253 s 1.08
peak memory 2719.418 MB 2717.016 MB 1.00
peak threads 397 threads 406 threads 0.98
disk read 23.402 MB 28.266 MB 0.83
disk write 11.297 MB 11.234 MB 1.01
network (transport) 2714.792 MB 2715.069 MB 1.00
network (external rx) 0.122 MB 0.074 MB 1.65
network (external tx) 0.166 MB 0.338 MB 0.49

This comment was automatically generated by workflow using github-action-benchmark.

Comment thread docs/development/message-source-packages.md
Comment thread docs/development/message-source-packages.md
Comment thread .github/workflows/message-codegen.yml
@github-actions github-actions Bot removed the ready-to-merge Required CI checks have passed on this PR label Oct 20, 2026
@TomCC7
TomCC7 requested a review from paul-nechifor October 20, 2026 05:22
@github-actions github-actions Bot added the ready-to-merge Required CI checks have passed on this PR label Oct 20, 2026
@github-actions github-actions Bot removed the ready-to-merge Required CI checks have passed on this PR label Oct 20, 2026
@github-actions github-actions Bot added the ready-to-merge Required CI checks have passed on this PR label Oct 20, 2026
@TomCC7
TomCC7 merged commit e9f17bb into main Oct 20, 2026
77 of 80 checks passed
@TomCC7
TomCC7 deleted the cc/feat/remove-lcm-encoding branch October 20, 2026 17:03

This branch was successfully deployed

1 active deployment
cachix — 00eaa9e8 Deployed Oct 20, 2026 by TomCC7 via cachix-build-macos #13446
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-to-merge Required CI checks have passed on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants