Skip to content

refactor(camera): use CDR values and explicit TF codecs in RealSense - #4657

Closed
TomCC7 wants to merge 5 commits into
cc/feat/cdr-native-camera-producersfrom
cc/feat/cdr-native-tf
Closed

TomCC7 wants to merge 5 commits into
cc/feat/cdr-native-camera-producersfrom
cc/feat/cdr-native-tf

Conversation

@TomCC7

@TomCC7 TomCC7 commented Oct 20, 2026 •

Copy link
Copy Markdown
Member

Linear project: Message Encoding Refactor

RealSense now publishes generated Image, CameraInfo, PointCloud2, Imu and project-owned ImuInfo values with CDR. The handwritten ImuInfo fingerprint/encoder is removed. Python port types and Rust registry names agree, including dimos_msgs/msg/ImuInfo.

The native TF SDK adds optional #[tf(decode = ..., encode = ...)] hooks. Existing bare #[tf], lookup and publish operations retain their behavior. RealSense selects canonical TFMessage CDR through these hooks; the generic SDK does not acquire a dependency on the built-in message catalog. Codec errors propagate, and one TF topic rejects mixed codec pairs.

The self-contained RealSense flake pins SDK/configuration source 62b899b2d3b0d315988df7db25664264391fb4ea, archive SHA256 8d920aba91b51ba63f18e4cd88cdfff95ff4f3aa2dfbb34f2ef209a59182c75c. It generates the same owner used by ordinary package builds. V4L2/RealSense crate hash keys now match the unchanged locked LCM transport source revision; no forked LCM code was changed.

An installed SDK test also exposed a packaging defect: the SDK Cargo manifest declared an example whose source was omitted from the wheel. The wheel/sdist now carry that existing source unchanged. The fix is also published in its owning SDK layer #4649 at 6065e638b117f3fac565afde4e27aeeed4d35a4d. The SDK example is shipped byte-identically. No benchmark was executed, investigated or tuned. The existing ignored RealSense benchmark only received the field/codec substitutions required for the generated value types to compile.

Dependent draft after #4656. ZED, downstream consumers, default-transport cutover and final cleanup remain later work. No camera worker main, device enumeration/capture, robot, installer, firmware tool or driver operation was run. No registry publication.

Validation

Published head: a4313e544d0eb064a455bbbe5b1e5f503b788f63. Functional/package acceptance ran on 9deac468ad978eaeba80bb29b384a19c6ab4ae10; the final two commits only align native hash metadata.

  • 148 native SDK tests and 22 macro tests passed. Clippy passed with warnings denied; default feature compilation passed.
  • Official librealsense 2.56.5 source was built in the task workspace. Its library linked six passing RealSense regression tests. The same six passed in the exact pinned SDK/generated-owner/local-worker source assembly. No install target or host ldconfig was run.
  • Five actual Rust CDR fixtures decoded as installed Python owners and re-encoded identically. Checks include TF, calibration, RGB image, point-cloud positions/colors/source time, and IMU noise metadata. Tests use real SDK intrinsics and synthetic frame data.
  • 48 Python camera/viewer checks passed from source and from the installed wheel. Strict typing passed eight camera/viewer sources.
  • The SDK export fix passed eight build checks and strict build-helper typing. A fresh sdist rebuilt a wheel; artifact inventory passed. Installed SDK + installed generated Rust owner then compiled and passed the six RealSense tests outside checkout imports. pip check passed. The wheel SHA256 is 48e8a41295eb8d9caf3275c525f3d9f45a67f778f1e4d6d533e187b10a3ae524.
  • Existing native path-isolation and locked-source hash checks pass at the published head. Available hooks passed. Nix lint/derivation execution remains unverified because this host has no Nix executable. ARM and physical capture remain unverified. CI was not polled; no all-green claim.

Try it without a camera

The embedded walkthrough lists ports, artifacts and prerequisites. On a Nix development host:

cd dimos/hardware/sensors/camera/realsense/rust
nix build path:.#tests

For a repository developer with the checkout installed, Rust/C toolchains and librealsense available through pkg-config:

cargo test --manifest-path dimos/hardware/sensors/camera/realsense/rust/Cargo.toml \
  --locked --bin realsense_native --jobs 2 -- tests:: --skip bench::

The Nix command was not run locally. The Cargo regressions were run with the isolated official SDK build tree. They check encoding, normalized timestamps in both byte orders, calibration/projection, frame/color alignment and IMU pairing; they do not start capture. The existing floating-point graph-time API is retained; raw CDR uses generated Time fields.

File change summary

Three-dot diff from merge-base 5385ae1df08b5ae8091cdc2f4aee9002cf73acd6: 17 files, 603 additions and 155 deletions, excluding binary lockfile line counts. Parent #4656 also has later autofix commits; final stack alignment remains scheduled after the remaining slices.

Category Files Added Deleted
Native TF SDK, macro and regressions 3 243 38
Python/native camera and regression tests 3 261 87
Native manifest and isolated Nix assembly 2 39 17
Wheel/sdist SDK export completeness 4 7 1
Locked-source hash metadata 2 4 8
Documentation and CI 2 49 4
Cargo dependency lock 1 binary binary

No generated message classes or upstream .msg snapshots are checked in.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant