Repository navigation
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add explicit
dimos mem convertfor historical MCAP/memory SQLite recordings. Conversion uses an allowlisted retained LCM decoder and the installed generated CDR registry, embeds full dependency schemas, and preserves an adjacent per-row audit. Runtime transports never fall back to legacy decoding.Parent: #4703. Message Encoding Refactor
This integrates the previously reviewed converter design into the restructured stack with the current registry/value ownership and storage contracts; the prior converter branch and its evidence are preserved.
Manual test with an existing trusted local recording (replace
old.dbwith its path, choose new output names):Use
.dboutput to preserve existing vectors; MCAP is recording-only and rejects vectors. For a directory,dimos mem convert ./old-recordings ./cdr-recordings --format mcap --dry-runinventories every candidate without outputs or downloads. Removing--dry-runperforms the batch. See the embedded-code walkthrough.Existing files/reports and competing writers are never overwritten. Originals are read-only. A late single-file payload error publishes no output; partial batch success is explicitly reported with nonzero exit. Pickle, unsupported/custom types, external component stores and attachments are rejected. Header.seq and SQLite IDs/value/tags are audited rather than silently dropped. JPEG bytes are not recompressed. Dry-run validates metadata/schema/counts; full payload integrity is checked during conversion.
Validation:
go2_mid360_stairs.db: all 4,451 messages, five streams.mem summaryreads every stream.go2_short.db-> CDR SQLite: all 2,546 messages, four streams, 108 exact 512-dimensional vectors and their observation associations preserved.dbb725b763f9d268c47c56e285f5c0150923f488e8fa03b6cdffd26d0dbd125e; go2_short4f056a042d58ad1dbc8248e408bd61d98bb90e5989cf0520c5400a1088dfe1b9.3a152b4b052af8cc66b82f1732ece372d6acfdd15c6906566accdacbc75525fa. Vector tests use official CPU PyTorch only, no model weights or GPU work.Remaining:
dimos mem rerunstill assumes legacyto_rerun()methods and skips plain generated values; its external-adapter migration is the next layer. Foxglove local human acceptance/sign-in remains external. Full consumer graphs and legacy dependency retirement remain open. No CI polling, package publication, hardware, history rewrite or merge.