Move the memory-source contracts and readers off the engine (#18 §B4)

[YellowSnnowmann]

Issue #18 §B4 — the source contracts and readers move off the engine.

core/src/sources/ described a source in the engine's vocabulary: its types, registry and readers were re-exports of, or thin delegations to, crate::engine::backend::sources. A host binding a different driver could not so much as describe a source. They live in a new tinymemory-sources crate now (5,952 lines moved), which names no engine.

Why the obvious smaller PR does not exist

Types-first was tried. Core's registry and readers delegate to the engine, and those delegations pass these exact types across the call — repointing the types alone produces E0308: mismatched types (two MemorySourceEntrys). The alternatives were a conversion layer (which §A1 just deleted) or moving types + registry + readers together. This PR is the latter.

Four decisions inside the move

1. MemoryConfig → &Path. The readers used exactly one field: config.workspace. They now take the workspace path itself — a narrower signature instead of an imported config type.

2. Errors retype exactly, not loosely. The contract's MemoryError already models PathEscape — the one variant ensure_within_base needs. SourceResult<T> = Result<T, tinymemory_api::error::MemoryError>, so a reader fails in the same vocabulary as the driver that stores what it read.

3. RawKind is a deliberate second copy. Verified safe before copying: raw_archive_coords is its only consumer, nothing outside the crate calls that, core never touches the type — it cannot cross a boundary. The doc at the definition says when to lift it into the contract instead.

4. The engine's sync feature gate is renamed network. tinymemory-sync is a sibling crate; one name for two things is how this workspace's SourceKind confusion started. The gate pays measurably:

   tinymemory-sources	crates
   default	63 — no HTTP stack
   network	150 — adds GitHub/RSS/web-page readers

Deliberately not done

No SourceItem/SourceKind rename. Both names also exist in tinymemory-api meaning different things (ingest entry with content vs. listing entry; content-kind vs. connector-kind). The pairs never appear in one scope, and renaming would churn ~150 call sites here plus 24 in OpenHuman. The crate doc records the distinction so the duplication reads as considered, not accidental.

Blast radius

• Core keeps its own SourceReader trait (over the host Config) and its two host-only readers (composio, twitter); only the delegation target changed. 9 core files touched.
• OpenHuman is untouched: it imports three functions from core::sources (apply_kind_defaults, get_source, list_sources) — all still re-exported, verified against its tree.
• The engine's own sources/ copy stays until a tinycortex companion PR deletes it — the same two-step §B3 used (Run Composio normalisation and storage on a driver that is not TinyCortex (#18 §B3) #41 + tinycortex#153). Companion follows this PR.

Lints

The moved code lands under this crate's stricter lints (unwrap_used, expect_used, missing_docs): two guarded unwraps rewritten to carry their proof (is_some_and, index carried in the Option), the registry's lock-poison expect replaced with PoisonError::into_inner recovery (guard data is () — nothing torn to inherit), reader structs documented. Tests keep unwrap via cfg_attr(test, allow) — a panic in a test is the failure report, and rewriting 159 assertions would obscure what each checks.

Validation

Command	Result
cargo fmt --all -- --check	clean
cargo clippy --workspace --all-targets	clean
cargo test --workspace	see checklist below
cargo test -p tinymemory-sources --features network	see checklist below
cargo deny check	advisories ok, bans ok, licenses ok, sources ok
dependency budget	minimal 40 ≤ 50 ceiling; --all-features 162→166 (the new crate's toml/uuid surface)

Merge order

Independent of everything currently open. The vendor/ gitlinks are untouched. Next after this: §B1 (sync onto the neutral types), then §B5/§E4 — the last unmet acceptance criterion.

[coderabbitai]

Warning

Review limit reached

@YellowSnnowmann, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 54 minutes

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits within each organization.

For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 625a35a8-e56e-4e8d-9af8-efcb0875befc

📥 Commits

Reviewing files that changed from the base of the PR and between dd6fd33 and 5fa80e5.

⛔ Files ignored due to path filters (1)

• Cargo.lock is excluded by !**/*.lock

📒 Files selected for processing (39)

• Cargo.toml
• core/Cargo.toml
• core/src/sources/readers/conversation.rs
• core/src/sources/readers/folder.rs
• core/src/sources/readers/github.rs
• core/src/sources/readers/rss.rs
• core/src/sources/readers/web_page.rs
• core/src/sources/registry.rs
• core/src/sources/sync.rs
• core/src/sources/types.rs
• sources/Cargo.toml
• sources/src/lib.rs
• sources/src/raw_kind.rs
• sources/src/readers/conversation.rs
• sources/src/readers/conversation_tests.rs
• sources/src/readers/folder.rs
• sources/src/readers/folder_tests.rs
• sources/src/readers/github.rs
• sources/src/readers/github/api.rs
• sources/src/readers/github/git.rs
• sources/src/readers/github/git_tests.rs
• sources/src/readers/github/issues.rs
• sources/src/readers/github/types.rs
• sources/src/readers/github_tests.rs
• sources/src/readers/mod.rs
• sources/src/readers/rss.rs
• sources/src/readers/rss/types.rs
• sources/src/readers/rss_tests.rs
• sources/src/readers/ssrf.rs
• sources/src/readers/ssrf_tests.rs
• sources/src/readers/web_page.rs
• sources/src/readers/web_page/types.rs
• sources/src/readers/web_page_tests.rs
• sources/src/registry.rs
• sources/src/registry_tests.rs
• sources/src/types.rs
• sources/src/types_tests.rs
• sources/src/validation.rs
• sources/src/validation_tests.rs

✨ Finishing Touches 💡 1

🛠️ Fix failing CI checks 💡

• Create stacked PR
• Commit on current branch

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[tinysweeper]

How this change flows

0 changed behaviours across 3 relationships. 5 surrounding behaviours are shown (60 graph nodes walked). 19 further behaviours left out to keep the diagram readable.

flowchart LR
  n0["SourceReader"]:::impacted
  n1["path"]:::impacted
  n2["MemorySourceEntry"]:::impacted
  n3["push"]:::impacted
  n4["parse_selector"]:::impacted
  n0 -->|uses| n1
  n0 -->|uses| n2
  n4 -->|calls| n3
  classDef changed fill:#0d4429,stroke:#238636,color:#e6edf3
  classDef impacted fill:#161b22,stroke:#6e7681,color:#c9d1d9
  classDef flagged fill:#5a1e02,stroke:#d93f0b,color:#ffffff
  classDef blocking fill:#67060c,stroke:#f85149,color:#ffffff

Loading

_{Green: changed behaviour. Grey: surrounding behaviour. Arrows name the call, use, implementation, or test relationship. Orange: has findings. Red: has a finding that blocks the merge.}

[tinysweeper]

tinysweeper found nothing blocking. Approving.

$0.0000 · 0 in / 0 out · 768 embedded · openrouter/openai/text-embedding-3-small