Run the conformance suite against the three hosted adapters (#18 §E1) - #38
Conversation
|
Important Review available on request
Reviews should be triggered manually for repositories with fewer than 10 stars. Select Trigger review above or comment ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![tinysweeper[bot]](https://avatars.githubusercontent.com/in/4526585?s=60&v=4){kind=small}
| submodules: recursive | ||
| persist-credentials: false | ||
|
|
||
| - uses: dtolnay/rust-toolchain@stable |
There was a problem hiding this comment.
dtolnay/rust-toolchain is pinned to stable, which is mutable
A tag or branch can be repointed by whoever owns dtolnay, and the new code runs with this workflow's secrets. Pin to a full commit SHA and let Dependabot bump it.
[RULE] unpinned-action ·
| with: | ||
| components: llvm-tools-preview | ||
|
|
||
| - uses: Swatinem/rust-cache@v2 |
There was a problem hiding this comment.
Swatinem/rust-cache is pinned to v2, which is mutable
A tag or branch can be repointed by whoever owns Swatinem, and the new code runs with this workflow's secrets. Pin to a full commit SHA and let Dependabot bump it.
[RULE] unpinned-action ·
|
|
||
| - uses: Swatinem/rust-cache@v2 | ||
|
|
||
| - uses: taiki-e/install-action@v2 |
There was a problem hiding this comment.
taiki-e/install-action is pinned to v2, which is mutable
A tag or branch can be repointed by whoever owns taiki-e, and the new code runs with this workflow's secrets. Pin to a full commit SHA and let Dependabot bump it.
[RULE] unpinned-action ·
tinysweeper
Bot
added
the
priority: p2
tinycortex#149 landed as a squash (8401346b), discarding the branch head this pin pointed at; 34cbb6c is diverged from tinycortex main rather than an ancestor of it. The merged commit is also the one that deletes the 33 duplicated files under api/src/, which is the state this stack depends on.
tinysweeper
Bot
added
priority: p3
|
Superseded by #39, which merged as a8dfab2 and carries this branch's work in full. #39 was built on this branch — its commit §E1 is complete either way — the conformance suite now runs against TinyCortex and all three hosted adapters, which is one of issue #18's eight acceptance criteria. Closing rather than merging: replaying it would re-apply changes |
1 participant
Summary
Issue #18's acceptance criterion 5 — "the conformance suite passes for TinyCortex and all three remote adapters."
The suite (#20) ran against the in-memory reference driver and the null driver. Both were written alongside it, so passing proved the assertions were self-consistent and not much else. The premise the whole issue rests on — that an engine other than TinyCortex can satisfy the contract — had never been exercised.
Each hosted adapter now runs the full
assert_providerover a real TCP socket, against a double that speaks its own HTTP shapes and retains what it is sent. All three pass: eleven assertions each, including taint preservation, upsert identity, namespace isolation, and the export/import round trip.That retention requirement is what separates these from #26's
failure_test, whose doubles only have to misbehave. These have to be working backends, because the suite writes and reads back.The paired retention probe earned itself immediately
Each adapter has a second test asserting its double genuinely retains. On the first run:
assert_providerreturns early when a driver does not retain — a legitimate contract state, since the null driver is exactly that. So it had run 4 assertions, skipped the 7 that matter, and reported success.Without the probe this PR would have said "Cognee passes the conformance suite", and that would have been false in precisely the way the suite exists to prevent.
The cause was my double, not the adapter. Cognee's data listing must carry a
nameending.tinymemory[.json]— the adapter skips anything else, because Cognee's own text loader strips the extension — and my listing returned onlyid, so every record was filtered out before the raw fetch. Found by instrumenting the double rather than reasoning about it.What this does and does not prove
It is not that Supermemory, Mem0 or Cognee uphold the contract — nobody here can prove that about someone else's service.
It is that the adapter does, given a backend answering its own documented shapes. A violation on the adapter's side of the wire — a dropped taint, a non-terminating export cursor, an upsert that duplicates instead of replacing — is caught here. The module docs state this so a reader does not over-claim from a green run.
Not covered: the TinyCortex adapter
Criterion 5 names it alongside the three, so it stays open. It needs
require_embedding_host(), a process-global, so driving it means installing host seams — which makes the test order-dependent unless isolated in its own target.AGENTS.mdrules out order-dependent tests, so this needs its own integration target rather than a line in this file.Public API changes
tinymemory_conformance::retains_writes— alreadypubin the suite module, now exported. A caller standing up its own backend double needs it, for the reason above.adapters/remotegains a dev-dependency on the conformance crate.Validation
cargo fmt --all -- --checkcargo clippy --all-targets --all-features -- -D warningscargo clippy -p tinymemory-tinycortex --all-targets --no-default-features -- -D warningscargo build --all-targets --all-featurescargo test --all-featuresRUSTDOCFLAGS="-D warnings" cargo doc --no-deps --all-featurescargo test -p tinymemory --no-default-features --test null_provider./scripts/ci/dependency-budget.shcargo test --libWhy this matters beyond #18
OPENCOMPANY_MEMORY=remoteis gated behindOPENCOMPANY_MEMORY_ALLOW_UNPROVEN_REMOTEin tinyhumansai/opencompany#936, and the gate's own documentation says it exists until the conformance suite covers these adapters and should then be deleted. This is that coverage for the three hosted drivers.Related
Part of #18 (§E1). Closes the hosted-adapter half of acceptance criterion 5.