Conversation
…e, and report what the neighbours say A patch on top of the release that replaced the LMDB chunk store, which has already merged and is running on nodes part-way through their migration. It changes no on-disk format and writes no migration state: a node mid-flight re-reads its marker, its first-start time, its shed count and its remaining keys and carries on. A test asserts that rather than the commit claiming it. Two things, and only two. **A node can no longer be made to deny a promise it already made.** Narrowing a promise is fine and is untouched: a node that sheds rotates to a commitment over fewer keys, and `rotate` keeps the old roots as retained slots, so a peer that pinned one can still challenge it and get an answer until it ages out by its gossip TTL. That is the staged narrowing this migration was designed around. The rotation also had a branch that fired when a node had NO responsible keys at all, not a smaller set. It dropped every retained root outright when the node "looked empty", decided from a key count, and that count was wrong in the same direction every time it was fixed: a node whose disk filled before it could copy anything, then one whose file had been dropped from the index by a failed read while its legacy copy was being restored, then a files-only node that had published bytes to disk but not yet indexed them, because a file is published before it is indexed. Each fix closed one window and left another. So the branch is gone, and with it the question. The asymmetry settles it: clearing repudiates a root a peer is still pinning, which answers UnknownCommitment and is graded a confirmed failure on the commitment-bound lane, enforced in every release and not the lane the migration holds off; retiring stops advertising and stays answerable until the gossip TTL lapses. Both set has_current = false. A genuinely empty node cannot answer either way, so retiring costs it nothing it had. **Every node says whether it still has an old chunk store, and reads what its neighbours say.** The state rides the user agent saorsa-core already sends with every signed message and keeps for each peer, so it costs no new message, no new field and no protocol version, and the `node/` prefix that gates DHT membership is preserved. Three states, never folded into two: a directory that could not be read is not one that is not there. It counts what the peers a node is connected to announced, each as of that peer own last start: saorsa-core copies the user agent when it builds the transport, so a node that finishes migrating keeps announcing legacy until it restarts. Two consequences, running in opposite directions, so the tally bounds nothing. A peer announcing legacy may have finished since, so the count can be too high. A node that is offline, or simply not connected to, is absent from it, so the count can be too low. An all-zero tally proves nothing on its own either, because a node connected to nobody produces one; the number of peers seen is what tells that apart. So this can surface nodes that have not finished. It cannot establish that none remain, and no amount of it adds up to that. A peer still on the previous release announces no token and counts as unreported, never as finished. The reporter holds a weak handle to the node. A reporter must not be the reason the thing it observes stays alive: a strong one would keep a dropped node's transport, and its bound port, for as long as the task ran. The signal module is crate-private; neither goal needs new public API. The terraform worker unit passes --enable-logging, without which a node on that path emits nothing at all and none of this is readable.
…ardening-follow-up fix(storage): stop the migration denying a promise a node already made, and report what the neighbours say
Once a node reaches `Committed`, the copier moves only the keys the rank check refuses to shed. A chunk that turns up in the legacy environment after that, and that this node never agreed to give up, is on no such list: nothing copies it, the shed gate correctly refuses to prove a single copy exists elsewhere, and the startup reconciliation only returns a node to `Bridging` when the file store holds less than it recorded keeping, which the file store never does. The node is terminal in that state. Free disk does not change it and neither does a restart. Seen on one node of 38 in the 2026-09-08 beta cohort: it committed with "nothing has to be shed", then pre-retirement verification found 317 chunks in the legacy environment that were in neither view and re-queued them. It has refused retirement every 30 seconds since. Once WithAutonomi#218 removes the LMDB reader those 317 stop being servable, and on the evidence they are the network's only copy. The fix is a third arm in the reconciliation block, next to the two of the same shape: when the marker says `Committed` and there are more legacy-only keys than the node ever approved for shedding, go back to `Bridging`, clear `committed_at_unix` and reset `rebuilds_since_commit`. `open_legacy` already computes the legacy-only set before the marker is read, so this is a comparison against a number that is already in memory. No new I/O, no marker format change. The count, not emptiness, is the test. A node that legitimately shed keeps exactly the keys it is giving up in the legacy environment until they stop being answerable, so a non-empty legacy-only set is its normal state; bouncing it to `Bridging` on every restart would reset its retention clock each time and a regularly restarted node would never retire. The arm fires only on `legacy_only > shed_key_count`. It logs `migration_event = "back_to_bridging"` with both numbers so the case is visible in the beta watch. The upgrade's own restart is enough to take an affected node out of the loop. Tests: `cargo test --lib` 1104 passed, 0 failed; fmt clean; clippy with `-D clippy::panic -D clippy::unwrap_used -D clippy::expect_used` clean on lib and tests. Two tests added, and both were mutation-checked: with the discriminator replaced by `legacy_only > 0`, the new regression test and the existing `the_migration_marker_survives_a_restart` both fail; with the arm disabled, the fire-case test fails. Closes V2-1232 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UrjwPBwPqMi3F2sT1yVYEa
fix: reopen the bridge for chunks a committed node never agreed to shed
…ported `tally_peers` lists the connected peers and then reads each one's user agent. saorsa-core drops a peer's agent when its last channel closes, so a peer that disconnects between the two reads has no agent, and the tally counted it as a node running a build from before the migration signal. On a 990-node testnet this put a departing client in `peers_unreported` on one tick in 21,402, which is exactly the count the release gate for removing the LMDB store reads. A peer with no agent that is no longer connected is now skipped: it is not a peer this node can see. A peer that is still connected with no agent recorded keeps its place in the unreported bucket, so the count stays conservative with or without the saorsa-core change that records the agent under the peer's connection entry.
7 tasks
dirvine
approved these changes
Sep 15, 2026
dirvine
left a comment
dirvine
left a comment
Member
There was a problem hiding this comment.
APPROVE — reviewed at exact head 674aa0d2eb9dcf794497a69ac6affc15accf47ff together with saorsa-core #163.
The tally fix is conservative in every relevant interleaving:
agent=Nonefollowed by disconnected means the peer left after theconnected_peerssnapshot, so skipping it removes the observed falseUnreportedtick.- If the peer reconnects between the two reads, it remains
Unreported; that is a possible false outstanding count, not false clearance. - If an agent is read before a disconnect, the snapshot's last announced state is counted, matching the existing snapshot semantics.
- A still-connected peer with no agent remains
Unreported.
The change therefore cannot manufacture a Files result or falsely clear the migration gate. It is safe on its own and gains the stronger missing-agent meaning from saorsa-core #163.
Verification:
- focused migration-signal suite: 15 passed
- clippy all targets/features: passed
- no-default-features library check: passed
- formatting and rustdoc warnings check: passed
- GitHub build/test/lint/filesystem/platform matrix is green
Non-blocking gap: no deterministic mid-tally disconnect/reconnect test was added. The branch logic is small and the paired transport invariant was checked directly, so I do not consider this a merge blocker.
The failing Security Audit is inherited, not introduced: Cargo.lock is byte-identical to the base and the newly published RUSTSEC-2026-0285 affects that existing rustls version. It should be handled by the release train in a separate dependency bump to rustls >=0.23.45.
Operationally, this is consistent with — and closes — the reported false-positive path in the #218 testnet. I have not independently reclassified the whole run from its Linear evidence here. The peer tally remains observational evidence rather than proof of whole-fleet completion, so the rest of the ADR-0015 gate still applies.
7 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Linear issue
Closes V2-1260
Risk tier
Changes the migration signal's output: the per-node tally no longer counts a peer that disconnected mid-tally, and no
peer_stateline is written for it. No wire, format or penalty change.Compatibility
Semver impact
Test evidence
cargo test --lib --features test-utils migration_signal: 15 passed, 0 failed.cargo clippy --all-targets --all-features -- -D warnings(the CI invocation),RUSTFLAGS=-D warnings cargo check --no-default-features --lib,cargo fmt --all -- --check,RUSTDOCFLAGS=--deny=warnings cargo doc --no-deps: clean.peers_unreported=1, aclient/0.27.3connection tallied withagent=none. The tally takes the peer list, then reads each agent, and saorsa-core drops the agent when the peer's last channel closes. No unit test: the race needs a peer to disconnect inside the loop, which the store harness cannot stage.saorsa-corepin inCargo.lockmoves when the rc picks that PR up.New dependency
none
ADR
https://github.com/WithAutonomi/ant-node/blob/rc-2026.9.2/docs/adr/ADR-0014-file-based-chunk-store-and-lmdb-retirement.md
Mitigation / rollback
Revert the commit. It only changes the migration signal's log output.