feat(reachability): gate relay publication with canary quorum

[mickvandijke]

Linear issue

https://linear.app/autonominetwork/issue/V2-812/gate-proactive-relay-publication-with-independent-canary-verification Tracking issue: #138. Transport PR: WithAutonomi/saorsa-transport#131.

Risk tier

• T0 — docs / tooling / CI / pure UX-output. Repo CI only.
• T1 — client-only, no network-facing behavior change. CI + prod compat smoke.
• T2 — node/client logic with behavioral surface, no protocol/format/economics change. Dev testnet + ADR.
• T3 — protocol / storage format / payments / routing. T2 evidence + adversarial testing.

Compatibility

• Wire: adds the versioned internal relay-canary-v1 request/response topic; mixed-version request timeouts are treated as ineligible evidence rather than relay failures.
• Storage: none; DHT record encoding is unchanged.
• API: saorsa-core now consumes saorsa-transport's opaque PreparedRelay lifecycle; no new external crate is added.

Semver impact

• breaking
• feature
• fix

Summary

Gate proactive MASQUE relay publication behind independent canary evidence and continuously revalidate established relays. A relay allocation is provisional until admission quorum succeeds; a completed maintenance majority rejection withdraws it immediately.

Changes

• Add randomized, non-close relay witnesses with per-source rate limiting and a versioned authenticated canary request/response protocol.
• Require all eligible admission witnesses to succeed before a provisional relay enters the DHT self-record.
• Require a maintenance majority for established relays; retain/retry inconclusive evidence, but immediately withdraw a majority-rejected relay.
• Bind prepare, publish, and abort to the exact opaque transport allocation, preventing stale same-address operations from affecting a replacement relay.
• Keep DHT address publication synchronized with current K-closest peers and retry only pending/new targets.
• Ignore transport relay hints that could bypass the sequenced, canary-gated self-record path.
• Depend on https://github.com/WithAutonomi/saorsa-transport.git, branch fix/pr136-provisional-relay; Cargo.lock pins transport commit 4a2ceb96 from PR Revert PR #121 #131.

Test evidence

• cargo test --lib --quiet: 499 passed, 0 failed.
• cargo clippy --all-targets --all-features -- -D warnings: clean.
• saorsa-transport cargo test --lib: 1,488 passed, 3 ignored, 0 failed.
• saorsa-transport strict all-target/all-feature clippy: clean.
• downstream ant-node cargo check --workspace --quiet: clean.
• One-hour pr136-final-1h-0729 testnet: 200 nodes, 30% NAT, one continuous 50 MiB uploader; 200/200 node services active, zero restarts, zero established-relay rejection/withdrawal, RelayLost, unhealthy-tunnel, publish/abort/teardown failures, and 11,169 successful maintenance revalidations.
• That run exposed a redundant accepted-connection lookup; transport commit 4a2ceb96 removes it and adds a real-QUIC regression. The requested follow-up fleet was canceled before observation at the requester's direction.

New dependency

None. The existing saorsa-transport dependency temporarily follows PR #131's Git branch until that PR is released.

ADR

https://github.com/WithAutonomi/saorsa-core/blob/feat/relay-canary-gate-rc-2026.6.2/docs/adr/ADR-014-proactive-relay-first-nat-traversal.md

Mitigation / rollback

Revert ae2dbda and the six original canary commits, then restore the released saorsa-transport dependency.

[Copilot]

Pull request overview

This PR introduces a canary-gated MASQUE relay publication flow in the reachability subsystem, ensuring relay allocations are only published to the DHT after independent third-party verification and periodically revalidated thereafter.

Changes:

• Adds a versioned relay canary request/response protocol and integrates witness selection + quorum verification into relay acquisition/maintenance.
• Refactors proactive relay lifecycle to a prepare → publish/abort model using saorsa-transport’s opaque PreparedRelay handle.
• Improves address-set publication behavior by tracking acknowledgements and retrying only pending/new K-closest peers.

Reviewed changes

Copilot reviewed 12 out of 13 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
src/transport/saorsa_transport_adapter.rs	Adapts proactive relay lifecycle to prepare/publish/abort and updates relay health API.
src/transport_handle.rs	Threads PreparedRelay through setup/publish/abort, and adds TLS-authenticated canary dial plumbing.
src/reachability/session.rs	Filters acquisition candidates using a per-round exclusion set and logs prepared relay allocation.
src/reachability/mod.rs	Introduces the new canary module and updates module-level documentation.
src/reachability/driver.rs	Implements the canary-gated state machine, periodic revalidation, and publication retry targeting.
src/reachability/canary.rs	New: relay canary protocol types, witness selection, validation, and probe/quorum logic.
src/reachability/acquisition.rs	Switches acquisition output from SocketAddr to PreparedRelay to bind lifecycle to a specific allocation.
src/network.rs	Stops merging relay-ish transport hints into DHT routing state; refactors reconnect path to use typed candidates.
src/identity/node_identity.rs	Adds derivation of PeerId from transport-authenticated ML-DSA SPKI and tests.
src/dht_network_manager.rs	Adds canary request handling + per-source rate limiting and makes address publication concurrent/ack-tracking.
src/adaptive/dht.rs	Exposes an internal ensure_peer_channel wrapper for reconnect callers.
Cargo.toml	Switches saorsa-transport dependency to a git reference (branch).
Cargo.lock	Pins saorsa-transport to the referenced git commit.

[grumbach]

Agree with the reviews above, both on the canary teardown and on the witness quorum. Nothing to add there.

A few smaller things I did not see raised, none of them blocking:

• saorsa_transport_adapter.rs: DIRECT_CONNECT_TIMEOUT goes from 1250ms to 3s. Not mentioned in the description, and the comment right above it still says to keep it short so lookups move on quickly. Every unreachable candidate in every lookup now costs 3s.
• clear_dial_failures_for_published: skipping addresses already in the previous set does stop the redial loop, but a relay that fails once and then passes maintenance canary at the same address stays suppressed for the full cache TTL. That is the common case once the canary is keeping the address proven.
• Maintenance rejection withdraws after one round. The doc comment in canary.rs says established relays confirm the result in a second round before withdrawal, but only Inconclusive retries.
• The startup tombstone publishes to find_closest_nodes_local, which is near empty at driver start, so it does not reach the replicas actually holding the old record. And when there are no dialable self addresses it publishes an empty address set at a fresh seq, which tells everyone that receives it that the node has no dialable address at all.
• KClosestPeersChanged and broadcast lag both clear last_published_typed_set, so each one triggers a full K-wide republish. Before this only relayer eviction did that, and K-closest changes are frequent under churn.
• Relay ADD_ADDRESS suppression is gone on the send side in Revert PR #121 #131 (the relay_setup_attempted gates), while the receive side here became a no-op. The DHT_BRIDGE task now drains updates, logs, and does nothing. Worth dropping one side or the other.

[Copilot]

Pull request overview

Copilot reviewed 17 out of 18 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (3)

src/dht_network_manager.rs:4638

• Invalid relay-canary requests are currently rejected without sending any response. Because the requester treats missing responses as success for mixed-version compatibility, a rejected request can be miscounted as an assumed-success witness result (and could be abused by modified clients). Consider replying with an explicit failure result instead of silently dropping.

        if let Err(reason) = validate_relay_canary_request(&source_peer, &request) {
            debug!(
                peer = %source_peer.to_hex(),
                reason = %reason.summary(),
                "Rejecting relay canary request"

src/dht_network_manager.rs:5217

• When the isolated relay-canary semaphore is exhausted, the code drops the request without replying. With the current requester logic, that timeout is interpreted as a legacy "no protocol" response (assumed success), which can incorrectly admit/retain a relay under load. Prefer sending an explicit rate-limited response when capacity is exhausted.

                                        let Ok(permit) = semaphore.try_acquire_owned() else {
                                            warn!(
                                                peer = %source_peer.to_hex(),
                                                "Dropping relay canary request: isolated probe budget exhausted"
                                            );

src/reachability/canary.rs:45

• The PR description/compatibility notes mention a relay-canary-v1 topic and that mixed-version request timeouts are treated as ineligible evidence. This implementation uses relay-canary-v2 and explicitly treats missing canary-protocol responses as positive (assumed-success) evidence. Please align the PR description and/or protocol semantics so reviewers/operators know which behavior is intended.

/// Request/response protocol name used with `TransportHandle::send_request`.
pub(crate) const RELAY_CANARY_PROTOCOL: &str = "relay-canary-v2";

/// Wire topic emitted by the request/response wrapper for canary requests.
pub(crate) const RELAY_CANARY_WIRE_TOPIC: &str = "/rr/relay-canary-v2";

[Copilot]

Pull request overview

Copilot reviewed 17 out of 18 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (4)

src/transport_handle.rs:1501

• This doc comment still points to ADR-014, but ADR-016 is introduced as the accepted design and ADR-014 is marked superseded in this PR. Update the ADR reference so readers land on the current reachability design.

    /// This is the caller-driven entry point for ADR-014 relay acquisition.

src/transport/saorsa_transport_adapter.rs:2293

• connect_happy_eyeballs_authenticated documents that the returned address is always normalized, but in the racing (tokio::select!) path the returned DialedPeer.remote_addr is not normalized (and the fallback return Ok(peer) paths also bypass normalization). This can reintroduce IPv4-mapped/unnormalized addresses into callers.

        tokio::select! {
            res6 = v6_fut => match res6 {
                Ok(peer) => Ok(peer),
                Err(_) => {
                    for addr in v4_targets {

src/reachability/canary.rs:45

• PR description says the change adds the versioned internal relay-canary-v1 request/response topic, but the implementation defines and uses relay-canary-v2 (and /rr/relay-canary-v2). Please align the on-wire topic name with the PR description (or update the PR description/ADR accordingly) so mixed-version behavior is predictable.

/// Request/response protocol name used with `TransportHandle::send_request`.
pub(crate) const RELAY_CANARY_PROTOCOL: &str = "relay-canary-v2";

/// Wire topic emitted by the request/response wrapper for canary requests.
pub(crate) const RELAY_CANARY_WIRE_TOPIC: &str = "/rr/relay-canary-v2";

src/transport_handle.rs:1484

• These doc comments still reference ADR-014, but this PR introduces ADR-016 and marks ADR-014 as superseded. Update the reference so readers can find the current design rationale.

This issue also appears on line 1501 of the same file.

    /// Returns `true` if no relay was established or the relay is healthy.
    /// Returns `false` if a relay was established but the QUIC connection
    /// has closed. Used by the relayer monitor (ADR-014 item 6).
    pub async fn is_relay_healthy(&self) -> bool {

[Copilot]

Pull request overview

Copilot reviewed 17 out of 18 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (3)

src/reachability/canary.rs:56

• The PR description says mixed-version request timeouts should be treated as ineligible evidence, but the canary logic documents and implements NoProtocolResponse/timeout as an assumed positive result (see comment and RelayCanaryProbeDisposition::AssumedSuccess). Please reconcile this (either update the PR description/ADR text, or adjust the code to treat timeouts as ineligible rather than success).

/// Positive witness results needed before a relay is publishable.
///
/// Publication is unanimous across the selected independent witnesses. During
/// the mixed-version rollout, a request that was delivered but received no
/// canary-protocol response counts as a positive result so legacy nodes do not
/// make relay availability worse than before the canary gate.
const RELAY_CANARY_ADMISSION_SUCCESSES: usize = RELAY_CANARY_WITNESS_TARGET;

src/reachability/driver.rs:494

• publish_typed_set_with_policy may attempt to publish an empty typed address set (when force is true). In the current DHT implementation, an empty PublishAddressSet is ignored (DhtCoreEngine::filter_publish_address_set returns None on empty input), but the network handler still ACKs it, so this "withdraw stale relay" publish will appear successful while leaving prior addresses intact on peers.

        let typed = self_addresses.into_typed_vec();
        if typed.is_empty() {
            info!(
                "driver: publishing empty authoritative address set to withdraw stale relay state"
            );
        }

src/reachability/canary.rs:45

• The PR description says the internal canary topic is relay-canary-v1, but the implementation defines/uses relay-canary-v2 (and wire topic /rr/relay-canary-v2). Please align the PR description and implementation so mixed-version expectations and any external references match the actual protocol name.

This issue also appears on line 50 of the same file.

/// Request/response protocol name used with `TransportHandle::send_request`.
pub(crate) const RELAY_CANARY_PROTOCOL: &str = "relay-canary-v2";

/// Wire topic emitted by the request/response wrapper for canary requests.
pub(crate) const RELAY_CANARY_WIRE_TOPIC: &str = "/rr/relay-canary-v2";