Route topology-discovered host removals through the cluster lifecycle #1007

Parent epic: #921

Builds on #1006 and is ordered after the removed-host DOWN-work fix tracked by #1034 / #1035.

Related to #922, #923, and #813.

Problem

Topology reconciliation removes host IDs that are absent from the latest system-table result with Metadata.remove_host_by_host_id(). This updates metadata directly and bypasses Cluster.on_remove().

The endpoint-collision path added by #1006 removes a predecessor through the full cluster lifecycle when a different ID is observed at an equal endpoint. That detection still depends on equality of the constructed EndPoint. Two supported endpoint types incorporate host identity into equality:

• SniEndPoint includes the server name derived from host_id.
• ClientRoutesEndPoint includes host_id directly.

When the advertised topology address is retained but the node UUID changes, the new row therefore produces a different endpoint key. The predecessor reaches the absent-ID cleanup instead of the lifecycle removal path.

Impact

The old host disappears from metadata while runtime state can remain active:

• its Session pool remains open and indexed;
• load-balancing policies retain the old host;
• registered listeners miss the removal;
• removal of a control host may not initiate recovery;
• metrics may continue reporting the stale pool;
• the terminal removal fences added by host: make host_id the canonical identity #1006 are not engaged, so in-flight work can still publish effects for the old object.

Expected behavior

Every host removed by topology reconciliation should pass through the complete cluster removal lifecycle exactly once, regardless of the equality semantics of its connection endpoint.

host_id is immutable after #1006. Removal must apply only to the exact Host object selected from the topology snapshot while that object is still the current metadata entry for its ID. Endpoint-index cleanup must likewise verify that the removed host still owns the mapping.

Node-list reconciliation through absent-ID cleanup must be serialized so lifecycle callbacks for an old object complete before another refresh can publish equal replacement state. A stale snapshot must not remove or run lifecycle work against a newer Host object carrying the same immutable ID.

Cleanup performed inside an active node-list refresh must not recursively start another node-list refresh. Suppressing that refresh must not suppress session-pool reconciliation or recovery when the removed host actually owns the active control connection. The same-endpoint replacement path from #1006 remains distinct because its replacement addition owns pool and control-connection follow-up.

Acceptance criteria

• Replacing a node behind SniEndPointFactory removes and shuts down the predecessor pool.
• The equivalent ClientRoutesEndPointFactory replacement behaves identically.
• Ordinary topology-discovered removals also run the complete lifecycle.
• Policies and listeners receive exactly one removal.
• A discovered replacement receives exactly one addition.
• Eligible session pools are reconciled after removal.
• Metadata contains no stale host or endpoint-index entries.
• Removal marks the predecessor object terminal so its in-flight pool or reconnection work cannot publish afterward.
• Cleanup does not recursively refresh topology.
• Removing the actual active control host still initiates recovery, while host: make host_id the canonical identity #1006 replacement-owner behavior is preserved.
• Removal is conditional on exact object identity; a stale snapshot cannot remove an equal but newer registered Host.
• Deterministic overlapping-refresh coverage verifies exactly-once lifecycle removal, consistent metadata indexes, and preservation of the current host in policies and sessions.