Skip to content

feat: relay selector with init/on_failure script - #8819

Closed
weavedwires wants to merge 4 commits into
chatmail:mainfrom
weavedwires:main
Closed

weavedwires wants to merge 4 commits into
chatmail:mainfrom
weavedwires:main

Conversation

@weavedwires

Copy link
Copy Markdown

What

Replaces the built-in transport ordering (sorted_transports + the
smtp_success table) with a user-definable Rhai script that decides which
relay is activated for sending. The core no longer picks or caches relays — it
asks the script. A built-in default script implementing the standard
ping-and-failover algorithm is used when no user script is installed.

Script contract

Two functions are required:

  • init(relays) -> addr | () — called when a relay is needed and the previous
    attempt did not fail (first run, restart, or a stale reconnect). It may
    return the remembered relay for a fast start.
  • on_failure(relays, bad_relay) -> addr | () — called after the previously
    selected relay failed, with the failed relay as the second argument.

relays is an array of object maps, one per configured transport:
#{ id, addr, host, added }.

Available host functions inside the functions:

  • ping(host) -> int — TCP connect to port 443, returns RTT in ms, throws on
    failure (catchable with try/catch). Only hosts from the current relay
    set can be pinged.
  • state_get(key) / state_set(key, value) — persistent state, stored in
    the relay_selector table and surviving restarts.
  • mem_get(key) / mem_set(key, value) — volatile state, kept in memory
    for the process lifetime and cleared on restart.
  • now(), log(message).

Splitting persistent and volatile state lets the default script keep its ping
cache (order) volatile and only its current/bad lists persistent.

Core flow

Smtp::connect_configured() is the only place that asks the selector:

  • stale reconnect (>60s) / restart → init(relays);
  • failed connect/auth/send → on_failure(relays, last_bad).

The core keeps only a transient failed/bad flag; the active relay lives in the
script's current. On a successful live connection the script is not called at
all.

Default script

  • init(): if current is set and still among the relays, return it (fast
    start, no pings); otherwise ping all relays, sort by ascending RTT
    (tie-break by id), pick the fastest.
  • on_failure(relays, bad_relay): add bad_relay to bad, return the next
    fastest; when all relays are known bad (or the volatile cache was lost on
    restart), reset and re-ping.

API

set_relay_selector, get_relay_selector ({ script, state, isDefault }) and
reset_relay_selector_state over JSON-RPC and the C FFI, plus Python client
wrappers. Setting None restores the built-in default. Setting a different
script resets its state; an identical script keeps it. Scripts are validated to
define both functions.

Storage

  • New table relay_selector(id, script, is_default, state).
  • smtp_success and record_success are removed; migration 170 drops the
    table and reshapes relay_selector.

Notes

  • ping() targets TCP port 443 on the relay host, which is the mail port for
    chatmail relays and a reasonable reachability proxy otherwise.
  • A test-only per-context switch keeps ping() offline in the Rust test suite.

Tests

  • Unit tests for the default script (fast start, cold start with no reachable
    relay, failover/demotion) and for set/get/reset.
  • cargo check --workspace, cargo clippy --all-targets clean; SQL schema docs
    updated.

Allow uploading a Rhai script via JSON-RPC/FFI that decides which
configured transport ("relay") is activated for sending. The core does
not cache or participate in the choice: on every new SMTP connection it
calls `select_relay(relays)` and connects to the returned relay.

The script may ping relays (`ping(host)`, TCP:443 RTT, restricted to the
current relay set) and keep persistent state across runs
(`state_get`/`state_set`), stored in the new `relay_selector` table.
Replacing the script resets its state; installing byte-identical source
keeps it. `smtp_success` ordering is bypassed while a selector is active.

Adds JSON-RPC methods set_relay_selector/get_relay_selector/
reset_relay_selector_state, matching FFI functions and Python client
wrappers.

rhai pulls in smartstring, whose Add impls make `String + &String`
ambiguous; the affected call sites now use explicit `&str`.
A previous commit moved the `strip_prefix(' ')` call inside the
`is_quote` branch while adjusting string concatenation, which broke
RFC 3676 space-stuffing for non-quote lines.
Replace the built-in transport ordering (sorted_transports + smtp_success)
with a user-definable Rhai script that decides which relay is activated for
sending. The core no longer picks or caches relays: it asks the script.

Script contract (both functions are required):
- init(relays) -> addr: fast-start returns the remembered `current` relay,
  otherwise pings relays and returns the fastest responder.
- on_failure(relays, bad_relay) -> addr: fails over to the next fastest relay.

The core keeps only a transient "failed relay" flag and calls the selector
on each new connection (a stale reconnect goes through init(), a failure
goes through on_failure()). A built-in default script implementing the
standard ping-and-failover algorithm is used when no user script is
installed.

State lives in the script: persisted `state_get/state_set` (survives
restarts, stored in the `relay_selector` table) and volatile
`mem_get/mem_set` (kept in the process). `ping(host)` checks TCP:443 of a
relay host and returns the RTT.

API: set/get/reset_relay_selector over JSON-RPC and FFI, with `isDefault`.
Migration 170 drops `smtp_success` and reshapes `relay_selector`.
@link2xt

link2xt commented Oct 12, 2026

Copy link
Copy Markdown
Collaborator

Do not use "AI" to create PRs here.
Read https://delta.chat/en/community-standards#collective-maintenance-standards

Not to say that this is not the feature we want, and we don't want a dependency on the script interpreter.

@link2xt link2xt closed this Oct 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants