feat(dgw): keep agent tunnel data in a shared gateway.db

[irvingouj@Devolutions (irvingoujAtDevolution)]

Moves the agent tunnel tables into gateway.db, the shared database we talked about on #1958, so the next feature that needs a table doesn't add another .db file. The job queue and the traffic audit keep their own files.

What changes

• New gateway-db crate. It only owns the file: it opens gateway.db, applies the PRAGMAs and hands each feature its own connection (GatewayDb::connect). It doesn't depend on any feature crate and never touches PRAGMA user_version.
• agent-tunnel-libsql stays the agent tunnel's adapter (same layout as job-queue/job-queue-libsql). It runs its own migrations on the connection it gets: its first migration creates the prefixed agent_tunnel_* tables directly, and its schema version is the schema_version row of agent_tunnel_metadata.
• First start after 2026.3 (agent_tunnel_libsql::import_2026_3_database, called from service.rs before GatewayDb::open): when agent_tunnel.db exists and gateway.db doesn't, the old file's user_version decides:
  • 1: copy it with VACUUM INTO, convert the copy (journal_mode = DELETE, add the table prefixes, write the version row, user_version = 0), flush it and rename it into place, then remove the old files.
  • 0: a first start that failed before creating the tables, so the empty file is removed.
  • anything else: refused, Gateway doesn't start.
  • If it stops halfway, the next start redoes it, removing the copy's -journal, -wal and -shm first.
• crates/gateway-db/AGENTS.md tells agents to keep gateway-db free of feature crates.

Behavioral changes (please confirm)

• The agent tunnel CA is loaded (never generated) once agent_tunnel_metadata holds a CA fingerprint. Before, this was decided by whether agent_tunnel.db existed, which no longer works now that the file is shared. A DB that has Agent data (accepted, deleted or enrollment rows) but no CA row is refused.
• gateway.db is created on every start, agent tunnel on or off. The agent tunnel tables are only created when the tunnel is on.
• Downgrading to 2026.3 after this means Agents re-enroll: 2026.3 only looks for agent_tunnel.db. If they upgrade again, the leftover agent_tunnel.db is ignored with a warning.

Why this shape

• One adapter crate per feature, gateway-db only owns the file: the stores build in parallel, and a change to one feature only rebuilds its adapter.
• A schema version per feature instead of one user_version: no single crate owns every table, and the table prefix is the compatibility boundary.
• A connection per feature, not a shared one: a cloned libsql Connection shares one SQLite handle, so features would land inside each other's transactions.

Where to look hardest

• crates/agent-tunnel-libsql/src/upgrade.rs: the 2026.3 conversion
• migrate and bind_ca in crates/agent-tunnel-libsql/src/lib.rs, and the CA load/generate check in service.rs

Tested

agent-tunnel-libsql 23/23 (incl. 7 upgrade tests: a 2026.3 file converted then authorizing its Agent, renamed in rollback journal mode with no journals left, Agents still in the 2026.3 WAL, an interrupted move with a stale journal, an existing gateway.db kept, an empty 2026.3 file, version 0 with tables and an unknown version refused) and gateway-db 3/3. The testsuite builds, but its agent tunnel tests weren't re-run after the split. The MSI was run on a lab Windows Gateway: fresh install, upgrade from 2026.3.0 with enrolled Agents, and restart. The import worked and the Agents reconnected without re-enrolling, but the first start after the upgrade can crash because of a libsql double close (#2070).

On Windows, the libsql test binaries sometimes crash with STATUS_ACCESS_VIOLATION when tests open databases in parallel. It also happens before this change (2 of 40 runs on the old gateway-db), and it's the same crash CI hit in tests/traffic, likely #2070.

🤖 Generated with Claude Code

[github-actions]

Let maintainers know that an action is required on their side

• Add the label release-required Please cut a new release (Devolutions Gateway, Devolutions Agent, Jetsocat, PowerShell module) when you request a maintainer to cut a new release (Devolutions Gateway, Devolutions Agent, Jetsocat, PowerShell module)

• Add the label release-blocker Follow-up is required before cutting a new release if a follow-up is required before cutting a new release

• Add the label publish-required Please publish libraries (`Devolutions.Gateway.Utils`, OpenAPI clients, etc) when you request a maintainer to publish libraries (Devolutions.Gateway.Utils, OpenAPI clients, etc.)

• Add the label publish-blocker Follow-up is required before publishing libraries if a follow-up is required before publishing libraries

[Copilot]

Copilot review overview

🟡 Changes recommended

Legacy data is deleted before the copied database is durably synchronized, risking corruption during power loss.

Review effort: Balanced
Findings: 1

Open (1)

• Ensure VACUUM INTO copy and rename are durable before deleting source · New

What changed in this PR

Moves Agent tunnel persistence into the shared gateway.db, centralizing schema migrations while retaining a dedicated connection per feature.

Changes:

• Adds gateway-db with migration and legacy-database adoption support.
• Migrates Agent tunnel tables and queries to prefixed shared tables.
• Updates Gateway startup and tests for shared database connections.

File	Description
Cargo.lock	Registers new dependencies.
crates/​gateway-db/​Cargo.toml	Defines the database crate.
crates/​gateway-db/​src/​lib.rs	Implements connections, migrations, and legacy adoption.
crates/​gateway-db/​migrations/​01_agent_tunnel.sql	Creates the legacy-compatible schema.
crates/​gateway-db/​migrations/​02_prefix_agent_tunnel_tables.sql	Prefixes Agent tunnel tables.
crates/​agent-tunnel-libsql/​Cargo.toml	Adds test dependencies.
crates/​agent-tunnel-libsql/​src/​lib.rs	Uses shared connections and prefixed tables.
devolutions-gateway/​Cargo.toml	Adds the database dependency.
devolutions-gateway/​src/​service.rs	Opens the shared database and selects the CA.
testsuite/​Cargo.toml	Adds the test dependency.
testsuite/​tests/​agent_tunnel/​common.rs	Uses an in-memory shared database.

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Benoît Cortier (CBenoit)]

Thanks, Irving!

gateway.db, the agent_tunnel_ table prefixes, and moving the old agent_tunnel.db, all of that sounds good to me, let’s keep it.

One thing to change before merging: please keep the crate split we agreed on in Slack: move crates/gateway-db/src/agent_tunnel.rs back into agent-tunnel-libsql. LibSqlAgentAuthorizationStore::open already takes a Connection, so it doesn't need much from gateway-db.

Two reasons:

• Separation of concerns: agent-tunnel defines the operations and agent-tunnel-libsql implements them with libsql. That's the same layout as job-queue/job-queue-libsql and traffic-audit/traffic-audit-libsql. gateway-db should only own the shared file.
• Compilation speed: if gateway-db holds every feature's store, it depends on every feature crate. It can't start building until all of them are built, and a change to any one of them rebuilds every store. With separate adapters, the stores build in parallel and a change only rebuilds the affected adapter.

Stemming from that, I want to introduce a dependency rule: gateway-db must never depend on crates that implement feature behavior, such as agent-tunnel. Adapters may depend on gateway-db if they need to, but never the other way around. Please add this to the invariants in crates/gateway-db/INTENT.md and/or in a local crates/gateway-db/AGENTS.md file.

About schema versions and the 2026.3.0 .db file: once the store is back in its own crate, one PRAGMA user_version for the whole file stops working, because no single crate owns every migration. Let's drop it:

• gateway-db doesn't read or write PRAGMA user_version. It opens the file, applies the PRAGMAs, and hands out a connection per feature.
• agent-tunnel-libsql keeps its own migrations and tracks its own schema version within its tables, for example as a row in agent_tunnel_metadata. Its first migration creates the prefixed tables directly.
• The upgrade from 2026.3.0 is based on the file name: only 2026.3.0 wrote agent_tunnel.db, and it only ever had one schema. When agent_tunnel.db exists and gateway.db doesn't, copy it, convert the copy (add the table prefixes, write the agent tunnel's version row, set user_version to 0), then flush it and rename it into place as you do today. gateway.db then only ever exists in the new layout, and the store needs no code for old databases.

Things to watch in the conversion:

• Check the old file's user_version first. 1 means convert. 0 means a first start that failed before creating the tables, so there's nothing to import. Refuse anything else.
• Write to the copy with journal_mode = DELETE (or checkpoint and close the connection), so the changes are in the file itself before the rename and not in a -wal file left behind.
• When redoing an interrupted move, also delete the copy's -journal, -wal and -shm files. A leftover journal would be replayed into the new copy.
• This code writes agent tunnel tables, so it belongs in agent-tunnel-libsql. Call it from service.rs before GatewayDb::open, because once gateway.db exists, the old file is ignored. It can use the flush and rename helpers from gateway-db.

Each feature's version then covers only its own prefixed tables, which is what we said in Slack: the table prefix is the compatibility boundary.

Last, I would like to record the long-term idea about how we will deal with migrations in the futer, please, add a not in the INTENT.md file (but do not implement in this PR):

Later, `gateway-db` could take migrations from each adapter.
An adapter would pass its prefix and its list of migrations, and `gateway-db` would run them and track a version per prefix.

[irvingouj@Devolutions (irvingoujAtDevolution)]

Updated as instructed

---

Irving Wrote, LLM Audited.

[Benoît Cortier (CBenoit)]

Thank you,

The PR description states that the agent tunnel tests were not re-run after the split, and the real binary wasn’t run. Can you verify that before merging, please? Run for fresh install, upgrade from 2026.3.0 and restart.

The current behavior is: the import runs on every start, even with the agent tunnel off.
The side effect: an unreadable or unexpected agent_tunnel.db now stops the Gateway even when the tunnel is disabled.
I think we could legitimately remove this when the agent tunnel is disabled, how does that sound? Would you prefer the current logic?

Approving now for velocity

[irvingouj@Devolutions (irvingoujAtDevolution)]

Ran it for real on a lab Windows Gateway with the CI MSI: headless 2026.3.0 install with enrolled Agents, upgrade to this build, restart, then a fresh install.
agent_tunnel.db moves into gateway.db (all 58 Agents kept), the CA is reused and the Agents reconnect without re-enrolling.

pr2039-gateway-db-e2e.mp4

Found one problem on the way: the first start after the upgrade can crash because of a libsql double close, tracked in #2070.
For the tunnel-off question, #2071 keeps a bad agent_tunnel.db from stopping the Gateway when the tunnel is off.

Update: closed #2071. It's wrong either way: skipping or tolerating the import while the tunnel is off can leave the old data behind for good. So we keep the current logic: if the import fails, the Gateway doesn't start, the data stays intact, and it gets fixed by hand if needed.

Note

Human-tuned, LLM-assisted content.