Skip to content

Route apps around the remote VPN without leaving TrackerControl - #730

Merged
kasnder merged 5 commits into
claude/per-app-protection-vpn-routing-42bb96from
claude/per-app-remote-vpn-routing
Aug 20, 2026
Merged

Route apps around the remote VPN without leaving TrackerControl#730
kasnder merged 5 commits into
claude/per-app-protection-vpn-routing-42bb96from
claude/per-app-remote-vpn-routing

Conversation

@kasnder

@kasnder kasnder commented Aug 19, 2026

Copy link
Copy Markdown
Member

Remote WireGuard egress applied globally: the only way to keep an app
off the remote tunnel was "Exclude from VPN", which also dropped local
monitoring and blocking. Whether an app is filtered and whether it is
forwarded through the remote VPN are independent choices, and #723 is
what happens when they are collapsed into one.

Add a global "Route through remote VPN" mode (All apps / Only selected
apps) plus a per-app override, orthogonal to the protection state. The
default is All apps, which is byte-identical to the behaviour before
this existed — unknown UIDs and system traffic follow that same default,
so nothing changes for anyone who does not opt in.

The fork is one expression in ip.c, now route_wants_tunnel(): pure, so
the decision table can be read straight through, with the caller still
deciding what to do when the tunnel is down, because only the write
attempt can say whether it is. RemoteRoutingLogic mirrors it on the Java
side so both halves are JVM-testable; the stacked follow-up #737 moves
the decision itself into wgbridge-rs/src/policy.rs, where CI already
runs cargo test, and reduces the C to a caller.

Java pushes down only the UIDs whose routing differs from the global
default, through a new jni_wireguard_route. Pushing the whole tunnelled
set instead looked reasonable and was not: in the default mode every
applied app is tunnelled, so the set held every installed app and was
indistinguishable from a heavily-overridden one. The packet path could
no longer tell that nobody had opted in, and paid a mutex and a
session-table walk per packet to rediscover it — for every user,
WireGuard enabled or not. With overrides the set is empty until someone
sets one, so the common case reads two lock-free atomics and stops.
Absence from the set means "follow the default", which is what makes the
default mode identical to the behaviour before this existed.

Resolving the UID at the fork needed more care than expected. Packet 2+
of an established flow arrives with uid == -1, since the expensive
lookup is deliberately skipped for existing UDP sessions and non-SYN
TCP, and get_uid_q is never re-run here: it is a binder call. The
session table answers for direct flows, but not for tunnelled ones —
those are handed to WireGuard and return before handle_tcp/handle_udp,
so no ng_session is ever created for them. Falling back to the global
default there is wrong in exactly the mode this feature exists for: in
"Only selected apps" the default is direct, so the rest of a tunnelled
TCP flow was routed direct and handle_tcp, finding no session for a
non-SYN segment, answered it with an RST. A per-flow verdict cache,
keyed on the 5-tuple and invalidated by generation on every reload,
keeps a flow on the answer its first packet was given; the session table
remains the fallback behind it.

DNS is the crux, and a privacy decision rather than plumbing. Forcing
every query into the tunnel is what stops the resolver seeing the user's
physical network, but it also breaks a directly-routed app whenever the
tunnel drops. Direct apps' DNS is therefore redirected to the system
resolver through the existing Allowed(raddr, rport) mechanism, which is
transparent — replies are rebuilt from the session's original addresses.
That costs a UID lookup, a JNI upcall and a real UDP session per query,
because port 53 is the only UDP traffic that skips all three today, so
it is only enabled once some app actually is routed direct — a property
of the resolved rules, not of the mode, since an override sends an app
direct in either one. Users of the default keep the current zero-cost
DNS path. The packet path learns this through its own flag rather than
by reading fwd53, which is also set by an unrelated port-53 forward:
Secure DNS installs one whenever the WireGuard config carries no DNS
line, and borrowing it silently switched off the rule that every
resolver query takes the tunnel. The reduced DNS privacy and the fact
that a direct app sees this device's real address are both stated in the
control's own copy.

Routes are the remaining constraint: they belong to the single tun every
app shares, so a narrower AllowedIPs would shrink them for tunnelled
apps too. gotatun silently drops packets matching no peer's AllowedIPs
(device/mod.rs), so this would blackhole rather than fail visibly. v1
therefore gates the control on a default-route tunnel and explains why
when it is unavailable.

Also adds the new wg_route store to uninstall cleanup and to XML
export/import, which the other per-package stores already had, and
corrects the wgbridge-rs README item claiming app DNS stays on the local
NetGuard path — ip.c has forced it into WireGuard since the DNS change.

Reconciling the native fork with its Java mirror also surfaced three
bugs that predate this work and are not fixed here, all one root cause:
handle_ip diverts to WireGuard before the DNS and session machinery
runs, so everything that machinery provides disappears while the tunnel
is on. Filed as #734, #735 and #736, independent of this stack.

Not device-tested yet.

Co-Authored-By: Claude Opus 5 noreply@anthropic.com


Stack created with GitHub Stacks CLIGive Feedback 💬

@kasnder

kasnder commented Aug 19, 2026

Copy link
Copy Markdown
Member Author

Pushed 1b376f02 on top — four bugs found while reconciling this with the Java mirror. It is additive, not a history rewrite, so earlier review comments still line up.

Push the exceptions, not the whole tunnelled set. mapUidTunnel held every applied app in the default mode, so the native count > 0 test for "is per-app routing configured at all" was always true and d7b0b69f's lock-free fast path never engaged. Every packet still paid a mutex and an O(active sessions) walk — for every user, WireGuard enabled or not, which means d7b0b69f did not actually fix the reported Brave DNS degradation. Java now pushes only the UIDs whose routing differs from the global default; that set is empty until someone sets an override.

Unlisted UIDs now follow the default. is_tunnel_uid treated absence from a non-empty set as "direct", so in the default mode a resolved UID with no rule of its own — an app installed since the last reload, a transient or shared UID — left outside the tunnel. That is the opposite of what the adjacent comment promised, and it was reachable in the shipped default rather than narrowly. Fixed in both the C and the Java mirror; no known-UID set needs pushing after all.

Direct-DNS redirection has its own flag. It was reading args->fwd53, which is also set by an unrelated port-53 forward — Secure DNS installs one whenever the WireGuard config carries no DNS = line. In that combination the rule that every resolver query takes the tunnel silently stopped applying.

A flow keeps its first verdict. A tunnelled packet is handed to WireGuard and returns before handle_tcp/handle_udp, so no ng_session is ever created for it; packet 2+ then arrives with uid == -1 and nothing to resolve it from. In "Only selected apps" the default is direct, so the rest of a tunnelled TCP flow was routed direct and handle_tcp, finding no session for a non-SYN segment, answered it with an RST — killing the connection in the mode this feature exists for. A 1024-entry per-flow verdict cache, keyed on the 5-tuple and generation-invalidated on reload, fixes it. Two smaller ones alongside: SNI research mode let a 443 SYN through without resolving its UID, so a tunnelled app's whole 443 flow inherited the global default; and the session-table fallback ignored ICMP.


Stacked follow-up: #737 moves the decision into wgbridge-rs/src/policy.rs, deletes route.c, and covers the decision table with cargo test in CI. Review this PR first.

Note the body above is now stale in places — it describes route_decide() (renamed in d7b0b69f), a "tunnelled UID set" (now an override set), and fwd53 being gated on the mode (changed in da3962e7 to gate on whether any app is actually routed direct). Happy to rewrite it.

Reconciling the two implementations also surfaced three master-level bugs, filed separately and independent of this stack: #734, #735, #736.

@kasnder
kasnder marked this pull request as ready for review August 20, 2026 10:28
kasnder and others added 5 commits August 20, 2026 12:52
Remote WireGuard egress applied globally: the only way to keep an app
off the remote tunnel was "Exclude from VPN", which also dropped local
monitoring and blocking. Whether an app is filtered and whether it is
forwarded through the remote VPN are independent choices, and #723 is
what happens when they are collapsed into one.

Add a global "Route through remote VPN" mode (All apps / Only selected
apps) plus a per-app override, orthogonal to the protection state. The
default is All apps, which is byte-identical to the behaviour before
this existed — unknown UIDs and system traffic follow that same default,
so nothing changes for anyone who does not opt in.

The fork is one expression in ip.c. It now runs through route_decide(),
which is deliberately free of I/O so the decision table can be read
straight through, and is mirrored by RemoteRoutingLogic on the Java side
so both halves are JVM-testable — the native path has no test harness at
all. Java pushes the tunnelled UID set down through a new
jni_wireguard_route as a sorted array; the packet path only reads it,
under a lock, because Java writes it during a reload.

Resolving the UID at the fork needed care. Packet 2+ of an established
flow arrives with uid == -1, since the expensive lookup is deliberately
skipped for existing UDP sessions and non-SYN TCP. Defaulting there
would divert an already-NATted flow mid-stream, so the session table
answers instead — it is authoritative and free. get_uid_q is never
called on this path: it is a binder call.

DNS is the crux, and a privacy decision rather than plumbing. Forcing
every query into the tunnel is what stops the resolver seeing the user's
physical network, but it also breaks a directly-routed app whenever the
tunnel drops. Direct apps' DNS is therefore redirected to the system
resolver through the existing Allowed(raddr, rport) mechanism, which is
transparent — replies are rebuilt from the session's original addresses.
That costs a UID lookup, a JNI upcall and a real UDP session per query,
because port 53 is the only UDP traffic that skips all three today, so
fwd53 is only enabled in "Only selected apps" mode. Users of the default
keep the current zero-cost DNS path. The reduced DNS privacy and the
fact that a direct app sees this device's real address are both stated
in the control's own copy.

Routes are the remaining constraint: they belong to the single tun every
app shares, so a narrower AllowedIPs would shrink them for tunnelled
apps too. gotatun silently drops packets matching no peer's AllowedIPs
(device/mod.rs), so this would blackhole rather than fail visibly. v1
therefore gates the control on a default-route tunnel and explains why
when it is unavailable.

Also adds the new wg_route store to uninstall cleanup and to XML
export/import, which the other per-package stores already had, and
corrects the wgbridge-rs README item claiming app DNS stays on the local
NetGuard path — ip.c has forced it into WireGuard since the DNS change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Device testing found the DNS redirect never engaged in the configuration
most people will actually reach first: leave the mode at "All apps" and
flip one app to direct. Brave then egressed directly — confirmed against
am.i.mullvad.net — while its DNS still went through the tunnel, which is
exactly the split the redirect exists to prevent, and made the control's
own copy ("its DNS queries go to your network's resolver") false.

The cause was gating fwd53 on the mode. Whether an app is routed direct
is a property of the resolved rules: a per-app override sends it direct
in either mode. Gate on that instead.

Fixing it surfaced a second bug on the same path, reachable only once
fwd53 is on: the redirect tested !mapUidTunnel.contains(uid), so every
UID with no rule of its own — unknown and system traffic — looked "not
tunnelled" and would have had its DNS redirected out of the tunnel. That
is the opposite of what the global default promises. Both sides now go
through RemoteRoutingLogic.routesDirect, which mirrors the native
is_tunnel_uid: in the tunnel set means tunnel, a known app outside it
means direct, and anything else follows the default.

Verified on a Pixel 8 against a full-tunnel Mullvad config: Brave direct
resolves and loads over the system resolver while reporting the cellular
IP, switching it back reports the Mullvad exit, and the VPN stays
validated with fwd53 on.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Reported symptom: DNS in Brave degrades after a while, even with Brave
routed through WireGuard and no per-app override configured — a path the
feature was supposed to leave alone.

It did not leave it alone. The routing fork ran, for every packet:

  - get_session_uid(), a walk of the entire session table, taken whenever
    uid < 0. That is not the rare case: the expensive UID lookup upstream
    is deliberately skipped for existing UDP sessions and non-SYN TCP, so
    most packets of every established flow arrive with uid == -1. The
    table grows toward SESSION_MAX under load, so the per-packet cost
    grows with it — which is exactly the "after a while" shape.
  - is_tunnel_uid(), taking a mutex.
  - wireguard_active(), taking the WireGuard outbound mutex, immediately
    before write_wireguard_packet took the same mutex again.

Before this feature the fork was a single boolean expression. All of that
work was also pointless in the default configuration: with no per-app
override the UID set is empty, so every UID resolves to the same global
default and the answer never depended on the UID at all.

So don't compute it. route_uid_relevant() and route_default_is_tunnel()
are lock-free atomic reads; the UID lookup and the lock now happen only
once an override actually exists. route_decide is replaced by the pure
route_wants_tunnel, and wireguard_active is gone: the control flow is
back to the original shape where the write attempt itself reports whether
the tunnel took the packet, so there is one lock per packet again, not
two.

Verified on a Pixel 8: with no override Brave reports the Mullvad exit as
before, and setting the override still sends it out over the cellular IP,
so the opt-in path still activates the slow path.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Four bugs found while reconciling the native and Java routing paths.

Push only the exceptions, not the whole tunnelled set. mapUidTunnel held
every applied app in the default mode, so the native `count > 0` test for
"is per-app routing configured at all" was always true and the lock-free
fast path added in the previous commit never engaged. Every packet still
paid a mutex and an O(active sessions) walk — for every user, WireGuard
enabled or not. Java now pushes the UIDs whose routing differs from the
global default, which is empty until someone sets an override.

That also fixes what an unlisted UID does. is_tunnel_uid treated absence
from a non-empty set as "direct", so in the default mode a resolved UID
with no rule of its own — an app installed since the last reload, a
transient or shared UID — left outside the tunnel, which is the opposite
of what the adjacent comment promised. Absence now means "follow the
default", in both the C and its Java mirror.

Give direct-DNS redirection its own flag. It was reading args->fwd53,
which is also set by an unrelated port-53 forward — Secure DNS installs
one whenever the WireGuard config carries no DNS line of its own. In that
combination the rule that every resolver query takes the tunnel silently
stopped applying, and DNS followed the per-app decision instead.

Remember a flow's verdict. A tunnelled packet is handed to WireGuard and
returns before handle_tcp/handle_udp, so no ng_session is ever created
for it; packet 2+ then arrives with uid == -1 and nothing to resolve it
from. In "selected" mode the default is direct, so the rest of a
tunnelled TCP flow was routed direct and handle_tcp, finding no session
for a non-SYN segment, answered it with an RST — killing the connection
in the mode the feature exists for. A 1024-entry per-flow verdict cache,
keyed on the 5-tuple and invalidated by generation on every reload, keeps
the flow on the answer its first packet was given.

Two smaller ones alongside it: SNI research mode let a 443 SYN through
without resolving its UID, so a tunnelled app's whole 443 flow inherited
the global default — now resolved once an override exists; and the
session-table fallback ignored ICMP.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@kasnder
kasnder force-pushed the claude/per-app-remote-vpn-routing branch from 1b376f0 to ed5baef Compare August 20, 2026 10:56
@kasnder
kasnder merged commit b8cec6c into master Aug 20, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant