feat(network): add policy DNS correlation store

[johntmyers]

Summary

Add the dormant policy-DNS resolution and correlation primitives needed by the direct DNS + transparent TCP stack. This PR is intentionally stacked on #2711.

Related Issue

Part of #2712.

Depends on #2711.

Changes

• snapshot DNS-eligible TCP endpoints from one OPA policy generation before upstream resolution
• resolve eligible names through an explicitly pinned trusted DNS server with bounded UDP/TCP and CNAME handling
• reuse the proxy destination controls to filter resolved addresses without consulting ambient DNS or /etc/hosts
• publish expiring synthetic-address mappings that retain all compatible endpoint identities and fail closed on stale generations, wrong ports, expiry, or pool exhaustion
• add bounded DNS wire handlers for one A/AAAA question and OCSF denial/publication events
• emit low-severity OCSF failure events for eligible upstream resolution, stale publication, invalid mappings, pool exhaustion, and store failures with stable reason codes
• remove internal counters that had no production exporter; operator-visible failure diagnostics now use structured OCSF events
• document lowercase DNS normalization as closing the DNS 0x20 case-encoding channel
• document the dormant architecture boundary

Intended Flow

PR 2 builds the policy-DNS decision and correlation system, but does not route real sandbox traffic into it yet.

flowchart TD
    subgraph FutureInput["PR 3 later: sandbox wiring"]
        App["Sandbox application"]
        Resolver["Normal DNS request<br/>db.example.com"]
        Connect["Normal TCP connect<br/>synthetic-IP:5432"]
        Capture["Transparent TCP capture"]
        App --> Resolver
        App --> Connect
        Connect --> Capture
    end

    subgraph PR2["PR 2: policy DNS and resolved-endpoint store"]
        Normalize["Normalize DNS name<br/>case, trailing dot, query type"]
        Eligible{"Explicit protocol: tcp endpoint<br/>matches name and port?"}
        Refused["Return DNS REFUSED<br/>No upstream query"]
        Upstream["Query trusted upstream DNS<br/>with timeout and response limits"]
        Filter["Filter A and AAAA answers<br/>through destination and SSRF controls"]
        Usable{"Any valid addresses?"}
        Failure["Return local DNS failure<br/>Publish no mapping"]
        Allocate["Allocate stable synthetic IP<br/>Never reuse for another endpoint"]
        Publish["Atomically publish correlation"]
        Answer["Return synthetic A or AAAA answer"]

        Lookup["Lookup synthetic IP + port"]
        Mapping{"Exact, live correlation exists?"}
        Deny["Fail closed with rationale"]
        Recheck["Verify current policy generation<br/>and endpoint identity"]
        Handoff["Produce pinned-address<br/>connector request"]

        Resolver -. "Wired in PR 3" .-> Normalize
        Normalize --> Eligible
        Eligible -- "No" --> Refused
        Eligible -- "Yes" --> Upstream
        Upstream --> Filter
        Filter --> Usable
        Usable -- "No" --> Failure
        Usable -- "Yes" --> Allocate
        Allocate --> Publish
        Publish --> Answer

        Capture -. "Wired in PR 3" .-> Lookup
        Lookup --> Mapping
        Mapping -- "Missing, expired, or wrong port" --> Deny
        Mapping -- "Valid" --> Recheck
        Recheck -- "Stale or changed policy" --> Deny
        Recheck -- "Current" --> Handoff
    end

    subgraph Record["Correlation stored by PR 2"]
        Fields["Sandbox identity<br/>Normalized hostname<br/>Synthetic address<br/>Allowed ports<br/>Validated real addresses<br/>Policy generation + endpoint ID<br/>Mapping generation + opaque ID<br/>Created time + expiration"]
    end

    Publish --> Fields

    subgraph FutureRelay["PR 3 later: connection authorization and relay"]
        Process["Resolve and authorize<br/>calling process"]
        Dial["Dial only a pinned real address<br/>No connect-time DNS"]
        Relay["Raw TCP relay"]
        Process --> Dial --> Relay
    end

    Handoff -. "Consumed in PR 3" .-> Process

Loading

Important Separation

1. DNS asks whether the hostname is eligible and which upstream addresses are safe.
2. PR 2 returns a synthetic address and stores a tamper-resistant correlation.
3. DNS does not authorize a process.
4. On connection, PR 3 will recover that correlation, authorize the calling process against the current policy, and dial only a previously validated address.
5. Missing, stale, wrong-port, or changed-policy state fails closed.

PR 2 creates the secure claim check. PR 3 will wire applications and TCP connections through it.

Synthetic IP Pooling in This PR

PR 2 implements an in-memory allocation and correlation store. The pools are injected by a caller; this PR does not select production ranges or wire the store into a running sandbox.

flowchart TD
    Pools["Injected inclusive ranges<br/>IPv4 start–end<br/>IPv6 start–end<br/>Shared lifetime mapping cap"] --> Store

    subgraph Store["One in-memory ResolvedEndpointStore"]
        Key["Stable allocation key<br/>normalized name + address family<br/>+ compatible endpoint-set digest"]
        Existing{"Key already reserved?"}
        Capacity{"Shared lifetime cap and<br/>family cursor available?"}
        Exhausted["PoolExhausted<br/>Publish nothing"]
        Allocate["Reserve next ascending address<br/>Reservation lasts for store lifetime"]
        Refresh["Reuse reserved synthetic IP"]
        Publish["Atomically publish or replace live record<br/>fresh UUID + global mapping generation<br/>policy generation + ports + endpoint IDs<br/>pinned real addresses + expiration"]

        Key --> Existing
        Existing -- "Yes" --> Refresh
        Existing -- "No" --> Capacity
        Capacity -- "No" --> Exhausted
        Capacity -- "Yes" --> Allocate
        Allocate --> Publish
        Refresh --> Publish
    end

    DNS["Eligible DNS result<br/>already resolved and filtered"] --> Key
    Publish --> Answer["Return synthetic A or AAAA answer"]

    Lookup["Lookup<br/>synthetic IP + requested port<br/>+ current policy generation"] --> Checks{"Live record exists?<br/>Not expired?<br/>Generation current?<br/>Port allowed?"}
    Publish --> Checks
    Checks -- "Yes" --> Handoff["Return exact endpoint identities<br/>and pinned real addresses<br/>No connect-time DNS"]
    Checks -- "No" --> Deny["Fail closed<br/>missing / expired / stale / wrong port"]

    Expire["Explicit expire(now)"] --> Tombstone["Remove live record<br/>retain expiry tombstone"]
    Tombstone --> Reservation["Keep key → synthetic IP reservation<br/>Address is not reassigned in this store"]
    Reservation --> Existing

    Restart["Supervisor/store restart"] --> Lost["All in-memory records, reservations,<br/>tombstones, and cursors are lost"]

    subgraph Later["Deferred to PR 3 runtime integration"]
        RuntimePool["Choose collision-safe production ranges<br/>and restart allocation epoch"]
        Ownership["Create one store per sandbox supervisor"]
        Wiring["Wire DNS listener, resolver injection,<br/>transparent capture, and expiry scheduling"]
    end

    Lost -. "Must not reassign cached addresses<br/>when workload survives restart" .-> RuntimePool

Loading

Within one store lifetime, a hostname/family/endpoint identity keeps the same synthetic IP across refreshes, while each refresh replaces the live correlation with a new mapping ID, mapping generation, pinned addresses, and expiration. IPv4 and IPv6 have separate cursors, but they share the lifetime mapping cap and store lock. Expiration removes the live correlation without freeing its reserved address.

PR 3 must establish the runtime ownership contract, select collision-safe pools, prevent cached-address reuse across a supervisor-only restart, and activate the DNS and transparent TCP paths atomically.

Testing

• mise run pre-commit
• cargo test -p openshell-supervisor-network policy_dns --lib — 37 passed
• package Clippy passed
• full pre-commit passes after rebasing onto feat(policy): establish direct TCP egress foundation #2711's cross-platform Podman import compatibility fix

End-to-end tests are deferred to the runtime activation PR because this slice does not expose a listener, inject resolver configuration, or capture transparent TCP traffic.

Non-goals

• no sandbox DNS listener activation
• no resolver configuration injection
• no nftables or transparent TCP capture
• no change to current user-facing network behavior

Checklist

• Commits use Conventional Commit format and include DCO sign-off
• Architecture documentation describes the new boundary
• Unit and crate integration tests cover the new invariants
• No secrets or credentials are logged
• This PR remains dormant until a later stacked runtime PR

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[johntmyers]

gator-agent

PR Review Status

Validation: Project-valid as stage 2 of the policy DNS and transparent TCP series tracked by #2712 and intentionally stacked on #2711. This review is limited to the author delta from the stage-1 head and does not require kernel capture, resolver injection, listener activation, process authorization, relay wiring, or E2E activation deferred to later stages.
Head SHA: df0a60231e4091bb0c8fcaffe8b70f46d01628bd
Base SHA: 2dedd9c4323f4378fac828232bc8c44b3d7a8e5b
Merge base SHA: 2dedd9c4323f4378fac828232bc8c44b3d7a8e5b
Patch ID: fc209f2a6276e94c58709346153192eadabe5c74
Gator payload: 4
Review mode: initial
Previous reviewed SHA: none
Review budget exhausted: no
Maintainer decision required: no

Blocking findings:

• GATOR-df0a6023-01: preserve the trusted host-gateway alias destination constraint in policy DNS.
• GATOR-df0a6023-02: linearize mapping publication against policy-generation advancement so stale work cannot overwrite a newer record.

Carried findings:

• None

Non-blocking suggestions:

• None

Docs: The dormant stage boundary is documented in architecture/sandbox.md; no direct user-facing behavior is activated in this PR, so Fern updates are not required for stage 2.

Next state: gator:in-review

[github-actions]

Label test:e2e applied for 17a43d8. Open the existing run and click Re-run all jobs to execute with the label set. The run will execute the standard E2E suite after building the required gateway and supervisor images once. The matching required CI gate status on this PR will flip green automatically once the run finishes.

[johntmyers]

gator-agent

PR Review Status

Validation: Project-valid as stage 2 of the policy DNS and transparent TCP series tracked by #2712, intentionally stacked on #2711. Review remained limited to the author delta from feat/policy-dns-tcp-foundation; kernel capture, resolver injection, listener activation, runtime wiring, and E2E activation remain deferred to later stages.
Head SHA: 17a43d8ce788eb2dd27b5a2910649864b5f72f1b
Base SHA: 2dedd9c4323f4378fac828232bc8c44b3d7a8e5b
Merge base SHA: 2dedd9c4323f4378fac828232bc8c44b3d7a8e5b
Patch ID: 3b6e97d0b66c3d6db61519b0c85784591b27b4b1
Gator payload: 4
Review mode: follow_up
Previous reviewed SHA: df0a60231e4091bb0c8fcaffe8b70f46d01628bd
Review budget exhausted: no
Maintainer decision required: no

Thanks @johntmyers. I checked your 17a43d8ce update against both prior obligations. The reserved gateway aliases now refuse before upstream resolution when no trusted gateway is configured and pin only the exact configured gateway. Mapping publication now linearizes through the OPA reload mutex, rejects older-generation replacement, and includes the ordered stale-publication regression.

Blocking findings:

• No blocking findings remain.

Carried findings:

• None. GATOR-df0a6023-01 and GATOR-df0a6023-02 are resolved by this head.

Docs: The dormant stage boundary remains documented in architecture/sandbox.md; no direct user-facing behavior is activated in stage 2, so Fern updates are not required here.

Tests: test:e2e is applied. The E2E Label Help bot requires Re-run all jobs for current-head run 31627569680, but the sandbox policy currently blocks that authorized REST action. Narrow policy proposal 2b1449af-9f84-4b7c-923a-2aa561269d1a is pending approval; no E2E run has been dispatched yet.

Next state: gator:blocked
Blocked reason: test_dispatch_required

[github-actions]

🌿 Preview your docs: https://nvidia-preview-pr-2713.docs.buildwithfern.com/openshell

[johntmyers]

gator-agent

PR Review Status

Validation: Project-valid as stage 2 of the policy DNS and transparent TCP series tracked by #2712 and intentionally stacked on #2711. Review remains limited to the author delta from feat/policy-dns-tcp-foundation; kernel capture, resolver injection, listener activation, runtime wiring, process authorization/relay activation, and E2E feature activation remain deferred to later stages.
Head SHA: 7cdac198ec3d7725755e1c47858b485f2a448a70
Base SHA: d73b4c5b25a99c1a309b8bf1b35479d4a0ece1ab
Merge base SHA: d73b4c5b25a99c1a309b8bf1b35479d4a0ece1ab
Patch ID: 01e3e82956f9b5470f68455c39fcd7ee715a1a4e
Gator payload: 4
Review mode: follow_up
Previous reviewed SHA: 17a43d8ce788eb2dd27b5a2910649864b5f72f1b
Review budget exhausted: no
Maintainer decision required: no

Thanks @johntmyers. I compared this head with the previously reviewed stage-2 patch and the updated #2711 stack. The three stage-2 commits are patch-equivalent to the reviewed versions, and the only additional patch is byte-for-byte identical to #2711's current explicit-TCP documentation commit. I treated that as inherited stage-1 context and verified that it does not invalidate the resolved policy-DNS obligations.

Blocking findings:

• No blocking findings remain.

Carried findings:

• None. GATOR-df0a6023-01 and GATOR-df0a6023-02 remain resolved.

Docs: The inherited stage-1 explicit-TCP documentation is present, and stage 2 remains dormant; no kernel capture, resolver injection, listener activation, or runtime capability is required in this PR.

Tests: test:e2e is applied. Current-head Branch E2E Checks run 31821305288 is queued, Branch Checks run 31821304618 is in progress, and Helm Lint run 31821304934 completed successfully.

Next state: gator:watch-pipeline

[johntmyers]

gator-agent

PR Review Status

Validation: Project-valid as stage 2 of the policy DNS and transparent TCP series tracked by #2712 and intentionally stacked on #2711. Review remains limited to the author delta from feat/policy-dns-tcp-foundation; kernel capture, resolver injection, listener activation, runtime wiring, process authorization/relay activation, production pool selection, and E2E feature activation remain deferred to later stages.
Head SHA: 170de834d0c521c7eca3fea9aa4ec2bcca918d3f
Base SHA: 94c3804056d2c8f496100352b5b0bb76667ac829
Merge base SHA: 94c3804056d2c8f496100352b5b0bb76667ac829
Patch ID: 3b6e97d0b66c3d6db61519b0c85784591b27b4b1
Gator payload: 4
Review mode: critical_only
Previous reviewed SHA: 7cdac198ec3d7725755e1c47858b485f2a448a70
Review budget exhausted: yes
Maintainer decision required: no

I checked the stack sync at 170de834. It merges the current #2711 head, produces a tree identical to the previously reviewed head, and leaves the effective stage-2 patch equivalent to the earlier reviewed 17a43d8c patch. The independent critical-only review found no newly introduced Critical defect.

Blocking findings:

• No blocking findings remain.

Carried findings:

• None. GATOR-df0a6023-01 and GATOR-df0a6023-02 remain resolved.

Docs: Stage 2 remains dormant and its architecture boundary is documented; no direct user-facing activation is introduced here, so additional Fern changes are not required for this stage.

Tests: test:e2e is applied. Current-head Branch Checks and Helm Lint are green; current-head E2E run 31821635986 is in progress.

Next state: gator:watch-pipeline

[johntmyers]

gator-agent

PR Review Status

Validation: Project-valid as stage 2 of the policy DNS and transparent TCP series tracked by #2712 and intentionally stacked on #2711. Review remains limited to the author delta from feat/policy-dns-tcp-foundation; kernel capture, resolver injection, listener activation, runtime wiring, process authorization and relay activation, production pool selection, and other stage-3/4 functionality remain deferred.
Head SHA: e563b07153cad705c8960de922e0440df45ca5f5
Base SHA: 22a4de473a88e7d8da23a93ed787a30afbf4ce3d
Merge base SHA: 22a4de473a88e7d8da23a93ed787a30afbf4ce3d
Patch ID: c4cb08765e4c900d641b17719621c395f671edf9
Gator payload: 4
Review mode: already_reviewed
Previous reviewed SHA: 065611dc4964213a960ffec30fd53a137a0f1155
Review budget exhausted: yes
Maintainer decision required: no

The feedback ledger identifies this head as rebase-equivalent to the previously reviewed stage-2 patch, so no duplicate reviewer run was performed. The stage-2 author delta against the updated #2711 stack base is unchanged.

Blocking findings:

• No blocking findings remain.

Carried findings:

• None. GATOR-df0a6023-01 and GATOR-df0a6023-02 remain resolved.

Docs: The dormant stage boundary is documented; this stage does not activate user-facing DNS or transparent TCP behavior, so additional Fern documentation remains deferred to the activation work.

Tests: test:e2e remains applied. Current-head Branch Checks are in progress, while Helm Lint, DCO, required-gate publication, and Branch E2E are queued. Required current-head workflows have been dispatched, so pipeline monitoring is active.

Next state: gator:watch-pipeline

[johntmyers]

gator-agent

PR Review Status

Validation: Project-valid as stage 2 of the policy DNS and transparent TCP series tracked by #2712 and intentionally stacked on #2711. Review remains limited to the author delta from feat/policy-dns-tcp-foundation; kernel capture, resolver injection, listener activation, runtime wiring, process authorization and relay activation, production pool selection, and other stage-3/4 functionality remain deferred.
Head SHA: cbee6e0e2fb751087fc70e35922bab4c3171ffbb
Base SHA: 5157062bebefdc9271432b10829964b093b950f0
Merge base SHA: 5157062bebefdc9271432b10829964b093b950f0
Patch ID: 7882719de27948c5888d4b79571cfe75471759ca
Gator payload: 4
Review mode: critical_only
Previous reviewed SHA: e563b07153cad705c8960de922e0440df45ca5f5
Review budget exhausted: yes
Maintainer decision required: no

I compared this head with the previously reviewed stage-2 patch and the updated #2711 stack. The first four stage-2 commits remain patch-equivalent; the only new author delta preserves trusted resolver answer order while deduplicating pinned addresses. The independent critical-only review found no newly introduced Critical defect.

Blocking findings:

• No blocking findings remain.

Carried findings:

• None. GATOR-df0a6023-01 and GATOR-df0a6023-02 remain resolved.

Docs: The dormant stage boundary remains documented; this stage does not activate user-facing DNS or transparent TCP behavior, so additional Fern documentation remains deferred to activation work.

Tests: test:e2e remains applied. Current-head Branch Checks, Helm Lint, DCO, required-gate publication, and Branch E2E workflows are queued, so pipeline monitoring is active.

Next state: gator:watch-pipeline

[johntmyers]

gator-agent

PR Review Status

Validation: Project-valid as stage 2 of the policy DNS and transparent TCP series tracked by #2712 and intentionally stacked on #2711. Review remains limited to the author delta from feat/policy-dns-tcp-foundation; kernel capture, resolver injection, listener activation, runtime wiring, process authorization and relay activation, production pool selection, and other stage-3/4 functionality remain deferred.
Head SHA: 2aea31316ccf5eaabb9d62f7db5d4d0893a56424
Base SHA: 26d10b1de8a1915e72aae2b87ad93e8986cac16c
Merge base SHA: 26d10b1de8a1915e72aae2b87ad93e8986cac16c
Patch ID: b4367a4f2dbd9e8558a434d645671964150f3504
Gator payload: 4
Review mode: critical_only
Previous reviewed SHA: cbee6e0e2fb751087fc70e35922bab4c3171ffbb
Review budget exhausted: yes
Maintainer decision required: no

I compared the current stage-2 series with the previously reviewed author patch using the stack range-diff. The resolver audit, publication hardening, and answer-order commits remain patch-equivalent. The rebase-only integration adjusts dependency-lock context and the dormant architecture wording for #2711's new explicit-TCP DNS-host validation. The independent critical-only review found no newly introduced Critical defect.

Thanks @pimlock. I checked the answer-order concern you raised and confirmed that the current resolver and pinned connector preserve stable first-seen address order while deduplicating; that fix remains patch-equivalent across this stack rebase.

Blocking findings:

• No blocking findings remain.

Carried findings:

• None. GATOR-df0a6023-01 and GATOR-df0a6023-02 remain resolved.

Docs: The dormant stage boundary remains documented; this stage does not activate user-facing DNS or transparent TCP behavior, so additional Fern documentation remains deferred to activation work.

Tests: test:e2e remains applied. Current-head Branch Checks, Helm Lint, DCO, required-gate publication, and Branch E2E workflows are queued or pending, so pipeline monitoring is active.

Next state: gator:watch-pipeline

[johntmyers]

gator-agent

PR Review Status

Validation: Project-valid as stage 2 of the policy DNS and transparent TCP series tracked by #2712 and intentionally stacked on #2711. Review remains limited to the author delta from feat/policy-dns-tcp-foundation; kernel capture, resolver injection, listener activation, runtime wiring, process authorization and relay activation, production pool selection, and other stage-3/4 functionality remain deferred.
Head SHA: 8a7b2f638f498d9c19489ed45778beed69df294b
Base SHA: 062ce26e11c2aff44b9f21d23afc7c7ac9b1c47a
Merge base SHA: 062ce26e11c2aff44b9f21d23afc7c7ac9b1c47a
Patch ID: b4367a4f2dbd9e8558a434d645671964150f3504
Gator payload: 4
Review mode: already_reviewed
Previous reviewed SHA: 2aea31316ccf5eaabb9d62f7db5d4d0893a56424
Review budget exhausted: yes
Maintainer decision required: no

The feedback ledger identifies the current stage-2 author delta as rebase-equivalent to the previously reviewed patch, so no duplicate reviewer run was performed. The policy-DNS and resolved-endpoint-store implementation remains unchanged across the #2711 stack-base update.

Thanks @pimlock. I confirmed your approval applies to the current head.

Blocking findings:

• No blocking findings remain.

Carried findings:

• None. GATOR-df0a6023-01 and GATOR-df0a6023-02 remain resolved.

Docs: The dormant stage boundary remains documented; this stage does not activate user-facing DNS or transparent TCP behavior, so additional Fern documentation remains deferred to activation work.

Tests: test:e2e remains applied. Current-head Branch Checks, Helm Lint, DCO, and required-gate publication are green. Current-head Branch E2E run 32309574977 is in progress, so pipeline monitoring remains active.

Next state: gator:watch-pipeline