feat(policy): establish direct TCP egress foundation

[johntmyers]

Summary

Establish the policy and proxy foundations for RFC 0005's direct DNS and transparent TCP work without enabling either runtime path yet. Explicit protocol: tcp is accepted as the current L4 host/port behavior, while proxy authorization now returns one generation-consistent decision snapshot that later adapters can consume safely.

Related Issue

Implements the first delivery slice of the accepted RFC in #2155. The RFC was merged without a separate tracking issue.

Changes

• Accept explicit protocol: tcp in policy YAML, profile validation, ambiguity checks, Rego evaluation, and incremental policy update parsing
• Keep explicit TCP equivalent to omitted protocol today and reject L7-only fields on TCP endpoints
• Replace split proxy policy lookups with one authoritative, generation-consistent authorization snapshot containing the action, matched policy, full matched endpoint metadata (including credential_binding), exact-host signal, and policy generation
• Fail closed when that combined authorization evaluation errors; missing optional metadata within a successful authorization snapshot still uses conservative defaults instead of re-querying policy
• Hydrate TLS, L7, destination constraints, and relay planning from that single snapshot
• Add dormant transparent-TCP transport and pinned-address destination-validation primitives for later stack PRs
• Document current behavior and update policy-generation/CLI agent guidance

Testing

• mise run pre-commit passes
• mise run docs passes
• mise run test passes
• Unit tests added/updated
• Regression proves an authoritative OPA evaluation error denies egress and exposes no fallback endpoint metadata
• Replaced poisoned-lock tests for the retired multi-query behavior with conservative-default tests for successful snapshots that omit optional metadata
• E2E tests added/updated (not applicable: DNS interception and transparent TCP routing remain disabled)

Checklist

• Follows Conventional Commits
• Commits are signed off (DCO)
• Architecture docs updated (if applicable)

[github-actions]

🌿 Preview your docs: https://nvidia-preview-pr-2711.docs.buildwithfern.com/openshell

[johntmyers]

gator-agent

PR Review Status

Validation: Project-valid as the explicitly authorized first delivery slice of accepted RFC #2155 and the authoritative policy-DNS/transparent-TCP series plan. This review evaluates only stage-1 responsibilities and does not require DNS, mapping-store, capture, or native transparent-TCP functionality deferred to later PRs.
Head SHA: c9ee490d4c2ea7ae7e93a1eee723425022414fed
Base SHA: 0310cbed6c809e8950fc513d0a25c2ec03946198
Merge base SHA: 0310cbed6c809e8950fc513d0a25c2ec03946198
Patch ID: 64c0cd43a0b601c291aab47f2c53c7bd1a8217f6
Gator payload: 4
Review mode: initial
Previous reviewed SHA: none
Review budget exhausted: no
Maintainer decision required: no

Blocking findings:

• GATOR-c9ee490d-01: The authoritative authorization result and ambiguity model must preserve explicit TCP eligibility and a generation-stable matched endpoint identity, even though explicit TCP and omitted protocol remain equivalent for CONNECT.
• GATOR-c9ee490d-02: A pinned resolved-address destination mode must construct an unopened connector from policy-DNS-pinned addresses without performing connect-time DNS resolution.
• GATOR-c9ee490d-03: Explicit protocol: tcp must reject fields that promise L7-only enforcement or transformation rather than accepting and silently ignoring them.
• GATOR-c9ee490d-04: Published policy and CLI reference documentation must list protocol: tcp and accurately describe its current L4-only constraints when the public parser accepts it.

Carried findings:

• None

Non-blocking suggestions:

• None

Docs: Missing canonical Fern updates for the newly accepted public protocol: tcp policy and CLI syntax.

Next state: gator:in-review

[johntmyers]

gator-agent

Re-check After Author Update

Thanks @johntmyers. I reviewed the new implementation commit and your note that the Fern documentation is being deferred to a separate branch. I checked the stage-1 code obligations against the authoritative policy-DNS/transparent-TCP series plan and checked the canonical published policy and CLI pages.

Validation: This remains project-valid as PR 1 of the authorized series. DNS, mapping-store, capture, and native transparent-TCP runtime functionality remain correctly deferred to later PRs.
Head SHA: 2dedd9c4323f4378fac828232bc8c44b3d7a8e5b
Base SHA: 0310cbed6c809e8950fc513d0a25c2ec03946198
Merge base SHA: 0310cbed6c809e8950fc513d0a25c2ec03946198
Patch ID: 544beca42375b85776ff2da0101389b02c7d4998
Gator payload: 4
Review mode: follow_up
Previous reviewed SHA: c9ee490d4c2ea7ae7e93a1eee723425022414fed
Review budget exhausted: no
Maintainer decision required: no

Resolved findings:

• GATOR-c9ee490d-01: the authoritative result now preserves matched endpoint identity and explicit TCP eligibility, with ambiguity coverage.
• GATOR-c9ee490d-02: the pinned-address validation plan now constructs an unopened connector without DNS resolution and is covered by deterministic tests.
• GATOR-c9ee490d-03: TCP activation now rejects the represented L7-only field family across policy and provider-profile paths.

Blocking findings:

• GATOR-c9ee490d-04 remains open. docs/reference/policy-schema.mdx still instructs users to omit the protocol for TCP passthrough, while docs/sandboxes/policies.mdx still omits tcp from the values accepted by policy update. Because this PR directly ships the public policy and CLI syntax and the stage-1 plan requires documenting its current L4-only semantics, a separate in-flight docs branch does not make this PR's published contract accurate and is not a maintainer waiver. Please update those two existing Fern pages here, or obtain an explicit maintainer waiver for deferring them. No navigation change is needed.

Carried findings:

• GATOR-c9ee490d-04: canonical Fern policy and CLI documentation must describe explicit protocol: tcp and its current L4-only constraints.

Docs: Missing for the direct public policy/CLI syntax change.

Next state: gator:in-review

[johntmyers]

@ gator, as mentioned in the reply to your comment we will defer the docs. #2711 (comment)

[github-actions]

Label test:e2e applied for 2dedd9c. Open the existing run and click Re-run all jobs to execute with the label set. The run will execute the standard E2E suite after building the required gateway and supervisor images once. The matching required CI gate status on this PR will flip green automatically once the run finishes.

[johntmyers]

gator-agent

Maintainer Review Nudge

This PR has been in gator:approval-needed for more than 48 business hours with no maintainer approval.

@NVIDIA/openshell-maintainers @NVIDIA/openshell-codeowners @mrunalp @maxamillion @derekwaynecarr, can someone review and either approve, request changes, or close this out?

[johntmyers]

gator-agent

Author Follow-Up Nudge

This PR has been in gator:in-review for more than 48 business hours with unresolved review feedback.

@johntmyers, please either update docs/reference/policy-schema.mdx and docs/sandboxes/policies.mdx for the new protocol: tcp contract, or obtain an explicit maintainer waiver for deferring those canonical docs. If the standalone docs PR is the intended resolution, please link it so maintainers can make that decision.

[johntmyers]

gator-agent

Maintainer Approval Needed

Gator validation and PR monitoring are complete for current head 2e672a075ad7db44e3e903933c803936b32c0814.

Head SHA: 2e672a075ad7db44e3e903933c803936b32c0814
Base SHA: 877ddbacb4b915fa1d5bc6f02302d9a5c13a18e1
Merge base SHA: 877ddbacb4b915fa1d5bc6f02302d9a5c13a18e1
Patch ID: 224d28db8efbec52eb27e36c674113f45e2e8807
Gator payload: 4

Validation: Project-valid as stage 1 of the accepted policy-DNS/transparent-TCP series. This stage establishes authoritative generation-consistent authorization, stable endpoint identity, the explicit protocol: tcp policy contract, future transport/correlation interfaces, and the unused pinned-address connector foundation. Policy DNS, mapping storage, kernel capture, resolver injection, and native transparent TCP remain correctly deferred to later PRs.

Review: The feedback ledger identifies this head as rebase-equivalent to reviewed head ed10d1cbb82267423b660cc9a0445fb48b0e1c67, so no duplicate reviewer run was performed. All four prior findings (GATOR-c9ee490d-01 through GATOR-c9ee490d-04) remain resolved, with no unresolved review feedback.

Docs: The canonical Fern policy pages document explicit protocol: tcp, its current L4 equivalence to omitted protocol, and its rejected L7-only fields.

Checks: DCO, OpenShell / Branch Checks, OpenShell / Helm Lint, and all required current-head gates are green.

E2E: test:e2e is applied and OpenShell / E2E is green for this head.

Human maintainer approval is now required.

[johntmyers]

gator-agent

PR Review Status

Validation: Project-valid as stage 1 of the accepted policy-DNS/transparent-TCP series. This head preserves the authoritative generation-consistent authorization boundary, stable endpoint identity, explicit protocol: tcp contract, future transport/correlation interfaces, and pinned-address connector foundation. Policy DNS, mapping storage, kernel capture, resolver injection, and native transparent TCP remain correctly deferred to later PRs.
Head SHA: 22a4de473a88e7d8da23a93ed787a30afbf4ce3d
Base SHA: 2eb0880a006737370616c93c4c6dbfafabc8ac88
Merge base SHA: 2eb0880a006737370616c93c4c6dbfafabc8ac88
Patch ID: 336a16be144a4e5982a0e8533712957aab90b200
Gator payload: 4
Review mode: critical_only
Previous reviewed SHA: ed10d1cbb82267423b660cc9a0445fb48b0e1c67
Review budget exhausted: yes
Maintainer decision required: no

Blocking findings:

• No newly introduced Critical findings in the current author-only rebase delta.

Carried findings:

• None. GATOR-c9ee490d-01 through GATOR-c9ee490d-04 remain resolved.

Docs: The canonical Fern policy pages continue to document explicit protocol: tcp, its current L4 equivalence to omitted protocol, and its rejected L7-only fields.

Tests: test:e2e remains applied. DCO is green, and the required current-head Branch Checks, Helm Lint, and E2E workflows are queued.

Next state: gator:watch-pipeline

[johntmyers]

gator-agent

Maintainer Convergence Decision

Thanks @johntmyers. I checked the current head after your response to @pimlock's request to require a non-empty host and a valid concrete port for explicit protocol: tcp endpoints. The independent critical-only review found no newly introduced Critical defect, and the four earlier stage-1 findings remain resolved.

Head SHA: 5157062bebefdc9271432b10829964b093b950f0
Base SHA: 2eb0880a006737370616c93c4c6dbfafabc8ac88
Merge base SHA: 2eb0880a006737370616c93c4c6dbfafabc8ac88
Patch ID: 31b4660a50039e2c01ad49c0137f06200b82e4b8
Gator payload: 4

The autonomous Warning budget is exhausted, and a specific maintainer decision is required before review can proceed.

Root-cause findings:

• The requested explicit-TCP destination validation was broadened to every network endpoint. The new MissingEndpointHost check rejects the existing hostless allowed_ips mode, even though proto/sandbox.proto documents that an empty host with allowed_ips matches any hostname resolving into the allowed range and sandbox-policy.rego implements that behavior. This is a concrete public policy compatibility change outside the plan's stage-1 requirement that only explicit protocol: tcp eligibility have a non-empty host and concrete port.

Scope growth:

• The remediation changes validation semantics for all endpoint protocols and removes a currently supported endpoint shape, rather than enforcing the TCP-specific eligibility boundary requested by the series plan and @pimlock.

Reviewer-quality signals:

• The normalized critical-only review proposed no new findings; this convergence request comes from the separate mandatory scope-growth check.

Maintainer action: @pimlock or another OpenShell maintainer, please decide whether this PR should (a) limit the new host/port requirement to explicit protocol: tcp, preserving hostless allowed_ips, or (b) intentionally retire the hostless mode here and require the corresponding public-contract, compatibility, and migration treatment. A maintainer confirmation of the intended boundary will unblock the next step.

Next state: gator:blocked
Blocked reason: review_convergence_decision_required

[johntmyers]

gator-agent

Blocked

Thanks @pimlock. I checked your maintainer clarification that the existing hostless non-TCP allowed_ips mode remains supported while explicit protocol: tcp requires a DNS hostname. Thanks @johntmyers. I checked current head 26d10b1de8a1915e72aae2b87ad93e8986cac16c; the shared validation now preserves that legacy mode and applies the DNS-host boundary to explicit TCP with dedicated validation coverage. That resolves the prior maintainer convergence decision.

Head SHA: 26d10b1de8a1915e72aae2b87ad93e8986cac16c
Gator payload: 4

Gator is now blocked because GitHub reports this head as unmergeable with merge conflicts (mergeable: false, mergeable_state: dirty). The required critical-only author-delta review cannot proceed against a conflicted patch.

Next action: @johntmyers, resolve the conflicts with main and push the conflict-resolved head. Gator will then rebuild the feedback ledger and review only the new author delta under the stage-1 boundary.