fix(policy): gate uninspected credentialed endpoints

Signed-off-by: Adrien Langou <alangou@nvidia.com>

Author: alangou

.agents/skills/generate-sandbox-policy/SKILL.md

@@ -384,6 +384,7 @@ Before presenting the policy to the user, verify correctness **and** flag breadt
 - [ ] Middleware `order` values are unique and no selected chain exceeds 10 stages
 - [ ] No fail-closed middleware selector can cover a `tls: skip` endpoint
 - [ ] Any required WebSocket control advertises `WEBSOCKET_MESSAGE/PRE_CREDENTIALS`, and the user understands that V1 does not inspect binary messages
+- [ ] Endpoints contributed by a credentialed provider are not L4-only or `tls: skip` unless `allow_uninspected_credentials: true` explicitly records the exception
 
 ### Schema Warnings (log-only, but should be fixed)
 
@@ -418,6 +419,7 @@ Evaluate the generated policy for overly broad access and **include warnings in
 | **Broad CIDR** in `allowed_ips` (e.g., `10.0.0.0/8`) | "This `allowed_ips` entry covers a very broad range. Consider narrowing to a specific subnet (e.g., `10.0.5.0/24`) to minimize exposure." |
 | **`on_error: fail_open`** | "This middleware can be bypassed when it is unavailable, rejects configuration, returns an invalid result, or exceeds its body limit. Use `fail_closed` unless availability is more important than this control." |
 | **Broad middleware host selector** | "This middleware attaches independently of the admitting network rule to every matching destination, then runs only for operation bindings its implementation advertises. Narrow `endpoints.include` or add exclusions if the attachment is not required for every matching host." |
+| **`allow_uninspected_credentials: true`** | "This endpoint may carry provider credentials on traffic OpenShell cannot inspect or rewrite. Prefer an inspected protocol and credential rewrite; keep this exception only when raw traffic is required." |
 
 Format breadth warnings clearly in the output, e.g.:
 

.agents/skills/openshell-cli/cli-reference.md

@@ -384,6 +384,7 @@ Incrementally merge live network policy changes into the current sandbox policy.
 Notes:
 
 - The sandbox name defaults to the last-used sandbox.
+- `--add-endpoint` options are comma-separated: `allowed-ip=<CIDR-or-IP>`, `websocket-credential-rewrite`, `request-body-credential-rewrite`, and `allow-uninspected-credentials`. The last option is a security-sensitive exception for provider-credentialed L4-only, `tls: skip`, or otherwise uninspectable traffic.
 - `--add-allow` and `--add-deny` operate on REST and WebSocket endpoints. Use full YAML for JSON-RPC, MCP, SQL, or other policy structure.
 - `--wait` cannot be combined with `--dry-run`.
 - Use `policy set` when replacing the full policy or changing static sections.

architecture/security-policy.md

@@ -97,6 +97,53 @@ raw relay by default. A `protocol: rest` endpoint can opt in to
 after an allowed `101` upgrade; server-to-client traffic and all other upgraded
 protocols remain raw passthrough.
 
+## Credentialed Endpoints
+
+OpenShell keeps provider credentials on paths it can inspect or rewrite by
+default. The gateway derives credential provenance from attached provider
+profiles and stamps it onto the effective policy at composition time. This
+provenance is internal, contains no credential identifiers or values, and is
+never trusted from user-authored policy.
+
+Every evaluation clears provenance across the whole policy and re-derives it
+from the full set of attached provider profiles. The stamp is an assignment,
+not an accumulation, so an endpoint that stops matching a credentialed scope
+loses its marker in the same pass. This must remain a full recomputation: a
+delta-based derivation would let a series of individually valid edits reach a
+state no single edit would have admitted.
+
+Credentialed L4-only and `tls: skip` endpoints fail policy validation unless the
+public `allow_uninspected_credentials` escape hatch is explicitly enabled. The
+flag defaults to `false` and is security-flagged in policy approval flows.
+Incremental merges only ever add the flag to a matching endpoint; clearing it
+requires removing the endpoint or replacing the policy.
+
+The network supervisor independently enforces the same boundary. Credentialed
+WebSocket upgrades use the parsed relay, binary frames fail closed, and text
+placeholders require rewrite. REST bodies can continue streaming when body
+rewrite is disabled, but the relay withholds enough trailing bytes to detect a
+placeholder split across reads before forwarding its marker. Explicitly opted-in
+endpoints retain raw passthrough behavior.
+
+Denials emit both the relevant network activity and a detection finding. Events
+identify only the destination, policy, and traffic surface; they never include
+credential names, placeholders, body content, or secret values.
+
+Credential provenance is gateway-derived and deliberately absent from the policy
+YAML schema, so it does not survive a policy that never transits the gateway.
+Gateway-delivered policy is the authoritative source for this control, and a
+policy without provenance applies neither the raw-tunnel refusal nor the
+WebSocket binary-frame refusal. The request-body backstop still applies, because
+it keys off the presence of a secret resolver rather than endpoint provenance.
+
+Two paths load a policy without provenance. A supervisor booting from a
+container-image policy is a bounded window: that policy is resynchronized to the
+gateway, which then serves a stamped effective policy. An explicit local Rego and
+data override is permanent, because gateway revisions are observed for settings
+and providers but never replace the local policy. When that override is combined
+with injected provider credentials, the supervisor emits a high-severity
+detection finding at startup naming the inactive controls.
+
 ## Live Updates
 
 The gateway stores sandbox-authored policy revisions separately from derived

crates/openshell-cli/src/main.rs

@@ -1791,6 +1791,8 @@ enum PolicyCommands {
         name: Option<String>,
 
         /// Add or merge an endpoint: host:port[:access[:protocol[:enforcement[:options]]]].
+        /// Options include allowed-ip=..., credential rewrite flags, and
+        /// allow-uninspected-credentials.
         #[arg(long = "add-endpoint")]
         add_endpoints: Vec<String>,
 

crates/openshell-cli/src/policy_update.rs

@@ -368,6 +368,9 @@ fn apply_add_endpoint_options(
             ));
         }
         match option {
+            "allow-uninspected-credentials" => {
+                endpoint.allow_uninspected_credentials = true;
+            }
             "websocket-credential-rewrite" => {
                 ensure_websocket_credential_rewrite_protocol(spec, endpoint)?;
                 endpoint.websocket_credential_rewrite = true;
@@ -379,7 +382,7 @@ fn apply_add_endpoint_options(
             _ => {
                 let Some(allowed_ip) = option.strip_prefix("allowed-ip=") else {
                     return Err(miette!(
-                        "--add-endpoint options segment supports only 'websocket-credential-rewrite', 'request-body-credential-rewrite', and 'allowed-ip=<CIDR-or-IP>'; got '{option}' in '{spec}'"
+                        "--add-endpoint options segment supports only 'allow-uninspected-credentials', 'websocket-credential-rewrite', 'request-body-credential-rewrite', and 'allowed-ip=<CIDR-or-IP>'; got '{option}' in '{spec}'"
                     ));
                 };
                 let allowed_ip = allowed_ip.trim();
@@ -604,6 +607,25 @@ mod tests {
         assert!(endpoint.request_body_credential_rewrite);
     }
 
+    #[test]
+    fn parse_add_endpoint_enables_allow_uninspected_credentials() {
+        let plan = build_policy_update_plan(
+            &["api.vendor.example:443::::allow-uninspected-credentials".to_string()],
+            &[],
+            &[],
+            &[],
+            &[],
+            &[],
+            None,
+        )
+        .expect("plan should build");
+
+        let PolicyMergeOp::AddRule { rule, .. } = &plan.preview_operations[0] else {
+            panic!("expected add-rule preview");
+        };
+        assert!(rule.endpoints[0].allow_uninspected_credentials);
+    }
+
     #[test]
     fn parse_add_endpoint_merges_allowed_ips_with_websocket_options() {
         let plan = build_policy_update_plan(

crates/openshell-core/src/secrets.rs

@@ -13,6 +13,24 @@ const PROVIDER_ALIAS_MARKER: &str = "OPENSHELL-RESOLVE-ENV-";
 /// Public access to the placeholder prefix for fail-closed scanning in other modules.
 pub const PLACEHOLDER_PREFIX_PUBLIC: &str = PLACEHOLDER_PREFIX;
 pub const PROVIDER_ALIAS_MARKER_PUBLIC: &str = PROVIDER_ALIAS_MARKER;
+/// Longest wire form of a reserved marker: percent-encoding expands every
+/// marker byte to three bytes (`%XX`), and detection decodes in a single pass.
+const LONGEST_RESERVED_MARKER_WIRE_BYTES: usize =
+    3 * if PLACEHOLDER_PREFIX.len() > PROVIDER_ALIAS_MARKER.len() {
+        PLACEHOLDER_PREFIX.len()
+    } else {
+        PROVIDER_ALIAS_MARKER.len()
+    };
+
+/// Retain this many trailing bytes when scanning a streamed request body so a
+/// reserved marker split across reads cannot be forwarded before detection.
+///
+/// A marker is only detected while all of its wire bytes sit in the scan buffer
+/// at once, so the retained window must hold every byte of the longest form but
+/// the last. A window shorter than that lets a caller split a fully
+/// percent-encoded marker so its leading bytes are forwarded before the rest
+/// arrives, and the reassembled remainder no longer decodes to the marker.
+pub const CREDENTIAL_MARKER_SCAN_TAIL_BYTES: usize = LONGEST_RESERVED_MARKER_WIRE_BYTES;
 
 /// Characters that are valid in an env var key name (used to extract
 /// placeholder boundaries within concatenated strings like path segments).
@@ -36,6 +54,15 @@ pub fn contains_reserved_credential_marker(value: &str) -> bool {
     contains_raw_reserved_marker(&decoded)
 }
 
+pub fn contains_reserved_credential_marker_bytes(value: &[u8]) -> bool {
+    if value.is_empty() {
+        return false;
+    }
+    String::from_utf8_lossy(value)
+        .split('\0')
+        .any(contains_reserved_credential_marker)
+}
+
 // ---------------------------------------------------------------------------
 // Error and result types
 // ---------------------------------------------------------------------------
@@ -1304,6 +1331,45 @@ mod tests {
 
     // === Existing tests (preserved) ===
 
+    #[test]
+    fn byte_marker_detection_handles_raw_encoded_and_binary_input() {
+        assert!(contains_reserved_credential_marker_bytes(
+            b"openshell:resolve:env:API_TOKEN"
+        ));
+        assert!(contains_reserved_credential_marker_bytes(
+            b"openshell%3Aresolve%3Aenv%3AAPI_TOKEN"
+        ));
+        assert!(!contains_reserved_credential_marker_bytes(&[
+            0xff, 0x00, 0x01, 0x02
+        ]));
+    }
+
+    fn fully_percent_encoded(marker: &str) -> String {
+        const HEX: &[u8; 16] = b"0123456789ABCDEF";
+        let mut encoded = String::with_capacity(marker.len() * 3);
+        for byte in marker.bytes() {
+            encoded.push('%');
+            encoded.push(char::from(HEX[usize::from(byte >> 4)]));
+            encoded.push(char::from(HEX[usize::from(byte & 0x0f)]));
+        }
+        encoded
+    }
+
+    #[test]
+    fn scan_tail_window_covers_longest_encoded_marker_form() {
+        for marker in [PLACEHOLDER_PREFIX, PROVIDER_ALIAS_MARKER] {
+            let encoded = fully_percent_encoded(marker);
+            assert!(
+                contains_reserved_credential_marker(&encoded),
+                "fully encoded {marker} must be detected"
+            );
+            assert!(
+                CREDENTIAL_MARKER_SCAN_TAIL_BYTES >= encoded.len() - 1,
+                "scan window must retain every byte of {encoded} but the last"
+            );
+        }
+    }
+
     #[test]
     fn provider_env_is_replaced_with_placeholders() {
         let (child_env, resolver) = SecretResolver::from_provider_env(

crates/openshell-policy/src/lib.rs

@@ -103,6 +103,10 @@ struct NetworkPolicyRuleDef {
 
 #[derive(Debug, Serialize, Deserialize)]
 #[serde(deny_unknown_fields)]
+#[allow(
+    clippy::struct_excessive_bools,
+    reason = "Endpoint DTO mirrors independent policy schema toggles."
+)]
 struct NetworkEndpointDef {
     #[serde(default, skip_serializing_if = "String::is_empty")]
     host: String,
@@ -144,6 +148,10 @@ struct NetworkEndpointDef {
     /// placeholders before forwarding upstream. Defaults to false.
     #[serde(default, skip_serializing_if = "std::ops::Not::not")]
     request_body_credential_rewrite: bool,
+    /// Explicitly permits credentials on traffic paths that `OpenShell` cannot
+    /// inspect or rewrite. Defaults to false.
+    #[serde(default, skip_serializing_if = "std::ops::Not::not")]
+    allow_uninspected_credentials: bool,
     #[serde(default, skip_serializing_if = "String::is_empty")]
     persisted_queries: String,
     #[serde(default, skip_serializing_if = "BTreeMap::is_empty")]
@@ -745,6 +753,10 @@ fn to_proto(raw: PolicyFile) -> Result<SandboxPolicy> {
                             allow_encoded_slash: e.allow_encoded_slash,
                             websocket_credential_rewrite: e.websocket_credential_rewrite,
                             request_body_credential_rewrite: e.request_body_credential_rewrite,
+                            allow_uninspected_credentials: e.allow_uninspected_credentials,
+                            // Provider credential provenance is derived by the
+                            // gateway and cannot be authored in policy YAML.
+                            provider_credentialed: false,
                             // Advisor provenance is internal runtime state, not
                             // a user-authored policy schema field.
                             advisor_proposed: false,
@@ -901,6 +913,7 @@ fn from_proto(policy: &SandboxPolicy) -> PolicyFile {
                             allow_encoded_slash: e.allow_encoded_slash,
                             websocket_credential_rewrite: e.websocket_credential_rewrite,
                             request_body_credential_rewrite: e.request_body_credential_rewrite,
+                            allow_uninspected_credentials: e.allow_uninspected_credentials,
                             persisted_queries: e.persisted_queries.clone(),
                             graphql_persisted_queries: e
                                 .graphql_persisted_queries
@@ -3486,6 +3499,32 @@ network_policies:
         assert!(yaml_out.contains("request_body_credential_rewrite: true"));
     }
 
+    #[test]
+    fn round_trip_preserves_allow_uninspected_credentials() {
+        let yaml = r"
+version: 1
+network_policies:
+  vendor_api:
+    endpoints:
+      - host: api.vendor.example
+        port: 443
+        tls: skip
+        allow_uninspected_credentials: true
+";
+        let proto1 = parse_sandbox_policy(yaml).expect("parse failed");
+        let yaml_out = serialize_sandbox_policy(&proto1).expect("serialize failed");
+        let proto2 = parse_sandbox_policy(&yaml_out).expect("re-parse failed");
+
+        let ep = &proto2.network_policies["vendor_api"].endpoints[0];
+        assert!(ep.allow_uninspected_credentials);
+        assert!(
+            !ep.provider_credentialed,
+            "provider provenance must not be authorable from policy YAML"
+        );
+        assert!(yaml_out.contains("allow_uninspected_credentials: true"));
+        assert!(!yaml_out.contains("provider_credentialed"));
+    }
+
     #[test]
     fn websocket_credential_rewrite_defaults_false() {
         let yaml = r"
@@ -3504,6 +3543,8 @@ network_policies:
         let ep = &proto.network_policies["gateway"].endpoints[0];
         assert!(!ep.websocket_credential_rewrite);
         assert!(!ep.request_body_credential_rewrite);
+        assert!(!ep.allow_uninspected_credentials);
+        assert!(!ep.provider_credentialed);
     }
 
     #[test]

crates/openshell-policy/src/merge.rs

@@ -1282,6 +1282,7 @@ fn merge_endpoint(
     existing.allow_encoded_slash |= incoming.allow_encoded_slash;
     existing.websocket_credential_rewrite |= incoming.websocket_credential_rewrite;
     existing.request_body_credential_rewrite |= incoming.request_body_credential_rewrite;
+    existing.allow_uninspected_credentials |= incoming.allow_uninspected_credentials;
     existing.advisor_proposed |= incoming.advisor_proposed;
     normalize_endpoint(existing);
     Ok(())
@@ -3100,6 +3101,48 @@ mod tests {
         assert!(endpoint.request_body_credential_rewrite);
     }
 
+    #[test]
+    fn add_rule_merges_allow_uninspected_credentials_flag() {
+        let mut policy = restrictive_default_policy();
+        policy.network_policies.insert(
+            "existing".to_string(),
+            NetworkPolicyRule {
+                name: "existing".to_string(),
+                endpoints: vec![NetworkEndpoint {
+                    host: "api.vendor.example".to_string(),
+                    port: 443,
+                    ports: vec![443],
+                    ..Default::default()
+                }],
+                ..Default::default()
+            },
+        );
+
+        let incoming = NetworkPolicyRule {
+            name: "incoming".to_string(),
+            endpoints: vec![NetworkEndpoint {
+                host: "api.vendor.example".to_string(),
+                port: 443,
+                ports: vec![443],
+                allow_uninspected_credentials: true,
+                ..Default::default()
+            }],
+            ..Default::default()
+        };
+
+        let result = merge_policy(
+            policy,
+            &[PolicyMergeOp::AddRule {
+                rule_name: "allow_api_vendor_example_443".to_string(),
+                rule: incoming,
+            }],
+        )
+        .expect("merge should succeed");
+
+        let endpoint = &result.policy.network_policies["existing"].endpoints[0];
+        assert!(endpoint.allow_uninspected_credentials);
+    }
+
     #[test]
     fn add_allow_expands_access_preset() {
         let mut policy = restrictive_default_policy();