Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 6 additions & 4 deletions .agents/skills/debug-openshell-cluster/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -172,7 +172,7 @@ Common findings:
- Docker daemon unavailable: start Docker Desktop or Docker Engine.
- Gateway process stopped: inspect exit status and logs.
- Sandbox image missing or pull denied: verify image reference and registry credentials.
- Sandbox fails before readiness with an identity-resolution error: inspect the image's OCI `USER` and matching `/etc/passwd` and `/etc/group` entries, or explicitly set both process identity fields in policy. Root and missing identities are rejected.
- Sandbox fails before readiness with an identity-resolution error: inspect the image's OCI `USER` and matching `/etc/passwd` and `/etc/group` entries, or explicitly set both process identity fields in policy. Numeric workload identities `1` through `4294967294` are accepted; root, the invalid identity sentinel, and missing identities are rejected.
- Sandbox fails before readiness with an OCI workspace validation error: inspect the image's `WorkingDir` using the immutable image ID reported by the gateway. Empty, `/`, and explicit `/sandbox` use the managed `/sandbox` compatibility workspace. Any other workdir must be an absolute normalized directory with no symlink components; the final policy UID, primary GID, and supplementary groups must pass the kernel's effective traverse/write checks, including POSIX ACL and LSM decisions. OpenShell does not create, chown, or chmod a non-default image workdir.
- Docker also rejects an image `VOLUME` that covers the workdir or one of its parents because the runtime would mask the immutable path before validation. Move the `VOLUME` below the workspace or remove the declaration.
- A workdir rejected as a special filesystem or OpenShell control-path collision cannot be made valid with permissions. Move the image workdir away from kernel-backed mounts and the concrete supervisor, TLS, token, runtime, and socket paths named in the error.
Expand Down Expand Up @@ -208,7 +208,7 @@ Common findings:
- Podman socket unavailable: start or expose the user socket.
- Rootless networking unavailable: inspect Podman network configuration.
- Sandbox image missing or pull denied: verify image reference and registry credentials.
- Sandbox fails before readiness with an identity-resolution error: inspect the image's OCI `USER` and matching `/etc/passwd` and `/etc/group` entries, or explicitly set both process identity fields in policy. Root and missing identities are rejected.
- Sandbox fails before readiness with an identity-resolution error: inspect the image's OCI `USER` and matching `/etc/passwd` and `/etc/group` entries, or explicitly set both process identity fields in policy. Numeric workload identities `1` through `4294967294` are accepted; root, the invalid identity sentinel, and missing identities are rejected.
- Supervisor cannot call back: check callback endpoint and gateway logs.
- Gateway exits before becoming healthy with a callback-listener discovery
error: inspect `podman info --debug`, the configured Podman network, and the
Expand Down Expand Up @@ -452,8 +452,10 @@ should be the only sidecar topology container with `NET_ADMIN`. It also needs
default binary-aware network sidecar runs as UID 0 with primary GID
`sandbox_gid` and adds `SYS_PTRACE` plus `DAC_READ_SEARCH`. When
`process_binary_aware_network_policy = false`, it runs as the configured
non-root `proxy_uid` without those inspection capabilities. The pod `fsGroup`
is set to `sandbox_gid` in both modes.
non-root `proxy_uid` without those inspection capabilities. That dedicated
proxy UID must remain at least `1000` and must not match the workload UID
because the pod egress fence exempts its traffic. The pod `fsGroup` is set to
`sandbox_gid` in both modes.

In sidecar topology only the network sidecar should mount the gateway bootstrap
credentials (`openshell-sa-token` and `openshell-client-tls`). The process
Expand Down
6 changes: 6 additions & 0 deletions .agents/skills/generate-sandbox-policy/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -502,6 +502,12 @@ environment. Gateway inference is configured separately through `openshell
inference set/get`. The generated `network_policies` block is the primary
output.

When the user explicitly requests `process.run_as_user` or
`process.run_as_group`, accept `sandbox` or a numeric UID/GID from `1` through
`4294967294`. Reject root (`0`) and the invalid identity sentinel
(`4294967295`). Warn that a low numeric identity inherits permissions granted
to the same ID on image files, mounted volumes, or devices.

If the user provides a file path, write to it. Otherwise, ask where to place it. A common convention is a project-local policy file (e.g., `sandbox-policy.yaml`) passed to `openshell sandbox create --policy <path>` or set via the `OPENSHELL_SANDBOX_POLICY` env var.

### Mode C: Present Only (no file write)
Expand Down
3 changes: 2 additions & 1 deletion .agents/skills/helm-dev-environment/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,8 @@ Both commands build the `gateway` and `supervisor` images and deploy the OpenShe
chart. The sidecar profile renders an `openshell-network-init` init container for
nftables setup and an `openshell-supervisor-network` runtime sidecar for proxying.
Binary-aware policy mode runs that sidecar as UID 0 with `SYS_PTRACE` and
`DAC_READ_SEARCH`; relaxed mode can run it as the configured proxy UID. The
`DAC_READ_SEARCH`; relaxed mode can run it as the configured proxy UID, which
must be at least `1000` and distinct from the workload UID. The
sidecar-mTLS profile reuses `ci/values-sidecar.yaml` and restores
`server.disableTls=false` inline for Skaffold. The `pkiInitJob` hook (a pre-install
Job that runs `openshell-gateway generate-certs`) generates mTLS secrets on first
Expand Down
5 changes: 4 additions & 1 deletion .agents/skills/openshell-cli/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -479,7 +479,10 @@ For Docker and Podman gateways, custom images should declare a non-root OCI
`USER`. Each explicit `process.run_as_user` or `process.run_as_group` policy
field wins independently; omitted fields fall back to the image declaration.
An image with no `USER` fails before readiness unless policy supplies both
fields.
fields. Explicit numeric fields may use any UID/GID from `1` through
`4294967294`; `0` is root and `4294967295` is the invalid identity sentinel.
Warn users that low IDs can inherit permissions from matching accounts, image
files, mounted volumes, or devices.

### Forward ports

Expand Down
7 changes: 7 additions & 0 deletions architecture/compute-runtimes.md
Original file line number Diff line number Diff line change
Expand Up @@ -214,6 +214,13 @@ driver then supplies one authoritative identity input to the supervisor:
SCC-derived values.
- VM keeps its existing guest identity behavior.

Explicit numeric workload identities may use any Linux UID/GID from `1`
through `u32::MAX - 1`. UID/GID `0` remains prohibited as root, and
`u32::MAX` remains prohibited because Linux APIs and POSIX ACLs use it as an
invalid identity sentinel. Infrastructure identities use separate validation:
the Kubernetes network proxy UID remains at least `1000` and must not match the
workload UID because its traffic bypasses the pod egress fence.

For Docker and Podman, policy values take precedence independently. An omitted
`run_as_user` or `run_as_group` falls back to the corresponding identity from
the image. The supervisor resolves names from the image's `/etc/passwd` and
Expand Down
34 changes: 26 additions & 8 deletions crates/openshell-driver-kubernetes/src/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -162,10 +162,13 @@ impl Default for KubernetesSidecarConfig {

impl KubernetesSidecarConfig {
pub fn validate_proxy_uid(&self) -> Result<(), String> {
if self.proxy_uid < openshell_policy::MIN_SANDBOX_UID {
if !(openshell_policy::MIN_SANDBOX_PROXY_UID..=openshell_policy::MAX_SANDBOX_UID)
.contains(&self.proxy_uid)
{
return Err(format!(
"sidecar.proxy_uid must be at least {}",
openshell_policy::MIN_SANDBOX_UID
"sidecar.proxy_uid must be in range [{}, {}]",
openshell_policy::MIN_SANDBOX_PROXY_UID,
openshell_policy::MAX_SANDBOX_UID,
));
}
Ok(())
Expand Down Expand Up @@ -1296,17 +1299,28 @@ mod tests {
}

#[test]
fn parse_openshift_uid_range_rejects_below_min() {
// 999 is below MIN_SANDBOX_UID (1000) — should be rejected.
fn parse_openshift_uid_range_accepts_non_root_system_uid() {
assert_eq!(
KubernetesComputeConfig::from_open_shift_uid_range("999/50000"),
Some(999)
);
assert_eq!(
KubernetesComputeConfig::from_open_shift_uid_range("1/50000"),
Some(1)
);
}

#[test]
fn parse_openshift_uid_range_rejects_root() {
assert_eq!(
KubernetesComputeConfig::from_open_shift_uid_range("0/50000"),
None
);
}

#[test]
fn parse_openshift_uid_range_rejects_above_max() {
// u32::MAX is well above MAX_SANDBOX_UID — should be rejected.
// u32::MAX is the invalid identity sentinel.
assert_eq!(
KubernetesComputeConfig::from_open_shift_uid_range("4294967295/10000"),
None
Expand All @@ -1316,8 +1330,8 @@ mod tests {
#[test]
fn validate_sandbox_identity_config_accepts_valid_range() {
let cfg = KubernetesComputeConfig {
sandbox_uid: Some(1000),
sandbox_gid: Some(1000),
sandbox_uid: Some(500),
sandbox_gid: Some(30),
..KubernetesComputeConfig::default()
};
assert!(cfg.validate_sandbox_identity_config().is_ok());
Expand Down Expand Up @@ -1355,6 +1369,10 @@ mod tests {
KubernetesComputeConfig::from_open_shift_supplemental_groups("1000/50000"),
Some(1000)
);
assert_eq!(
KubernetesComputeConfig::from_open_shift_supplemental_groups("30/50000"),
Some(30)
);
}

#[test]
Expand Down
10 changes: 10 additions & 0 deletions crates/openshell-driver-vm/src/driver.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6950,6 +6950,16 @@ mod tests {
)));
}

#[test]
fn validate_sandbox_identity_accepts_non_root_system_ids() {
let config = VmDriverConfig {
sandbox_uid: Some(500),
sandbox_gid: Some(30),
..Default::default()
};
assert!(config.validate_sandbox_identity().is_ok());
}

#[test]
fn build_guest_environment_uses_token_file_without_raw_token_env() {
let config = VmDriverConfig {
Expand Down
55 changes: 28 additions & 27 deletions crates/openshell-policy/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -959,13 +959,24 @@ fn from_proto(policy: &SandboxPolicy) -> PolicyFile {
// Sandbox UID/GID constants
// ---------------------------------------------------------------------------

/// Minimum accepted UID for sandbox process identity.
/// UIDs below this are reserved for system users and are rejected.
pub const MIN_SANDBOX_UID: u32 = 1000;
/// Minimum accepted UID/GID for sandbox workload identity.
///
/// Linux reserves only identity `0` for root. Non-root system identities are
/// valid workload identities when selected explicitly by the operator.
pub const MIN_SANDBOX_UID: u32 = 1;

/// Maximum accepted UID for sandbox process identity.
/// UIDs above this exceed typical OS limits and are rejected.
pub const MAX_SANDBOX_UID: u32 = 2_000_000_000;
/// Maximum accepted UID/GID for sandbox workload identity.
///
/// `u32::MAX` represents an invalid or unchanged identity in Linux APIs and
/// POSIX ACLs, so the largest usable workload identity is one less.
pub const MAX_SANDBOX_UID: u32 = u32::MAX - 1;

/// Minimum UID for the Kubernetes network proxy identity.
///
/// The proxy UID is exempt from the pod egress fence, so it remains in a
/// dedicated infrastructure range even though workload identities may use
/// non-root system IDs.
pub const MIN_SANDBOX_PROXY_UID: u32 = 1000;

/// The literal string value accepted as a valid sandbox user/group name.
const SANDBOX_NAME: &str = "sandbox";
Expand All @@ -977,8 +988,8 @@ const SANDBOX_NAME: &str = "sandbox";
///
/// Rejects:
/// - The empty string (represents an omitted policy field)
/// - UID 0 or values below `MIN_SANDBOX_UID`
/// - Values above `MAX_SANDBOX_UID`
/// - UID/GID 0 (root)
/// - `u32::MAX`, the invalid identity sentinel
/// - Non-numeric strings other than `"sandbox"` (e.g. `"root"`, `"nobody"`)
pub fn is_valid_sandbox_identity(value: &str) -> bool {
if value == SANDBOX_NAME {
Expand Down Expand Up @@ -2786,7 +2797,11 @@ network_policies:
}

#[test]
fn valid_identity_accepts_numeric_uid_in_range() {
fn valid_identity_accepts_non_root_numeric_uid() {
assert!(is_valid_sandbox_identity("1"));
assert!(is_valid_sandbox_identity("30"));
assert!(is_valid_sandbox_identity("500"));
assert!(is_valid_sandbox_identity("999"));
assert!(is_valid_sandbox_identity("1000"));
assert!(is_valid_sandbox_identity("50000"));
assert!(is_valid_sandbox_identity("1000660000"));
Expand All @@ -2804,14 +2819,7 @@ network_policies:
}

#[test]
fn valid_identity_rejects_system_uids_below_min() {
assert!(!is_valid_sandbox_identity("999"));
assert!(!is_valid_sandbox_identity("100"));
assert!(!is_valid_sandbox_identity("1"));
}

#[test]
fn valid_identity_rejects_uid_above_max() {
fn valid_identity_rejects_invalid_uid_sentinel() {
assert!(!is_valid_sandbox_identity(
&MAX_SANDBOX_UID.saturating_add(1).to_string()
));
Expand Down Expand Up @@ -2864,20 +2872,13 @@ network_policies:
}

#[test]
fn validate_rejects_uid_out_of_range_low() {
fn validate_accepts_non_root_system_uid() {
let mut policy = restrictive_default_policy();
policy.process = Some(ProcessPolicy {
run_as_user: "500".into(),
run_as_group: "sandbox".into(),
run_as_group: "30".into(),
});
let violations = validate_sandbox_policy(&policy).unwrap_err();
assert!(violations.iter().any(|v| matches!(
v,
PolicyViolation::InvalidProcessIdentity {
field: "run_as_user",
..
}
)));
assert!(validate_sandbox_policy(&policy).is_ok());
}

#[test]
Expand Down
31 changes: 22 additions & 9 deletions crates/openshell-sandbox/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -472,16 +472,23 @@ fn run_network_init(

#[cfg(target_os = "linux")]
fn validate_network_init_ids(proxy_user_id: u32, proxy_primary_group_id: u32) -> Result<()> {
if proxy_user_id != 0 && proxy_user_id < openshell_policy::MIN_SANDBOX_UID {
if proxy_user_id != 0
&& !(openshell_policy::MIN_SANDBOX_PROXY_UID..=openshell_policy::MAX_SANDBOX_UID)
.contains(&proxy_user_id)
{
return Err(miette::miette!(
"--proxy-uid must be 0 or at least {}",
openshell_policy::MIN_SANDBOX_UID
"--proxy-uid must be 0 or in range [{}, {}]",
openshell_policy::MIN_SANDBOX_PROXY_UID,
openshell_policy::MAX_SANDBOX_UID,
));
}
if proxy_primary_group_id < openshell_policy::MIN_SANDBOX_UID {
if !(openshell_policy::MIN_SANDBOX_UID..=openshell_policy::MAX_SANDBOX_UID)
.contains(&proxy_primary_group_id)
{
return Err(miette::miette!(
"--proxy-gid must be at least {}",
openshell_policy::MIN_SANDBOX_UID
"--proxy-gid must be in range [{}, {}]",
openshell_policy::MIN_SANDBOX_UID,
openshell_policy::MAX_SANDBOX_UID,
));
}
Ok(())
Expand Down Expand Up @@ -806,17 +813,23 @@ mod tests {
#[cfg(target_os = "linux")]
#[test]
fn network_init_accepts_root_proxy_uid_for_binary_aware_sidecar() {
validate_network_init_ids(0, openshell_policy::MIN_SANDBOX_UID).unwrap();
validate_network_init_ids(0, 30).unwrap();
}

#[cfg(target_os = "linux")]
#[test]
fn network_init_still_rejects_low_non_root_proxy_ids() {
fn network_init_still_rejects_low_non_root_proxy_uid_and_root_gid() {
let uid_err =
validate_network_init_ids(999, openshell_policy::MIN_SANDBOX_UID).unwrap_err();
assert!(uid_err.to_string().contains("--proxy-uid"));

let gid_err = validate_network_init_ids(0, 999).unwrap_err();
let gid_err = validate_network_init_ids(0, 0).unwrap_err();
assert!(gid_err.to_string().contains("--proxy-gid"));
}

#[cfg(target_os = "linux")]
#[test]
fn network_init_accepts_non_root_system_proxy_group() {
validate_network_init_ids(openshell_policy::MIN_SANDBOX_PROXY_UID, 30).unwrap();
}
}
4 changes: 4 additions & 0 deletions crates/openshell-supervisor-process/src/identity.rs
Original file line number Diff line number Diff line change
Expand Up @@ -662,6 +662,10 @@ mod tests {
gid: 1235
}
);
assert_eq!(
DriverIdentity::from_values(None, Some("500".into()), Some("30".into())).unwrap(),
DriverIdentity::Resolved { uid: 500, gid: 30 }
);
assert_eq!(
DriverIdentity::from_values(
Some(String::new()),
Expand Down
6 changes: 3 additions & 3 deletions crates/openshell-supervisor-process/src/process.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2230,14 +2230,14 @@ mod tests {

#[test]
#[cfg(unix)]
fn explicit_identity_rejects_non_root_system_ids() {
fn explicit_identity_accepts_non_root_system_ids() {
let policy = policy_with_process(ProcessPolicy {
run_as_user: Some("101".into()),
run_as_group: Some("102".into()),
});

assert!(validate_sandbox_user(&policy).is_err());
assert!(validate_sandbox_group(&policy).is_err());
assert!(validate_sandbox_user(&policy).is_ok());
assert!(validate_sandbox_group(&policy).is_ok());
}

#[test]
Expand Down
4 changes: 2 additions & 2 deletions docs/kubernetes/topology.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -216,8 +216,8 @@ topology = "sidecar"
proxy_uid = 1337
```

`proxy_uid` configures only the relaxed endpoint/L7-only sidecar. It must be a
non-root UID and must not match the sandbox UID. The default binary-aware mode
`proxy_uid` configures only the relaxed endpoint/L7-only sidecar. It must be at
least `1000` and must not match the sandbox UID. The default binary-aware mode
runs the sidecar as UID 0 instead. The network init container exempts the
effective sidecar UID from proxy redirection so the sidecar can reach the
gateway.
Expand Down
7 changes: 5 additions & 2 deletions docs/reference/gateway-config.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -517,7 +517,7 @@ provider_spiffe_workload_api_socket_path = "/spiffe-workload-api/spire-agent.soc
# Explicit sandbox UID/GID for the supervisor container securityContext and
# PVC init container. When unset, the driver auto-detects from OpenShift SCC
# namespace annotations (openshift.io/sa.scc.uid-range) if present, falling
# back to 1000 on non-OpenShift clusters.
# back to 1000 on non-OpenShift clusters. Any non-root Linux UID/GID is valid.
# sandbox_uid = 1500
# sandbox_gid = 1500
# Operator-mode namespace discovery. At least one must be set when
Expand All @@ -537,7 +537,9 @@ gateway_pod_selector = { "app.kubernetes.io/name" = "openshell", "app.kubernetes
# UID used by relaxed long-running network sidecars. Strict process/binary-aware
# sidecars run as UID 0 so Kubernetes grants the required /proc inspection
# capabilities into the effective set. In sidecar topology the network init
# container installs nftables rules that exempt the effective sidecar UID.
# container installs nftables rules that exempt the effective sidecar UID, so
# this dedicated infrastructure UID must remain at least 1000 and must not
# match the sandbox workload UID.
proxy_uid = 1337
# Keep process/binary-aware network policy enabled in sidecar topology. Set
# false to run the sidecar as proxy_uid, drop the sidecar's extra /proc
Expand Down Expand Up @@ -737,6 +739,7 @@ guest_tls_cert = "/var/lib/openshell/guest-tls/client.pem"
guest_tls_key = "/var/lib/openshell/guest-tls/client-key.pem"
# Resolved sandbox UID/GID for the rootfs /etc/passwd entry.
# Defaults to 10001 when unset; matching GID is used if sandbox_gid is empty.
# Any non-root Linux UID/GID is valid.
# sandbox_uid = 20001
```

Expand Down
Loading
Loading