Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .agents/skills/debug-openshell-cluster/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,14 @@ The target deployment flow is:
4. The CLI registers a reachable gateway endpoint with `openshell gateway add`.
5. The gateway creates sandboxes through the selected compute driver.

The `openshell-gateway` composition crate explicitly installs its compiled
Docker, Podman, Kubernetes, and VM registrations at startup; `openshell-server`
does not link compute-driver crates. With no configured driver, the
gateway probes only installed registrations in priority order (Kubernetes,
Podman, then Docker); VM has no probe and remains opt-in. A custom gateway
binary may install a different set, so confirm the binary's registered drivers
when auto-detection reports that no suitable driver is available.

For local evaluation only, TLS may be disabled and the gateway can be reached through `http://127.0.0.1:<port>`.

## Prerequisites
Expand Down
18 changes: 17 additions & 1 deletion .github/workflows/branch-e2e.yml
Original file line number Diff line number Diff line change
Expand Up @@ -196,6 +196,20 @@ jobs:
e2e-task: e2e:kubernetes:workspace-managed
cli-artifact-prefix: rust-binary-cli

kubernetes-external-driver-e2e:
needs: [pr_metadata, build-gateway, build-supervisor, build-cli]
if: needs.pr_metadata.outputs.should_run == 'true' && needs.pr_metadata.outputs.run_core_e2e == 'true'
permissions:
actions: read
contents: read
packages: read
uses: ./.github/workflows/e2e-kubernetes-test.yml
with:
image-tag: ${{ github.sha }}
job-name: Kubernetes E2E (external compute driver)
e2e-task: e2e:kubernetes:external-driver
cli-artifact-prefix: rust-binary-cli

kubernetes-workspace-operator-e2e:
needs: [pr_metadata, build-gateway, build-supervisor, build-cli]
if: needs.pr_metadata.outputs.should_run == 'true' && needs.pr_metadata.outputs.run_core_e2e == 'true'
Expand Down Expand Up @@ -240,7 +254,7 @@ jobs:

core-e2e-result:
name: Core E2E result
needs: [pr_metadata, build-gateway, build-supervisor, build-cli, build-driver-vm-linux, e2e, kubernetes-e2e, kubernetes-workspace-managed-e2e, kubernetes-workspace-operator-e2e]
needs: [pr_metadata, build-gateway, build-supervisor, build-cli, build-driver-vm-linux, e2e, kubernetes-e2e, kubernetes-external-driver-e2e, kubernetes-workspace-managed-e2e, kubernetes-workspace-operator-e2e]
if: always() && needs.pr_metadata.outputs.should_run == 'true' && needs.pr_metadata.outputs.run_core_e2e == 'true'
runs-on: ubuntu-latest
steps:
Expand All @@ -252,6 +266,7 @@ jobs:
BUILD_DRIVER_VM_RESULT: ${{ needs.build-driver-vm-linux.result }}
E2E_RESULT: ${{ needs.e2e.result }}
KUBERNETES_E2E_RESULT: ${{ needs.kubernetes-e2e.result }}
KUBERNETES_EXTERNAL_DRIVER_E2E_RESULT: ${{ needs.kubernetes-external-driver-e2e.result }}
KUBERNETES_WORKSPACE_MANAGED_E2E_RESULT: ${{ needs.kubernetes-workspace-managed-e2e.result }}
KUBERNETES_WORKSPACE_OPERATOR_E2E_RESULT: ${{ needs.kubernetes-workspace-operator-e2e.result }}
run: |
Expand All @@ -264,6 +279,7 @@ jobs:
"build-driver-vm-linux:$BUILD_DRIVER_VM_RESULT" \
"e2e:$E2E_RESULT" \
"kubernetes-e2e:$KUBERNETES_E2E_RESULT" \
"kubernetes-external-driver-e2e:$KUBERNETES_EXTERNAL_DRIVER_E2E_RESULT" \
"kubernetes-workspace-managed-e2e:$KUBERNETES_WORKSPACE_MANAGED_E2E_RESULT" \
"kubernetes-workspace-operator-e2e:$KUBERNETES_WORKSPACE_OPERATOR_E2E_RESULT"; do
name="${item%%:*}"
Expand Down
9 changes: 6 additions & 3 deletions .github/workflows/e2e-kubernetes-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -96,14 +96,17 @@ jobs:
run: mise install --locked

# The openshell-policy crate transitively pulls in z3-sys, whose
# build script needs the z3 C/C++ headers and clang/bindgen to
# compile. The bare runner doesn't ship them; the CI container
# build script needs the z3 C/C++ headers, clang/bindgen, and CMake to
# compile both system-linked and bundled-Z3 builds. The bare runner
# doesn't ship them; the CI container
# image used by other Rust e2e jobs does, but we can't run this job
# there (the runner's container handler injects its own --network
# bridge, which conflicts with the --network host we need so kind's
# API server is reachable from the test process).
- name: Install z3 build deps
run: sudo apt-get update && sudo apt-get install -y --no-install-recommends libz3-dev clang
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends libz3-dev clang cmake

- name: Log in to GHCR
run: echo "${{ secrets.GITHUB_TOKEN }}" | docker login ghcr.io -u "${{ github.actor }}" --password-stdin
Expand Down
8 changes: 7 additions & 1 deletion .github/workflows/e2e-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,9 @@ jobs:
- suite: rust-docker
cmd: "mise run --no-deps --skip-deps e2e:rust"
apt_packages: "openssh-client"
- suite: rust-docker-external-driver
cmd: "env -u OPENSHELL_GATEWAY_BIN mise run --no-deps --skip-deps e2e:docker:external-driver"
apt_packages: "openssh-client"
- suite: mcp
cmd: "mise run --no-deps --skip-deps e2e:mcp"
apt_packages: ""
Expand Down Expand Up @@ -264,6 +267,9 @@ jobs:
- name: Run rootless Podman E2E
run: mise run --no-deps --skip-deps e2e:podman:rootless

- name: Run external Podman driver E2E
run: env -u OPENSHELL_GATEWAY_BIN mise run --no-deps --skip-deps e2e:podman:external-driver

- name: Print AppArmor denials
if: always()
run: sudo dmesg | grep -E 'apparmor=.*DENIED|profile="unprivileged_userns"' | tail -100 || true
Expand Down Expand Up @@ -362,4 +368,4 @@ jobs:
cache-on-failure: "true"

- name: Run VM E2E
run: mise run --no-deps --skip-deps e2e:vm
run: env -u OPENSHELL_GATEWAY_BIN mise run --no-deps --skip-deps e2e:vm:external-driver
2 changes: 1 addition & 1 deletion .github/workflows/release-dev.yml
Original file line number Diff line number Diff line change
Expand Up @@ -442,7 +442,7 @@ jobs:
run: |
set -euo pipefail
mise x -- rustup target add ${{ matrix.target }}
mise x -- cargo zigbuild --release --target ${{ matrix.zig_target }} -p openshell-server --bin openshell-gateway --features bundled-z3
mise x -- cargo zigbuild --release --target ${{ matrix.zig_target }} -p openshell-gateway --bin openshell-gateway --features bundled-z3
mkdir -p artifacts/bin
install -m 0755 target/${{ matrix.target }}/release/openshell-gateway artifacts/bin/openshell-gateway

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/release-tag.yml
Original file line number Diff line number Diff line change
Expand Up @@ -478,7 +478,7 @@ jobs:
run: |
set -euo pipefail
mise x -- rustup target add ${{ matrix.target }}
mise x -- cargo zigbuild --release --target ${{ matrix.zig_target }} -p openshell-server --bin openshell-gateway --features bundled-z3
mise x -- cargo zigbuild --release --target ${{ matrix.zig_target }} -p openshell-gateway --bin openshell-gateway --features bundled-z3
mkdir -p artifacts/bin
install -m 0755 target/${{ matrix.target }}/release/openshell-gateway artifacts/bin/openshell-gateway

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/rust-native-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ jobs:

case "$COMPONENT" in
gateway)
crate=openshell-server
crate=openshell-gateway
binary=openshell-gateway
zig_target=
;;
Expand Down
1 change: 1 addition & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ These pipelines connect skills into end-to-end workflows. Individual skill files
| `crates/openshell-otel/` | OpenTelemetry support | Shared OTLP trace provider, resource, and tracing-layer construction |
| `crates/openshell-core/` | Shared core | Common types, configuration, error handling |
| `crates/openshell-extension-core/` | Extension core | Shared extension identity, JWT claims, bearer-token rotation, and TLS transport primitives |
| `crates/openshell-gateway/` | Gateway binary composition | Links selected first-party compute drivers into the backend-agnostic server registry |
| `crates/openshell-sdk/` | Shared client SDK | Async Rust gateway client (gRPC transport, TLS, OIDC refresh, edge tunnel); consumed by CLI, TUI, and `@openshell/sdk` |
| `crates/openshell-providers/` | Provider management | Credential provider backends |
| `crates/openshell-tui/` | Terminal UI | Ratatui-based dashboard for monitoring |
Expand Down
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -445,7 +445,7 @@ The following Bazel commands are available alongside the mise tasks above. Cargo
| Build everything | `bazel build //...` | All crates and protos |
| Run all tests | `bazel test //...` | Unit tests only, no E2E |
| Build the CLI | `bazel build //crates/openshell-cli:openshell` | |
| Build the gateway | `bazel build //crates/openshell-server:openshell-gateway` | |
| Build the gateway | `bazel build //crates/openshell-gateway:openshell-gateway-bin` | |
| Build the supervisor | `bazel build //crates/openshell-sandbox:openshell-sandbox-bin` | |
| Clean | `bazel clean` | |

Expand Down
31 changes: 28 additions & 3 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -258,7 +258,7 @@ OpenShell collects anonymous telemetry to help improve the project for developer

Disable telemetry at runtime by setting `OPENSHELL_TELEMETRY_ENABLED=false` on the gateway deployment. For Helm installs, set `server.telemetryEnabled=false`. OpenShell propagates this deployment setting into sandbox supervisor environments so sandbox-side telemetry collection is disabled as well.

You can also compile telemetry out entirely. Telemetry support is a default-on `telemetry` Cargo feature; building with `--no-default-features` produces binaries that contain no telemetry endpoint, no telemetry HTTP client, and no emission code. Build telemetry-free artifacts with, for example, `cargo build --release -p openshell-server --no-default-features` (gateway) and the equivalent for `openshell-sandbox` and `openshell-driver-vm`. With telemetry compiled out, the gateway emits nothing and reports telemetry disabled to the sandboxes it launches.
You can also compile telemetry out entirely. Telemetry support is a default-on `telemetry` Cargo feature; building with `--no-default-features` produces binaries that contain no telemetry endpoint, no telemetry HTTP client, and no emission code. Build a telemetry-free gateway with `cargo build --release -p openshell-gateway --no-default-features --features in-tree-compute-drivers`, and use the equivalent feature selection for `openshell-sandbox` and `openshell-driver-vm`. With telemetry compiled out, the gateway emits nothing and reports telemetry disabled to the sandboxes it launches.

Telemetry events are limited to anonymous operational categories and counts, such as sandbox lifecycle outcomes, provider profile buckets, policy decision counts, and aggregate network activity denial categories. OpenShell telemetry does not collect sandbox names or IDs, hostnames, file paths, binary paths, prompts, credentials, provider names, model names, or user content.

Expand Down
2 changes: 1 addition & 1 deletion architecture/build.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ Sandbox community images are built outside this repository.
Anonymous telemetry emission is gated behind a default-on `telemetry` Cargo
feature. It is defined in `openshell-core` (where the emission code, HTTP
client, and endpoint live) and forwarded by the binary crates that emit or
collect telemetry: `openshell-server` (gateway), `openshell-sandbox`
collect telemetry: `openshell-gateway`, `openshell-sandbox`
(supervisor), and `openshell-driver-vm`. Every crate depends on
`openshell-core` with `default-features = false`, so the binary crate's feature
is the single switch that enables `openshell-core/telemetry` for its build
Expand Down
41 changes: 34 additions & 7 deletions architecture/compute-runtimes.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,33 @@ The gateway records driver identity and version from the startup capability
response. Elevated gateway info reports that initialized driver snapshot instead
of re-querying drivers on each request.

## Compiled Driver Selection

The gateway binary explicitly installs the compute drivers compiled into that
binary before entering server startup. The server selects a configured driver
by normalized registry name. When no driver is configured, it evaluates only
the installed drivers' probes and chooses the lowest registered priority.
Drivers without a probe, including VM, remain opt-in.

This follows the same composition model as SQLx's `Any` drivers: the binary
defines the available implementation set, while the runtime consumes a generic
registry. Adding or removing a compiled driver therefore changes registration
rather than the server's selection flow. Alternate gateway binaries can install
their own `ComputeDriverFactory` registrations and hand the completed registry
to `run_cli_with_compute_drivers`; factories receive merged driver config and
return either an in-process driver or a gateway-managed remote endpoint. The
server constructs the common runtime adapter and snapshots `GetCapabilities`
for either result. A configured UDS endpoint still takes precedence over a
compiled registration with the same name.

The `openshell-gateway` composition crate groups first-party registrations
behind the `in-tree-compute-drivers` feature. `openshell-server` has no compute
driver dependencies or backend-name dispatch. Protocol-only gateway builds
disable the composition feature and link no compute-driver crates. E2E lanes
compose that gateway with Docker, Podman, Kubernetes, and VM driver executables
over the public UDS gRPC contract so an in-tree driver cannot silently depend
on a server-only API.

## Stop and Start Lifecycle

The gateway persists lifecycle intent before mutating compute:
Expand Down Expand Up @@ -377,13 +404,13 @@ image-pull Secrets in every operator-managed namespace.

**Operator** uses pre-provisioned namespaces discovered through two optional
sources: a K8s label selector (`operator_namespace_label`) and a drop-in
allowlist file (`operator_namespace_file`). At least one must be configured.
The `OperatorNamespaceAllowlist` (`Arc<RwLock<BTreeSet<String>>>`) is populated
at runtime by background watchers and read by the namespace resolver. Sandbox
creation fails closed if the workspace is not in the current allowlist. Platform
teams manage namespace lifecycle externally. RBAC uses the same ClusterRole as
managed mode but without namespace `create`/`delete` or ServiceAccount
permissions.
allowlist file (`operator_namespace_file`). Exactly one must be configured.
The compute driver and the gateway's ServiceAccount authenticator independently
watch that public config source; no in-process driver state crosses into the
server. Sandbox creation and token bootstrap fail closed if the workspace is
not in the current allowlist. Platform teams manage namespace lifecycle
externally. RBAC uses the same ClusterRole as managed mode but without namespace
`create`/`delete` or ServiceAccount permissions.

### Watching and Querying

Expand Down
6 changes: 3 additions & 3 deletions bazel/releases/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -59,15 +59,15 @@ platform_transition_binary(
platform_transition_binary(
name = "openshell_gateway_linux_x86_64",
basename = "openshell-gateway",
binary = "//crates/openshell-server:openshell-gateway",
binary = "//crates/openshell-gateway:openshell-gateway-bin",
tags = ["manual"],
target_platform = ":linux_x86_64_gnu_2_28",
)

platform_transition_binary(
name = "openshell_gateway_linux_aarch64",
basename = "openshell-gateway",
binary = "//crates/openshell-server:openshell-gateway",
binary = "//crates/openshell-gateway:openshell-gateway-bin",
tags = ["manual"],
target_platform = ":linux_aarch64_gnu_2_28",
)
Expand All @@ -91,7 +91,7 @@ platform_transition_binary(
platform_transition_binary(
name = "openshell_gateway_macos_aarch64",
basename = "openshell-gateway",
binary = "//crates/openshell-server:openshell-gateway",
binary = "//crates/openshell-gateway:openshell-gateway-bin",
tags = ["manual"],
target_platform = "@rules_rs//rs/platforms:aarch64-apple-darwin",
)
1 change: 1 addition & 0 deletions crates/openshell-core/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ tempfile = { version = "3", optional = true }

[target.'cfg(unix)'.dependencies]
nix = { workspace = true }
rustix = { workspace = true }

[features]
default = ["telemetry"]
Expand Down
Loading
Loading