Skip to content

test(e2e): expand supervisor middleware behavioral coverage #2794

Description

@pimlock

Description

Expand supervisor middleware end-to-end coverage beyond the authenticated HTTP redaction happy path introduced by #2784.

The suite should exercise externally observable behavior across HTTP and WebSocket middleware, including rejection, transformation, configuration validation, failure handling, chaining, and payload limits. Keep these scenarios in the dedicated supervisor middleware E2E lane so setup can be reused rather than adding one CI matrix job per case.

Context

Related to the supervisor middleware roadmap in #1733. Depends on the authenticated middleware E2E foundation in #2784.

Middleware already has substantial unit and component-level coverage, but those tests do not prove the complete gateway → sandbox supervisor → extension service → upstream path. Without E2E coverage, regressions in policy attachment, extension authentication, streaming lifecycle, error-mode handling, and forwarding can pass lower-level tests.

For normal behavioral scenarios, prefer using examples/supervisor-middleware-content-guard rather than growing a second middleware implementation solely for E2E. The current example is plaintext and does not authenticate RPCs. Reuse the TLS, CLI, smoke-script, and RPC-authorization work from 2623-authenticated-middleware-example/pimlock, but replace that branch's local verifier and direct JWT dependencies with the openshell-sdk extension feature. The example may continue using openshell-core for protobuf interfaces until those interfaces move into the SDK.

A small controllable fixture may remain appropriate for transport-only failures such as timeouts or service unavailability. Every network RPC on any fixture must authenticate before returning a result, including an unadvertised WebSocket RPC that returns Unimplemented. All upstream HTTP and WebSocket services should run locally; the automated suite must not depend on public endpoints.

Invalid signatures, audiences, expiry, and individual JWT claim-shape cases should remain SDK unit tests unless a specific cross-component boundary requires E2E coverage.

SDK Verification Follow-up

Keep #2784 focused on establishing the SDK surface. Strengthen the verifier afterward with direct negative tests for:

  • wrong issuer, wrong audience, and wrong signing key
  • unknown or missing kid
  • a non-EdDSA algorithm
  • expired tokens, future iat, and lifetimes over one hour
  • an empty jti
  • malformed, duplicate, and unsupported JWKS entries
  • a gateway token carrying a sandbox ID
  • a supervisor token without a sandbox ID

Key refresh remains deployment-owned at this stage. A future SDK lifecycle layer may manage cached-key refresh after bootstrap, but the deployment must continue supplying the cold-start trust configuration.

Definition of Done

  • The authenticated HTTP redaction happy path uses the shared example middleware, or the issue documents why a dedicated behavioral fixture remains necessary.
  • The content-guard example consumes openshell-sdk authentication instead of maintaining a local JWT verifier.
  • Every fixture RPC authenticates its caller before returning, including unsupported operation stubs.
  • Direct SDK tests cover the negative verifier and JWKS cases listed above.
  • HTTP deny prevents the request from reaching the upstream and returns the expected sandbox-visible failure.
  • WebSocket text redaction forwards the transformed message to the upstream.
  • WebSocket text denial prevents the denied message from reaching the upstream and verifies the expected connection behavior.
  • Invalid middleware configuration is rejected before the policy is persisted or used.
  • Middleware timeout or unavailability is covered for both fail_closed and fail_open.
  • An ordered middleware chain demonstrates cumulative evaluation in ascending order.
  • HTTP body and WebSocket message payload limits exercise the configured error behavior.
  • Scenarios reuse the existing supervisor middleware E2E lane and deterministic local infrastructure.
  • mise run e2e:supervisor-middleware and mise run pre-commit pass.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions