Skip to content

Run bounded PHPUnit plans in one disposable runtime #1920

Description

@chubes4

Problem

The public PHPUnit recipe executes one suite per workspace lifecycle. A large suite matrix therefore creates a fresh Playground runtime and disposable database service for every entry. This is correct for isolated one-off execution but prohibitively expensive for CI-scale plans where entries intentionally share one immutable source tree and service allocation while isolating their process/database identities.

The immediate consumer is a 680-entry PHPUnit parity manifest, but the owning primitive must remain generic and contain no WPCOM or TeamCity knowledge.

Expected contract

WP Codebox should accept a deterministic PHPUnit execution plan and run it inside one prepared workspace recipe lifecycle:

  1. Materialize the workspace/runtime once.
  2. Start declared disposable services once.
  3. Execute plan entries with configurable bounded concurrency.
  4. Give every entry its own argv, environment overrides, timeout, process identity, artifact namespace, and deterministic input index.
  5. Preserve stable output ordering independent of completion order.
  6. Isolate entry failures/timeouts so remaining entries complete unless an explicit fail-fast policy is selected.
  7. Stop services and dispose the runtime exactly once after all admitted entries terminalize.
  8. Return a typed aggregate with per-entry exit status, timeout/early-exit state, stdout/stderr refs, duration, and aggregate counts.
  9. Resume/retry selected failed entries without rerunning successful entries when the caller supplies a prior aggregate.

Safety and compatibility

  • Keep production credentials, APIs, data, and writes outside the contract.
  • Preserve the existing single PHPUnit recipe behavior and schemas.
  • Make concurrency explicit and bounded; default to one.
  • Maintain existing secret redaction and reviewer-safe artifact projection.
  • Keep database-index isolation caller-supplied through per-entry environment/argv rather than adding product-specific policy.

Acceptance criteria

  1. Deterministic lifecycle coverage proves one runtime prepare, one service start, one service stop, and one runtime disposal for a multi-entry plan.
  2. Deterministic scheduler coverage proves the concurrency ceiling, stable result ordering, failure isolation, timeout classification, and selective retry.
  3. A real disposable MariaDB integration executes at least two PHPUnit entries against one service allocation with distinct database identities.
  4. The CLI/API/schema contract is documented and validated.
  5. Existing build, schema, public API, and PHPUnit recipe tests remain green.

Tracked downstream by https://linear.app/a8c/issue/BET-12/local-wordpresscom-development-with-wpcom-codebox.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions