Skip to content

test(e2e): define GPU validation image artifacts #1476

Closed
#1484
Closed
test(e2e): define GPU validation image artifacts#1476
#1484
Assignees
elezar
Labels
area:buildRelated to CI/CD and buildsfeature requeststate:agent-readyApproved for agent implementationstate:pr-openedPR has been opened for this issuestate:staleInactive item at risk of automatic closure.test:e2eRequires end-to-end coveragetest:e2e-gpuRequires GPU end-to-end coveragetopic:testing

Description

@elezar
elezar
opened on May 20, 2026

Description

Define the driver-agnostic GPU validation image artifacts needed by GPU e2e tests.

Issue #1472 will add sandbox-side GPU execution validation. Those tests should consume prebuilt sandbox images rather than building validation images during the test run. This task defines the image contract, repository layout, and initial image artifacts that can validate the execution flow before a CUDA-specific reference image is enforced in GPU CI.

Context

Parent roadmap: #1444
Related GPU validation test issue: #1472

The validation image contract should stay independent of the compute driver. Docker is the first driver integration for #1472, but the same image contract should be usable later by Podman, Kubernetes, and VM-backed sandboxes.

The image base is not part of the contract. The important properties are that the image is usable directly as an OpenShell sandbox image, runs without network access inside the sandbox, and emits stable output that the e2e harness can assert.

Proposed Scope

Add GPU validation image specifications under a driver-neutral location:

e2e/gpu/
  images/
    smoke-pass/
      README.md
      Dockerfile
    smoke-fail/
      README.md
      Dockerfile
    cuda-basic/
      README.md
      Dockerfile

Start with simple fixture images to validate the basic sandbox execution flow:

  • smoke-pass: default command prints OPENSHELL_GPU_WORKLOAD_SUCCESS and exits 0.
  • smoke-fail: default command prints a stable diagnostic marker and exits non-zero.

These fixtures do not prove GPU functionality. They exist to validate image publishing, sandbox-image compatibility, and e2e assertion plumbing.

Define the first CUDA validation image contract separately in cuda-basic/. The CUDA image should eventually run a deterministic CUDA workload, emit OPENSHELL_GPU_WORKLOAD_SUCCESS on success, and return non-zero with useful diagnostics on failure.

Definition of Done

  • Document the GPU validation image contract: default command, success marker, failure behavior, and stdout/stderr expectations.
  • Add the driver-neutral e2e/gpu/images/ layout for validation image specs.
  • Add a simple always-pass fixture image that can run directly as an OpenShell sandbox image.
  • Add a simple always-fail fixture image, or document why the pass fixture is sufficient for the initial flow validation.
  • Define the initial cuda-basic validation image requirements.
  • Document how validation images are built and published.
  • Document how tests should reference published images by immutable tag or digest.

Activity

  1. elezar
    added a parent issue on May 20, 2026
  2. github-project-automation
    added this to OpenShell Roadmapon May 20, 2026
  3. github-project-automation
    moved this to Todo in OpenShell Roadmapon May 20, 2026

  4. elezar commented on May 20, 2026

    @elezar
    elezar
    on May 20, 2026
    MemberAuthor

    📋 triage-agent

    Triage Assessment

    Classification: feature-valid

    Summary

    This is a valid prerequisite feature for the GPU execution validation work in #1472. Defining driver-neutral workload image artifacts first gives the e2e tests a concrete sandbox image contract to consume, and the proposed smoke fixtures let the test plumbing be validated before a CUDA-specific reference image is enforced in CI.

    Investigation

    • There is no existing e2e/gpu/ image contract or OPENSHELL_GPU_WORKLOAD_SUCCESS fixture in the repository today, so this issue would introduce a new driver-neutral test artifact area rather than refactor existing files.
    • Existing e2e tests already exercise sandbox image selection through openshell sandbox create --from ..., so the proposed workload images can be consumed by the same mechanism that Define GPU validation tests for GPU-enabled drivers #1472 should use.
    • Current Docker build and publish helpers under tasks/docker.toml and tasks/scripts/docker-*.sh are focused on gateway/supervisor images. The implementation should either add explicit GPU workload image tasks/workflows or document the manual build/publish commands clearly enough that Define GPU validation tests for GPU-enabled drivers #1472 can reference immutable tags or digests.
    • The proposed smoke-pass and smoke-fail fixtures are useful even though they do not prove GPU functionality: they validate default-command execution, image compatibility, success-marker assertions, and failure diagnostics without depending on CUDA runtime availability.
    • The cuda-basic image should stay a contract/spec initially unless the PR also supplies a real deterministic CUDA workload and publish path. That keeps test(e2e): define GPU validation image artifacts #1476 from blocking on CUDA image infrastructure that may need separate review.

    Recommendation

    Route this as the first buildable step before #1472. Suggested labels: feature request, topic:testing, test:e2e, test:e2e-gpu, and area:build. Do not add state:agent-ready during triage; that remains the human approval step before implementation.

  5. elezar
    added
    feature request
    topic:testing
    test:e2eRequires end-to-end coverage
    test:e2e-gpuRequires GPU end-to-end coverage
    area:buildRelated to CI/CD and builds
    on May 20, 2026

  6. elezar commented on May 20, 2026

    @elezar
    elezar
    on May 20, 2026
    MemberAuthor

    🏗️ build-plan

    Implementation Plan

    Issue type: feat
    Complexity: Medium
    Confidence: High — design agreed locally; CUDA static-link validation remains the main implementation detail

    Summary

    Define GPU workload image artifacts in this repository first so #1472 has concrete images to consume, while preserving an immediate follow-up migration path to NVIDIA/OpenShell-Community. The PR will add smoke pass/fail images, a working cuda-basic image using CUDA 12.8 samples, a local Docker/Podman build task, an env manifest for built image refs, and documentation for local validation and future publishing.

    Scope

    • e2e/gpu/README.md: document the workload image contract, success/failure markers, /usr/local/bin/openshell-gpu-workload, direct container-engine validation, latest.env, and the planned OpenShell-Community migration.
    • e2e/gpu/images/smoke-pass/: add a source directory for gpu-workload-smoke-pass, based on ghcr.io/nvidia/openshell-community/sandboxes/base:latest, whose workload prints OPENSHELL_GPU_WORKLOAD_SUCCESS and exits 0.
    • e2e/gpu/images/smoke-fail/: add a source directory for gpu-workload-smoke-fail, based on the same community base image, whose workload prints OPENSHELL_GPU_WORKLOAD_FAILURE, emits a stable diagnostic, and exits non-zero.
    • e2e/gpu/images/cuda-basic/: add a source directory for gpu-workload-cuda-basic. It builds CUDA 12.8 deviceQuery and vectorAdd samples from NVIDIA/cuda-samples tag v12.8, copies the binaries into the community base final image, and runs both through /usr/local/bin/openshell-gpu-workload.
    • tasks/test.toml and a helper script under tasks/scripts/: add mise run e2e:gpu:images:build. Build all image directories containing a Dockerfile by default. Use OPENSHELL_GPU_WORKLOAD_IMAGES=smoke-pass,smoke-fail to select source directories.
    • e2e/gpu/images/.build/latest.env: generate a shell-safe env file with the image refs that were just built. Do not add latest.json in this first implementation.

    Implementation Decisions

    • Image directories use short source names: smoke-pass, smoke-fail, and cuda-basic.
    • Image tags use prefixed image names: gpu-workload-smoke-pass, gpu-workload-smoke-fail, and gpu-workload-cuda-basic.
    • Local tags use localhost/openshell/<image>:<tag>. The default tag is the source commit short SHA, with -dirty appended for dirty local-tree builds. OPENSHELL_GPU_WORKLOAD_IMAGE_TAG=<tag> can override this for manual workflows.
    • The build task uses the existing tasks/scripts/container-engine.sh helper and does not introduce another engine selector. Callers may set existing CONTAINER_ENGINE=docker|podman; unset follows the helper's current auto-detection. Consistency with the Rust e2e helper is tracked separately in fix(e2e): align container engine selection across helpers #1481.
    • The first implementation builds from local e2e/gpu/images only. Add Git-ref source support when the image definitions migrate to OpenShell-Community.
    • Add only the build task in this PR; do not add a clean task yet.
    • cuda-basic uses a CUDA 12.8 base builder image plus explicit minimal build dependencies such as build-essential, cmake, curl, g++, and cuda-nvcc-12-8. Do not use a CUDA devel image by default.
    • Prefer static CUDA runtime linkage for copied sample binaries. Do not add CUDA runtime library copying unless static linkage proves impossible.
    • Do not vendor GPU driver libraries such as libcuda.so.1; those must come from host GPU runtime/CDI injection. CUDA compat libraries are the only acceptable future exception and should not be needed for CUDA 12.8.

    Implementation Steps

    1. Add the e2e/gpu/images/ layout with SPDX headers, per-image READMEs, workload scripts, and Dockerfiles.
    2. Implement smoke-pass and smoke-fail on top of ghcr.io/nvidia/openshell-community/sandboxes/base:latest.
    3. Implement cuda-basic as a multi-stage image: CUDA 12.8 base builder, pinned CUDA samples v12.8, final OpenShell community base image, wrapper script that runs deviceQuery then vectorAdd.
    4. Tag images deterministically as localhost/openshell/<image>:<short-sha> or localhost/openshell/<image>:<short-sha>-dirty, unless OPENSHELL_GPU_WORKLOAD_IMAGE_TAG is set.
    5. Generate e2e/gpu/images/.build/latest.env with OPENSHELL_E2E_GPU_SMOKE_PASS_IMAGE, OPENSHELL_E2E_GPU_SMOKE_FAIL_IMAGE, OPENSHELL_E2E_GPU_CUDA_WORKLOAD_IMAGE, tag/source metadata, and the selected container engine.
    6. Document direct Docker/Podman validation commands, including CDI GPU validation for cuda-basic.
    7. Document future publishing under OpenShell-Community, with tests consuming immutable tags or digests.

    Test Plan

    • Unit tests: N/A — this adds image definitions, scripts, tasks, and documentation rather than Rust/Python library logic.
    • Integration checks: Run mise run e2e:gpu:images:build and verify that all selected images build and latest.env identifies the resulting image refs.
    • Direct container validation: Run smoke images with Docker or Podman and verify marker output and exit codes. On a CUDA-capable host with CDI configured, run cuda-basic directly with --device nvidia.com/gpu=all and verify deviceQuery, vectorAdd, and OPENSHELL_GPU_WORKLOAD_SUCCESS.
    • Repository checks: Run mise run pre-commit.
    • Deferred: Full OpenShell sandbox-side GPU execution remains in Define GPU validation tests for GPU-enabled drivers #1472.

    Risks & Open Questions

    • cuda-basic final stage uses the OpenShell community base image, not a CUDA runtime image. The implementation must verify that the copied sample binaries run with static CUDA runtime linkage.
    • Existing OpenShell sandbox create does not run OCI image CMD; the contract must expose /usr/local/bin/openshell-gpu-workload for OpenShell e2e tests while still setting CMD for direct container-engine validation.
    • The image definitions intentionally start in this repo for fast iteration, but should migrate to OpenShell-Community immediately after the contract proves useful.
    • LSM compatibility risk is low: the images and build tasks do not depend on host process identity, /proc/<pid>/exe, or cross-label process inspection.

    Documentation Impact

    • Add internal e2e documentation under e2e/gpu/.
    • Add per-image READMEs for local build/run behavior and CUDA samples licensing notes.
    • No published user docs under docs/ are expected for this internal e2e artifact contract.

    Revision 2 — recorded implementation decisions from local design review
    Revision 1 — initial plan

  7. elezar
    added
    state:review-readyReady for human review
    state:agent-readyApproved for agent implementation
    state:in-progressWork is currently in progress
    on May 20, 2026
  8. elezar
    self-assigned this
    on May 20, 2026
  9. elezar
    changed the title [-]test(e2e): define GPU workload image artifacts[/-] [+]test(e2e): define GPU validation image artifacts[/+] on May 20, 2026
  10. github-project-automation
    moved this from Todo to In progress in OpenShell Roadmapon May 20, 2026

  11. elezar commented on May 20, 2026

    @elezar
    elezar
    on May 20, 2026
    MemberAuthor

    🏗️ build-from-issue-agent

    Implementation Complete

    PR: #1484

    What was built

    Added local GPU workload image artifacts for smoke-pass, smoke-fail, and cuda-basic validation, plus a mise run e2e:gpu:images:build task that discovers image directories, supports Docker or Podman, tags images with the source revision, and writes latest local refs.

    Tests

    • Unit: N/A - image artifacts and task wiring only.
    • Integration: N/A.
    • E2E: Docker GPU e2e lane passed locally: 7 tests passed. Direct Docker validation also passed for smoke-pass, expected-failure smoke-fail, and cuda-basic with CDI.

    Docs updated

    • e2e/gpu/README.md
    • e2e/gpu/images/*/README.md

    The issue will auto-close when the PR is merged.

  12. elezar
    added
    state:pr-openedPR has been opened for this issue
    and removed
    state:in-progressWork is currently in progress
    state:review-readyReady for human review
    on May 20, 2026
  13. elezar
    added 2 commits that reference this issue on Jun 4, 2026

    test(e2e): add GPU workload image artifacts

    07810e6

    test(e2e): add GPU workload image artifacts

    9c42461

  14. github-actions commented on Jun 14, 2026

    @github-actions
    github-actionsbot
    on Jun 14, 2026 – with GitHub Actions

    This issue has had no activity for 14 days and is now marked stale. It may be closed in 7 days if there is no further activity. Comment or remove the state:stale label to keep it open.

  15. github-actions
    added
    state:staleInactive item at risk of automatic closure.
    on Jun 14, 2026
  16. elezar
    added a commit that references this issue on Jun 15, 2026

    test(e2e): add GPU workload image artifacts (#1484)

    8c01534
  17. elezar
    closed this as completedin #1484on Jun 15, 2026
  18. github-project-automation
    moved this from In progress to Done in OpenShell Roadmapon Jun 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area:buildRelated to CI/CD and buildsfeature requeststate:agent-readyApproved for agent implementationstate:pr-openedPR has been opened for this issuestate:staleInactive item at risk of automatic closure.test:e2eRequires end-to-end coveragetest:e2e-gpuRequires GPU end-to-end coveragetopic:testing

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions