Build caching for the deploy actions (sccache) — sub-plan 1: container digest-pin gate - #347
Draft
aram356 wants to merge 9 commits into
Draft
Build caching for the deploy actions (sccache) — sub-plan 1: container digest-pin gate#347aram356 wants to merge 9 commits into
aram356 wants to merge 9 commits into
Conversation
First increment of the build-caching feature (sub-plan 1, Task 1) per docs/specs/edgezero-deploy-build-caching.md (v6.14) and docs/superpowers/plans/2026-08-20-build-cache-container.md. The pinned build container's platform-id keys the whole feature on a sha256 manifest digest, so check-image-pin.sh fails closed on a tag, missing digest, or malformed JSON. Colocated unit test: 6 cases, all green; shellcheck clean.
aram356
marked this pull request as draft
August 27, 2026 04:46
…tion) The build-caching design spec was authored via the brainstorming flow, whose specs live under docs/superpowers/specs alongside their plans (the container sub-plan is already in docs/superpowers/plans). Relocate it there from docs/specs and update the two references (the plan's Spec: link and the validator's comment). Vitepress builds clean; the validator test stays green.
…nvention Siblings in docs/superpowers/specs are dated YYYY-MM-DD-<topic>-design.md; rename edgezero-deploy-build-caching.md to 2026-08-20-edgezero-deploy-build-caching-design.md (its authoring/plan date) and update the plan link + validator comment.
…ion gap)
The validator used jq -r, which coerces a numeric field to a string, so a
{"repository": 123, "tag": 1} would pass despite the contract requiring strings.
Assert the JSON type is string for repository, tag, and digest before the value
checks, and add a wrong-type test case. 7/7 green, shellcheck clean.
Harden the sccache design toward plan-ready. env: add PATH and RUSTUP_HOME and an absolute RUSTC_WRAPPER so rustc starts under env -i (rustup-image layout) (1). Narrow the sccache correctness claim (it hashes dep-info/args/deps/env/cwd) and make the undeclared-input proc-macro/build.rs risk an explicit cache opt-in (2). Bounded, collision-free generation: run_id-run_attempt-artifact, SCCACHE_CACHE_SIZE 2G, sccache --stop-server before save, aggregate bounded by GitHub's LRU (3). A complete FIXED mount table with a constant /work/app cwd so sccache's cwd hash is stable across host paths (4). Prove the writable /work/app is a faithful copy (content/modes/symlinks/ submodules, hardlinks broken) and state build/deploy use separate container instances (5). Warm test via sccache --show-stats ONLINE (dependency sources are not cached, so the network cannot be disabled for the fetch) (6). Public, anonymously-fetchable sources only; private auth is out of scope (7). RFC 8785 (JCS) canonical JSON and ustar-only archive with binary-size equality (8). Full 40-hex app-ref and length-framed hash encodings with golden vectors (9). Hardened validator smoke: --cap-drop=ALL, no-new-privileges, memory/pids/ timeout (10). Plan: fix the first-publish deadlock (authenticated smoke in the workflow; anonymous pull is the operator's post-make-public step) and drop the stale four-root-prune language (11). Design only.
The validator accepted any non-empty repository, so a pin naming a foreign repository could become platform-id. Require repository == the canonical ghcr.io/stackpop/edgezero-build-app-cli and add a foreign-repository reject case (8/8). A trusted digest is only trustworthy for the repository we publish.
…findings
Stable host cache path: actions/cache folds the on-disk path into the cache
version, so a per-run mktemp path forces permanent misses; use one fixed
${RUNNER_TEMP}/edgezero-sccache-v1, emptied before restore, mounted at
/work/sccache (1). Whole-repo /work/repo working copy with the compile cwd at the
relative working-directory, so a nested working-directory (apps/api under a parent
workspace) keeps its enclosing Cargo config and sibling path-deps; the flattened
/work/app is gone (2). Frozen source: git-ignored files excluded from the copy and
initialized submodules validated, and the SAME copy is reused across the separate
build/deploy container instances so build outputs reach deploy as derived state (3).
Storage restated as repository-global LRU that can evict unrelated caches and may be
billable, not family-local (4). Generation keyed on an app-cli-artifact unique across
every cache-writing invocation (fail-closed on a detectable collision), with concurrent
lineages forked, not merged (accepted) (5). PATH includes /usr/local/bin where Fastly
and sccache live; enumerated compile/validation/deploy env profiles listing EDGEZERO_*
by name, not the namespace (6). app-checkout-token assigned to the host-side app-repo-id
API check and barred from containers/copies/artifacts/caches (7). Exact byte contracts:
length-framed <len>:<bytes> hash encoding with normalized relative paths, normalized
ustar headers (zero mtime/uid/gid, fixed names), and abi as recomputed ELF metadata
(machine/interp=null-if-static/direct-DT_NEEDED; transitive resolved, dlopen out of
scope) (8). sccache undeclared-input risk stated as accepted (no proc-macro
input-declaration mechanism exists); fail-cold restore/audit/read failures; skip-save on
--stop-server failure (9). Plan: two-tier pin policy (major action tags per the repo's
own check-action-pins gate, image digests) resolving the apparent checkout@v7
inconsistency; validator canonical-repo requirement reflected; image.json rigor scoped
(the JCS/schema/dup-key provenance machinery is for produced artifacts, sub-plan 3) (10).
Design only.
…findings Reusable workflow is BUILD-ONLY: no provider inputs, emits the artifact plus every ExpectedIdentity field as outputs; the shared-copy build->deploy lifecycle moves to the consumer's own deploy job, resolving the one-container-builds-and-deploys contradiction (1). Undeclared-input staleness restated as may-pass-every-downstream- check: a stale proc-macro result can be internally consistent and pass digest/ELF/--help, so provenance/ABI is not a staleness safety net (2). A deploy-compile env/mount profile: fastly compute deploy compiles the wasm, so it carries pinned Rustup/Cargo + fresh target/cargo but NO RUSTC_WRAPPER, SCCACHE_DIR, or cache save, with the token (3). The shared writable copy's tracked files/modes/symlinks/gitlinks are re-verified before the token-bearing deploy-compile; derived state only in declared output paths, so a build.rs that mutates tracked source fails closed (4). Per-operation mount profiles instead of one common table: the unauthenticated validator --help smoke gets no writable repo/target/ cargo/sccache; only cached-compile mounts sccache (5). Cache holds compiled results incl. replayed compiler stdout/stderr (warnings, paths, source excerpts), widening cross-repo disclosure to build diagnostics (6). Cross-repo topology predicates split: deployer ref, called-workflow SHA, and app-checkout SHA checked separately (deployer HEAD != app SHA is fine); path deps permitted anywhere beneath git-root, only a git-root escape rejected (7). Byte-exact path hashing: drop NFC (Linux/Git paths are byte strings; NFC vs NFD are distinct files), define the '.' root, reject non-UTF-8; NFC/NFD golden vectors that must differ (8). job.check_run_id generation, SCCACHE_IGNORE_SERVER_IO_ERROR=1 (per-object IO error -> miss not cold reset), name reserved before save, compiler errors never retried; full recursive non-sparse checkout with LFS/filter content materialized; wall-time is telemetry (11,12 spec parts). Plan: baked project-owned validator (JCS/dup-key/schema/ ustar/ELF, smoke-tested at publish) since jq/tar cannot do it (10); SHA-pin the actions in the write-privileged publish workflow (contents/packages/PRs write) while leaving the repo-wide migration of low-privilege references as a separate decision (9); single-manifest check rejects a one-entry OCI index (leaf manifest required) and the anonymous-pull check reads the merged digest, not the placeholder (12 plan parts). Design only.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Starts the build-caching implementation for the deploy actions (design merged in #316). Tracks #346. This PR lands the feature in dependency order behind the pinned build container; the first increment is the container's fail-closed digest-pin validator.
Spec:
docs/superpowers/specs/2026-08-20-edgezero-deploy-build-caching-design.md(v6.14 — sccache pivot). Plan:docs/superpowers/plans/2026-08-20-build-cache-container.md(sub-plan 1 of 4).In this PR
.github/docker/build-app-cli/check-image-pin.sh— fail-closed validator that the build container'simage.jsonis pinned by asha256manifest digest (not a mutable tag), since the whole feature keysplatform-idon that digest. Rejects a tag, a short/missing digest, a missing repository, and malformed JSON (fail-closed)..github/actions/deploy-core/tests/check-image-pin.test.sh— 6 colocated unit cases, all green;shellcheck -S warningclean.Nothing here is wired into the runtime yet (no
image.json, Dockerfile, or workflow), so this is CI-neutral and self-contained.Next increments (same PR, dependency order)
wasm32-wasip1+ pinnedsccache+ Fastly CLI), the GHCR verify-by-digest → reviewableimage.jsonPR publish workflow, and wiring the digest pin into the contract suite.Notes
The design has been through extensive hardening review and continues to evolve; this PR intentionally starts from the most approach-stable, self-contained piece so review can proceed incrementally rather than against the whole feature at once.