Skip to content

CI: give Windows a persistent Cargo target and enable sccache - #5946

Draft
jdetter wants to merge 3 commits into
masterfrom
tars/windows-cache-parity
Draft

jdetter wants to merge 3 commits into
masterfrom
tars/windows-cache-parity

Conversation

@jdetter

@jdetter jdetter commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Description of Changes

Windows CI rebuilds everything from scratch on every run, and the cause is a single else branch.

The set-cargo-target-dir anchor sends Linux to $HOME/actions-runner/_work/target — outside the workspace, so it survives — and sends everything else to $GITHUB_WORKSPACE/target, which actions/checkout wipes at the start of each run. Windows also never had sccache enabled at all: the Configure sccache step is gated if: runner.os == 'Linux'.

This change points Windows at C:/actions-runner/_work/target and enables sccache there, mirroring the Linux setup already on master. Three steps touched, +29/−5:

  1. &set-cargo-target-dir — adds a Windows branch. Gated on the runner work directory existing, so GitHub-hosted windows-latest jobs (the update-flow matrix) keep their current workspace-local behaviour. macOS unchanged.
  2. Configure sccache — widened from runner.os == 'Linux' to also cover Windows, with SCCACHE_DIR=C:/actions-runner/_work/sccache.
  3. Show sccache statistics — widened the same way.

These are shared YAML anchors, so the Linux path is worth checking explicitly in review; it is unchanged in effect, but the diff touches code Linux jobs execute.

RUSTC_WRAPPER is set to an explicit C:/sccache/sccache.exe rather than the output of command -v. These steps run under Git Bash, whose command -v returns an MSYS path (/c/sccache/sccache.exe) that Cargo — a native Windows process — cannot execute. The command -v guard is kept purely to detect whether the runner image was warmed, so an unwarmed image degrades to a cold but working build rather than failing outright.

The Windows-only actions/cache entries for rusty_v8 objects and vendored OpenSSL are deliberately left in place for this iteration, as cheap insurance while the warmed image is unproven. They are the obvious follow-up removal once hit rates are demonstrated.

Measured on the golden image, so expectations are calibrated

Comparing the warm's population pass against its seed pass (cold target, warm sccache — roughly what a CI job sees):

build population seed
release CLI + standalone 441s 406s
smoketest archive 72s 34s
package build (msvc target) 448s 425s

The read-only verifier reports 100% hit rate, but on a small cacheable set — sccache logged 930 calls non-cacheable for crate-type during the warm. sccache cannot cache linking, build-script output, or proc macros, and on Windows the release link of spacetimedb-cli and spacetimedb-standalone dominates. The larger win should come from the warm target directory carrying rusty_v8 and vendored OpenSSL, which sccache structurally cannot help with.

Measured staging qualification

Measured in Actions run 35128480324 using warmed image github-windows-runner-vm26.qcow2. The baseline is the median job wall time from three successful master runs on the production vm25 pool. Durations exclude queue time.

Windows job master median vm26 staging change
Upload build artifacts 5m48s 3m15s 44.0% faster
Build smoketests 3m33s 3m53s 9.4% slower
Smoketests 1/4 9m58s 8m43s 12.5% faster
Smoketests 2/4 11m08s 9m16s 16.8% faster
Smoketests 3/4 13m08s 11m22s 13.5% faster
Smoketests 4/4 9m48s 7m55s 19.2% faster
Sum of job wall times 53m23s 44m24s 16.8% faster (8m59s saved)

For a production-sized pool, the Windows pipeline critical path is upload + smoketest build + the slowest partition. Using the same medians, that falls from 22m29s to 18m30s, a 17.7% reduction. The staging pool had only one Windows VM, so its observed queue-to-finish wall clock is not representative of production concurrency.

The release artifact build is the strongest cache signal: it beat the fastest of the three baseline runs (3m15s versus 3m45s). The smoketest build's 20-second regression remains inside the baseline's existing 3m11s–4m08s spread. Partition jobs do not compile Rust, so their gains should be treated as runner/run variance rather than attributed to the Cargo cache.

sccache itself contributed almost nothing:

  • Upload build: 43 compile requests, 1 hit, 37 misses — 2.63% hit rate.
  • Smoketest build: 614 compile requests, 1 hit, 434 misses — 0.23% hit rate.

The meaningful improvement therefore comes from the baked Cargo target avoiding compiler requests entirely, not from sccache serving objects. This is consistent with the image being warmed at commit 37476b30 while this branch builds a later commit, and with workspace-owned artifacts being deliberately pruned before sealing.

Before merge: revert the temporary commit 703a08cc1 that points these jobs at spacetimedb-windows-runner-staging. The permanent workflow must target spacetimedb-windows-runner after vm26 is promoted to live.

API and ABI breaking changes

None. CI configuration only — no crate, module, client SDK, or on-disk format is touched.

Rollback safety impact

n/a

Expected complexity level and risk

1 - this is just a CI change

Testing

The warm and verification have been exercised end to end on the Windows golden image (details and logs in infra#1349), and the complete Windows path passed on the staging vm26 runner in Actions run 35128480324.

  • Warm script runs to completion on the golden image (32m 49s) and the read-only verifier reports All warmed sccache probes passed
  • Confirmed the warm resolves toolchain 1.93.0 from rust-toolchain.toml rather than the image default 1.94.1 — warming on the wrong compiler yields a 100% miss rate while still appearing to succeed
  • ci.yml parses as valid YAML, and the Linux branches of all three modified steps are unchanged in effect
  • With vm26 on spacetimedb-windows-runner-staging, all six Windows jobs passed and the measured results are recorded above
  • Reviewer: confirm the Linux jobs in this PR's CI run behave identically to master
  • Reviewer: decide whether to retain sccache on Windows given the measured 0.23%–2.63% hit rates
  • Before merge: revert temporary staging-routing commit 703a08cc1

Authored by OpenClaw (an AI agent) on John Detter's infrastructure, at his direction. Reviewed by no human yet — please read it as you would any unreviewed change.

Windows CI rebuilds everything from scratch on every run for one reason: the
set-cargo-target-dir anchor sends Linux to $HOME/actions-runner/_work/target,
outside the workspace, and everything else to $GITHUB_WORKSPACE/target --
which checkout wipes each run. Windows also never had sccache enabled at all.

Point Windows at C:/actions-runner/_work/target and enable sccache there,
mirroring the Linux setup already on master. The Windows branch is gated on
the runner work directory existing, so GitHub-hosted windows-latest jobs keep
their current workspace-local behaviour.

RUSTC_WRAPPER is set to an explicit C:/sccache/sccache.exe rather than the
result of `command -v`: these steps run under Git Bash, whose `command -v`
returns an MSYS path (/c/sccache/sccache.exe) that Cargo, a native Windows
process, cannot execute. The `command -v` guard is kept only to detect
whether the runner image was warmed, so an unwarmed image degrades to a cold
but working build instead of failing.

The Windows-only actions/cache entries for rusty_v8 and vendored OpenSSL are
deliberately left in place for this iteration.

Requires a runner image warmed by warm-cache-windows.ps1 in
clockworklabs/infra. Without it these paths are simply empty and builds
behave as they do today.
@jdetter
jdetter marked this pull request as draft September 16, 2026 04:35
Points upload-build-artifacts-windows, smoketest_build_windows and
smoketest_partitions_windows at spacetimedb-windows-runner-staging so this PR
can be qualified against runner image vm26, which carries the warmed sccache
store and dependency-only Cargo target.

This commit exists only to measure the effect and MUST be reverted before
merge. Leaving it in would send all production Windows CI to the staging pool.

The windows-latest matrix in the update-flow job is deliberately untouched --
it runs on GitHub-hosted runners and has no warmed image.

Revert with:
  git revert <this commit>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant