The panel test boots an image without sshserver, and a CI step's cargo finds fresh what the job's cargo built

issues/the-guest-suite-compiles-every-guest-crate-cold-on-every-run.md

@@ -0,0 +1,77 @@
+---
+status: open
+kind: tooling
+opened: 2026-10-08
+---
+
+# The guest suite compiles every guest crate cold on every run
+
+`guest.yml`'s `suite` restores the sysroot and nothing compiled with it: every
+kernel, loader, userland crate and test binary a guest boots is built in the
+job, for both architectures. Four jobs of 2026-10-08, in seconds, by each
+log's own timestamps and the suite's `building` and `testing` figures: A is
+113110642665, a merge group; B is 113190683753, a pull request that moved the
+kernel; C is 113188345621, one that moved a single userland crate; D is
+113260475768, the merge group that put the one workspace on main (#746,
+`1084ddc9a`), and the only one of the four that builds as the tree now does.
+
+| | A | B | C | D |
+|---|---|---|---|---|
+| `deps`: apt and rustup | 103 | 171 | 141 | 116 |
+| the driver's build | 146 | 102 | 148 | 125 |
+| the harness's build | 54 | 36 | 53 | 45 |
+| the suite, building | 610 | 368 | 542 | 465 |
+| the suite, testing | 136 | 124 | 131 | 133 |
+| the job | 1072 | 821 | 1036 | 911 |
+
+The `BUILT` lines behind `building`, A / B / C / D:
+
+- x86_64 kernel, loader, ROOT of tests/netcase: 138 / 84 / 123 / 89
+- x86_64 test kernel, ROOT of tests/testcases: 65 / 37 / 56 / 54
+- x86_64 ROOT of tests/metalcase: 212 / 121 / 186 / 158
+- aarch64 test kernel, loader, ROOT of tests/testcases: 110 / 70 / 97 / 91
+- aarch64 kernel: 33 / 20 / 31 / 27
+- aarch64 `mask-windows` kernel, ROOT of tests/virtsmpcase: 33 / 21 / 30 / 27
+- the test binaries and the small ROOTs together: 19 / 15 / 19 / 19
+
+Two of those costs were taken out after D, and are in none of the four. The
+metalcase line: the one guest test that booted it boots tests/panelcase, whose
+`BUILT` line read 8 s on the development machine where metalcase's read 53 s
+after the same builds. And the harness's build: in all four logs it compiles
+ring, rustls, rustls-webpki, ureq and the build system a second time, and
+`ci::dispatch` no longer hands a step the variables that made its cargo do so.
+
+What an entry of main's targets could save a run is bounded by its `building`
+figure. A run that moved the kernel compiles it five times whatever is
+restored, once per feature set and architecture: the two lines above that are
+a kernel and little else are 20 to 33 s each.
+
+Measured on the development machine before the workspace fold, when the guests
+built into four target directories that are now `target/<triple>/toyos/`, and
+not measured since:
+
+- With every target fresh the whole suite's builds took 23 s, 8 s of them an
+  AArch64 userland that was not.
+- After one whole suite the four directories were 724,457,182 B in 2953 files
+  without cargo's incremental state and 2,259,625,434 B of incremental state
+  beside it; `tar | zstd -T0` of the first was 201,025,465 B.
+
+What stands in the way:
+
+- The repository's caches are past GitHub's 10 GB and no gate reads a stored
+  size (`issues/the-host-caches-limit-reaches-the-10-gb-only-through-one-measured-ratio.md`).
+- Only main's scope is read by every ref, so the writer is a run on main, and
+  no pull request can show the saving before the writer has landed.
+- A restored target is trusted by content or not at all: `src/cicache.rs` is
+  the tree's one reader that does.
+
+Not taken: `CARGO_INCREMENTAL=0` in the job. A cold
+x86-64 kernel took 43 s of CPU and left 35 MiB without it, 62 s and 339 MiB with
+it, on the development machine under load. It also turns rustc's MIR inliner
+on (the root `Cargo.toml`, `[profile.toyos]`), so the job would boot other bytes
+than `cargo run` builds.
+
+Owner: the guest job (`src/ci.rs`'s `guest`, `.github/workflows/guest.yml`).
+
+**Exit**: `guest / suite` on a pull request that moves no guest source prints
+a `building` figure under 60 s.

src/build.rs

@@ -3104,6 +3104,7 @@ mod tests {
         "tests/metalcase/system.toml",
         "tests/metaldevicecase/system.toml",
         "tests/netcase/system.toml",
+        "tests/panelcase/system.toml",
         "tests/proctreecase/system.toml",
         "tests/testcases/system.toml",
         "tests/virtjobcase/system.toml",

src/ci.rs

@@ -72,6 +72,7 @@ pub fn dispatch(root: &Path, args: &[String]) {
         eprintln!("Error: {refusal}\n{USAGE}");
         std::process::exit(2);
     });
+    forget_own_package();
     let steps = match &job {
         Job::Host => host(root),
         Job::Seal => seal(root),
@@ -94,6 +95,22 @@ pub fn dispatch(root: &Path, args: &[String]) {
     }
 }
 
+/// Take what `cargo run` told the driver about its own package out of what
+/// every step inherits, before any thread.
+///
+/// Cargo judges a build script's `rerun-if-env-changed` by its own
+/// environment, and ring's names these: left in, a step's cargo finds ring
+/// stale against the build the job's cargo made without them, and builds it,
+/// its dependents and this crate a second time.
+fn forget_own_package() {
+    for (name, _) in std::env::vars_os() {
+        let own = name.to_str().is_some_and(|n| n.starts_with("CARGO_PKG_") || n.starts_with("CARGO_MANIFEST_"));
+        if own {
+            std::env::remove_var(name);
+        }
+    }
+}
+
 /// One step's verdict: a sentence on green, the refusal on red.
 struct Step {
     label: String,
@@ -1052,6 +1069,49 @@ mod tests {
         assert!(!rustc.contains("-C incremental"), "{rustc}");
     }
 
+    /// A build script naming what `cargo run` hands the driver is fresh for a
+    /// step's cargo once the job's cargo, which ran without them, has built
+    /// it. The driver is a process of its own, as above.
+    #[test]
+    fn a_steps_cargo_finds_fresh_what_the_jobs_cargo_built() {
+        const NAMED: [&str; 2] = ["CARGO_PKG_NAME", "CARGO_MANIFEST_DIR"];
+        let fixture = toyos_tmpdir::TempDir::new("ci-own-package");
+        std::fs::create_dir(fixture.join("src")).unwrap();
+        let manifest = "[package]\nname = \"one\"\nversion = \"0.1.0\"\nedition = \"2021\"\n\n[workspace]\n";
+        std::fs::write(fixture.join("Cargo.toml"), manifest).unwrap();
+        std::fs::write(fixture.join("src/lib.rs"), "").unwrap();
+        let script = NAMED.map(|name| format!("    println!(\"cargo:rerun-if-env-changed={name}\");\n")).concat();
+        std::fs::write(fixture.join("build.rs"), format!("fn main() {{\n{script}}}\n")).unwrap();
+        let mut jobs = Command::new("cargo");
+        jobs.args(["build", "--offline"]).current_dir(&fixture);
+        for name in NAMED {
+            jobs.env_remove(name);
+        }
+        let out = jobs.output().expect("run cargo");
+        assert!(out.status.success(), "{}", String::from_utf8_lossy(&out.stderr));
+        let out = crate::buildlock::tests::rerun("ci::tests::a_steps_cargo")
+            .env(FIXTURE, fixture.path())
+            .output()
+            .expect("run the driver");
+        let said = String::from_utf8_lossy(&out.stdout).into_owned() + &String::from_utf8_lossy(&out.stderr);
+        assert!(out.status.success() && said.contains("test result: ok. 1 passed"), "{said}");
+    }
+
+    #[test]
+    #[ignore = "the driver of the test above; never runs on its own"]
+    fn a_steps_cargo() {
+        let fixture = PathBuf::from(std::env::var_os(FIXTURE).unwrap_or_else(|| panic!("run without {FIXTURE}")));
+        assert!(std::env::var_os("CARGO_PKG_NAME").is_some(), "cargo names no package to this process");
+        forget_own_package();
+        let out = Command::new("cargo")
+            .args(["build", "-v", "--offline"])
+            .current_dir(&fixture)
+            .output()
+            .expect("run cargo");
+        let said = String::from_utf8_lossy(&out.stderr);
+        assert!(out.status.success() && said.contains("Fresh one v0.1.0"), "{said}");
+    }
+
     fn repo_root() -> PathBuf {
         PathBuf::from(env!("CARGO_MANIFEST_DIR"))
     }

tests/metalcase/system.toml

@@ -1,13 +1,10 @@
-# The metal-sim boot: what `Profile::Metal` puts in front of userland.
-#
 # soundserver, netstack and sshserver are here for exactly the reason they have no device:
 # their graceful exit is half of what this config certifies.
 # sshserver is the one whose absence *is* the device absence — it has no device of
 # its own and reaches the machine's shape only through netstack.
 # test-runner makes the boot announce itself, so the harness waits for a
 # marker instead of guessing from pixels — and it is what lets an in-guest
-# binary run on the machine shape that gets flashed. Under `mute` it simply
-# parks on a console that never produces a byte.
+# binary run on the machine shape that gets flashed.
 
 assets = ["assets"]
 

tests/panelcase/system.toml

@@ -0,0 +1,20 @@
+# The boot `screen_fatal_halt_composited` reads the panel of: a compositor
+# holding the scanout when the kernel dies, on `Profile::Metal`'s machine.
+#
+# Not tests/metalcase: that image's sshserver is most of what it compiles, and
+# no guest speaks SSH.
+
+assets = ["assets"]
+
+[boot]
+start = ["logkeeper", "compositor"]
+
+# `every_boot_config_runs_logkeeper` is what refuses a boot config without it.
+[programs.logkeeper]
+service = true
+syscap = ["logread"]
+
+[programs.compositor]
+service = true
+serves = ["compositor"]
+devices = ["framebuffer", "keyboard", "mouse"]

tests/toyos.rs

@@ -2591,7 +2591,7 @@ fn run_screen_test(name: &str, profile: qemu::Profile, test_config: &Path) -> Re
             // Driven by `metal-panic-probe`, which is the same kernel the owner
             // flashes: a gate that staged this with SYS_DEBUG would certify a
             // path his image does not contain.
-            let config = Path::new(env!("CARGO_MANIFEST_DIR")).join("tests/metalcase");
+            let config = Path::new(env!("CARGO_MANIFEST_DIR")).join("tests/panelcase");
             let options = BootOptions {
                 profile,
                 smp: 8,