From f24a492c0e4cf64d7381a9deebf47ea29ddf24c1 Mon Sep 17 00:00:00 2001 From: japabu Date: Mon, 28 Sep 2026 09:35:45 +0200 Subject: [PATCH 1/6] Record the Linux LLVM-build bar on the T14 in the self-hosting track The goal is ToyOS building LLVM on the T14 as fast as Linux does; the measurement existed only in a private scratch file. Put the command, source commit, configuration, conditions and numbers into the track where the goal lives, with the exit that follows from it. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01W6rME2DoqwjcYFStYHHY4j --- issues/build/toyos-builds-itself.md | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/issues/build/toyos-builds-itself.md b/issues/build/toyos-builds-itself.md index f98f9d5a65c..1a9405facad 100644 --- a/issues/build/toyos-builds-itself.md +++ b/issues/build/toyos-builds-itself.md @@ -53,6 +53,34 @@ enough for an LLVM build tree memory beyond what 2 MiB process pages allow (`issues/kernel/process-memory-is-2-mib-pages-and-that-caps-the-process-count.md`). +**The bar: Linux building the same LLVM on the same T14.** Measured +2026-09-27: stage 2, the pinned commit's own stage-1 clang+lld building +clang+lld from the same source, at `rust-lang/llvm-project` commit +`52ed14fcd56afc30f9cccd8ca8ce237c2eef7e04`, Release, targets X86 and +AArch64: + +``` +cmake -S src/llvm -B s2 -G Ninja -DCMAKE_BUILD_TYPE=Release \ + -DLLVM_ENABLE_PROJECTS=clang;lld -DLLVM_TARGETS_TO_BUILD=X86;AArch64 \ + -DLLVM_ENABLE_ASSERTIONS=OFF -DLLVM_INCLUDE_TESTS=OFF \ + -DLLVM_INCLUDE_BENCHMARKS=OFF -DLLVM_INCLUDE_EXAMPLES=OFF \ + -DCMAKE_C_COMPILER=s1/bin/clang -DCMAKE_CXX_COMPILER=s1/bin/clang++ \ + -DLLVM_ENABLE_LLD=ON +ninja -C s2 -j8 clang lld +``` + +Conditions: Ubuntu 24.04.4, kernel 6.8.0-142-generic, ext4 on LVM; +i5-1135G7, 8 threads, 16 GB RAM; platform_profile=performance (RAPL PL1 +20 W, PL2 64 W), on AC, intel_pstate active. Result: wall 45:08.77, user +21018.09 s, sys 573.98 s, 797% CPU, max RSS 1,121,732 KB, 160,044,528 +minor faults, 874,817 involuntary and 43,204 voluntary context switches, +4,146,616 filesystem output blocks. Stage 1 (gcc building the stage-1 +clang+lld) ran under a mixed power profile and is excluded from the +baseline. + +*Exit*: a ToyOS build of the same commit, same configuration, on the +same T14, at or under 45:08.77 wall. + **What M2 must do to delete toyos-ld.** It is frozen and links nothing the host builds; what keeps it is that it is the one linker a ToyOS process can run, shipped as `/system/bin/toyos-ld` by `system.toml`'s `[programs]` row and named From b0b7f1e3f5d21449d26af9911e9d5b54ae6fa967 Mon Sep 17 00:00:00 2001 From: japabu Date: Mon, 28 Sep 2026 12:14:33 +0200 Subject: [PATCH 2/6] Define the T14 LLVM bar as a recipe with a measured spread The bar in issues/build/toyos-builds-itself.md was one sample, its command block did not configure what the measurement configured, and its exit fixed neither the building compiler nor the power envelope. - The command block is now the measurement's own lines, run in bash, where the quoted $CONF keeps `clang;lld` and `X86;AArch64` as single arguments. On the T14, the block's W, CONF and s2-configure lines wrote a cache with LLVM_ENABLE_PROJECTS=clang;lld, LLVM_TARGETS_TO_BUILD=X86;AArch64, the stage-1 clang and LLD. The previous block's text, run the same way, configured clang alone for every target with the system gcc and exited 127 on `lld`. - The bar is a recipe: source commit, stage-1 compiler and configure, stage-2 configure, the ninja line alone as the timed span, a warm cache, the power envelope and the machine. - Three more stage-2 runs ran back to back on AC. A root sampler read platform_profile, the RAPL limits through the MSR and MMIO, the energy counter, governor, EPP, AC and package temperature every 60 s and at each run's start and end. Walls: 44:01.03, 45:08.78 and 45:08.76 next to the earlier 45:08.77. MSR PL1/PL2 read 64/64 W at every sample. MMIO PL1 read 20 W at every sample except the first four of the run that started cold, where it read 64 W, which is the 44:01.03 run. - The exit requires a ToyOS run to match every element of the recipe and to finish at or under the fastest sample, 44:01.03. - Removed: the date, the Linux-only counters and the stage-1 exclusion sentence. RAM is the measured 16476082176 B. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01W6rME2DoqwjcYFStYHHY4j --- issues/build/toyos-builds-itself.md | 65 +++++++++++++++++++---------- 1 file changed, 43 insertions(+), 22 deletions(-) diff --git a/issues/build/toyos-builds-itself.md b/issues/build/toyos-builds-itself.md index 1a9405facad..26b171b551d 100644 --- a/issues/build/toyos-builds-itself.md +++ b/issues/build/toyos-builds-itself.md @@ -53,33 +53,54 @@ enough for an LLVM build tree memory beyond what 2 MiB process pages allow (`issues/kernel/process-memory-is-2-mib-pages-and-that-caps-the-process-count.md`). -**The bar: Linux building the same LLVM on the same T14.** Measured -2026-09-27: stage 2, the pinned commit's own stage-1 clang+lld building -clang+lld from the same source, at `rust-lang/llvm-project` commit -`52ed14fcd56afc30f9cccd8ca8ce237c2eef7e04`, Release, targets X86 and -AArch64: +**The bar: Linux on this T14 building the same LLVM.** It is a recipe, and a +ToyOS run matches every element of it: + +- *Source*: `rust-lang/llvm-project` at + `52ed14fcd56afc30f9cccd8ca8ce237c2eef7e04`. +- *Stage 1*: `gcc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0`, cmake 3.28.3 and + ninja 1.11.1 build clang+lld with the `s1` lines below: Release + (`-O3 -DNDEBUG`), `LLVM_ENABLE_LTO=OFF`, `LLVM_BUILD_INSTRUMENTED=OFF`, no + profile data. +- *Stage 2*: that clang+lld builds clang+lld from the same source with the + `s2` lines below, configured afresh. +- *Timed span*: the last line's `ninja` alone. +- *Cache*: warm. The source and stage 1 were read or written just before the + span, and `File system inputs` was 16 in the 45:08.77 run and 0 in the + others. +- *Power envelope*, read back every 60 s and at the start and end of the + three runs that sampled it: on AC; `platform_profile` `performance`; + intel_pstate active, governor `powersave`, EPP `balance_performance`. RAPL + through the MSR: PL1 64 W with a 27983872 µs window, PL2 64 W. RAPL through + MMIO: PL2 64 W, and PL1 20 W at every sample but the first four of the one + run that started cold, which read 64 W up to 3:01 into it at up to 100 °C. + Mean package power over a run was 19.96 W to 21.22 W. +- *Machine*: i5-1135G7, 8 threads, 16476082176 B RAM; Ubuntu 24.04.4, kernel + 6.8.0-142-generic, ext4 on LVM. + +The lines, run in bash, which splits `$CONF`: ``` -cmake -S src/llvm -B s2 -G Ninja -DCMAKE_BUILD_TYPE=Release \ - -DLLVM_ENABLE_PROJECTS=clang;lld -DLLVM_TARGETS_TO_BUILD=X86;AArch64 \ - -DLLVM_ENABLE_ASSERTIONS=OFF -DLLVM_INCLUDE_TESTS=OFF \ - -DLLVM_INCLUDE_BENCHMARKS=OFF -DLLVM_INCLUDE_EXAMPLES=OFF \ - -DCMAKE_C_COMPILER=s1/bin/clang -DCMAKE_CXX_COMPILER=s1/bin/clang++ \ - -DLLVM_ENABLE_LLD=ON -ninja -C s2 -j8 clang lld +W=$HOME/llvm-baseline; SHA=52ed14fcd56afc30f9cccd8ca8ce237c2eef7e04 +mkdir -p $W; cd $W +git init -q src && git -C src remote add origin https://github.com/rust-lang/llvm-project.git +git -C src fetch -q --depth 1 origin $SHA && git -C src checkout -q FETCH_HEAD +CONF="-G Ninja -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_PROJECTS=clang;lld -DLLVM_TARGETS_TO_BUILD=X86;AArch64 -DLLVM_ENABLE_ASSERTIONS=OFF -DLLVM_INCLUDE_TESTS=OFF -DLLVM_INCLUDE_BENCHMARKS=OFF -DLLVM_INCLUDE_EXAMPLES=OFF" +rm -rf s1; cmake -S src/llvm -B s1 $CONF -DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++ > s1-config.log +/usr/bin/time -v -o s1-time.txt ninja -C s1 -j8 clang lld > s1-build.log +rm -rf s2; PATH=$W/s1/bin:$PATH cmake -S src/llvm -B s2 $CONF -DCMAKE_C_COMPILER=$W/s1/bin/clang -DCMAKE_CXX_COMPILER=$W/s1/bin/clang++ -DLLVM_ENABLE_LLD=ON > s2-config.log +PATH=$W/s1/bin:$PATH /usr/bin/time -v -o s2-time.txt ninja -C s2 -j8 clang lld > s2-build.log ``` -Conditions: Ubuntu 24.04.4, kernel 6.8.0-142-generic, ext4 on LVM; -i5-1135G7, 8 threads, 16 GB RAM; platform_profile=performance (RAPL PL1 -20 W, PL2 64 W), on AC, intel_pstate active. Result: wall 45:08.77, user -21018.09 s, sys 573.98 s, 797% CPU, max RSS 1,121,732 KB, 160,044,528 -minor faults, 874,817 involuntary and 43,204 voluntary context switches, -4,146,616 filesystem output blocks. Stage 1 (gcc building the stage-1 -clang+lld) ran under a mixed power profile and is excluded from the -baseline. +The timed span's wall over four runs: 44:01.03, 45:08.76, 45:08.77 and +45:08.78; min 44:01.03, median 45:08.765, max 45:08.78. The 44:01.03 run is +the one that started cold. The 45:08.77 run's envelope was not read back. -*Exit*: a ToyOS build of the same commit, same configuration, on the -same T14, at or under 45:08.77 wall. +*Exit*: a ToyOS run on this T14 that matches every element of the recipe +finishes at or under the fastest Linux sample, 44:01.03. Its clang+lld is +`52ed14fc` built by the stage-1 recipe, the span is the stage-2 `ninja` +alone after a fresh configure, the cache is warm, and it runs on AC with the +MSR and MMIO PL1 and PL2 above, read back during the run. **What M2 must do to delete toyos-ld.** It is frozen and links nothing the host builds; what keeps it is that it is the one linker a ToyOS process can run, From 4fe59fd8d702bcc3ece8612f25613e67d4f22eb5 Mon Sep 17 00:00:00 2001 From: japabu Date: Mon, 28 Sep 2026 16:43:10 +0200 Subject: [PATCH 3/6] Make the T14 LLVM bar stage 3 and count only samples at the recipe's envelope The bar timed stage 2, whose building compiler is a clang+lld that gcc built. A ToyOS clang+lld is clang-built, since gcc has no ToyOS target, so stage 2 compared unlike compilers. And the bar took its fastest sample, 44:01.03, from a run whose MMIO PL1 read 64 W for its first four samples and 20 W after, and it counted the original 45:08.77 run, whose envelope was never read back. - The bar is stage 3: the stage-2 clang+lld (clang-built, same configuration) builds the same source again, and that ninja is the timed span. The recipe's building compiler is clang+lld at 52ed14fc, Release -O3 -DNDEBUG, no LTO, no PGO, built by clang+lld at 52ed14fc with the same configuration. Stage 1 (gcc) stays only as how the first clang exists. The block gains the two s3 lines. - Validity rule: a sample counts only if every read-back during its span matches the recipe's power envelope. The r2 cold-start run and the original run are no longer samples. - On the T14, a driver eval'd the block's own W, CONF, s2 and s3 lines from the committed text: an untimed-as-a-sample stage-2 rebuild (44:53.95, started at 46 C after 7 min idle), then three stage-3 runs and two stage-2 runs back to back, each 8 s after the previous build. The root sampler of r2 read the envelope every 60 s and at each span's start and end. Every sample of every run matched the envelope. - Stage 3: 41:49.40, 41:48.02, 41:47.28. Stage 2, with r2's two valid runs: 45:08.76, 45:08.78, 45:12.48, 45:14.15. - The exit: a ToyOS build by a clang+lld built by the recipe, warm, at the read-back envelope for its whole span, at or under 41:47.28. - The recipe now names the peak limits (121 W) and a warm element: each span begins within 8 s of another build, the package at 68 to 75 C. Removed: mean package power and the cold-run description. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01W6rME2DoqwjcYFStYHHY4j --- issues/build/toyos-builds-itself.md | 57 ++++++++++++++++------------- 1 file changed, 31 insertions(+), 26 deletions(-) diff --git a/issues/build/toyos-builds-itself.md b/issues/build/toyos-builds-itself.md index 26b171b551d..b9c04b3d1c9 100644 --- a/issues/build/toyos-builds-itself.md +++ b/issues/build/toyos-builds-itself.md @@ -53,28 +53,30 @@ enough for an LLVM build tree memory beyond what 2 MiB process pages allow (`issues/kernel/process-memory-is-2-mib-pages-and-that-caps-the-process-count.md`). -**The bar: Linux on this T14 building the same LLVM.** It is a recipe, and a -ToyOS run matches every element of it: +**The bar: Linux on this T14 building the same LLVM with a clang-built +clang+lld.** It is a recipe, and a ToyOS run matches every element of it: - *Source*: `rust-lang/llvm-project` at `52ed14fcd56afc30f9cccd8ca8ce237c2eef7e04`. -- *Stage 1*: `gcc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0`, cmake 3.28.3 and - ninja 1.11.1 build clang+lld with the `s1` lines below: Release - (`-O3 -DNDEBUG`), `LLVM_ENABLE_LTO=OFF`, `LLVM_BUILD_INSTRUMENTED=OFF`, no - profile data. -- *Stage 2*: that clang+lld builds clang+lld from the same source with the - `s2` lines below, configured afresh. +- *Building compiler*: clang+lld at `52ed14fc`, Release (`-O3 -DNDEBUG`), + `LLVM_ENABLE_LTO=OFF`, `LLVM_BUILD_INSTRUMENTED=OFF`, no profile data, + built by clang+lld at `52ed14fc` with the same configuration: the `s2` + lines below. Stage 1 is only how the first clang exists: gcc, which has + no ToyOS target, builds it with the `s1` lines + (`gcc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0`). +- *Build*: that clang+lld builds clang+lld from the same source with the + `s3` lines below, configured afresh; cmake 3.28.3, ninja 1.11.1. - *Timed span*: the last line's `ninja` alone. -- *Cache*: warm. The source and stage 1 were read or written just before the - span, and `File system inputs` was 16 in the 45:08.77 run and 0 in the - others. -- *Power envelope*, read back every 60 s and at the start and end of the - three runs that sampled it: on AC; `platform_profile` `performance`; - intel_pstate active, governor `powersave`, EPP `balance_performance`. RAPL - through the MSR: PL1 64 W with a 27983872 µs window, PL2 64 W. RAPL through - MMIO: PL2 64 W, and PL1 20 W at every sample but the first four of the one - run that started cold, which read 64 W up to 3:01 into it at up to 100 °C. - Mean package power over a run was 19.96 W to 21.22 W. +- *Warm*: each span begins within 8 s of the end of another build of this + source, the package at 68 to 75 °C and `File system inputs` 0. +- *Power envelope*: on AC; `platform_profile` `performance`; intel_pstate + active, governor `powersave`, EPP `balance_performance`. RAPL through the + MSR: PL1 64 W, PL2 64 W, peak 121 W. RAPL through MMIO: PL1 20 W, PL2 + 64 W, peak 121 W. Both PL1 windows are 27983872 µs. All but the + intel_pstate mode and the windows are read back every 60 s and at the + span's start and end. +- *Validity*: a sample counts only if every read-back during its span + matches the power envelope. - *Machine*: i5-1135G7, 8 threads, 16476082176 B RAM; Ubuntu 24.04.4, kernel 6.8.0-142-generic, ext4 on LVM. @@ -90,17 +92,20 @@ rm -rf s1; cmake -S src/llvm -B s1 $CONF -DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMP /usr/bin/time -v -o s1-time.txt ninja -C s1 -j8 clang lld > s1-build.log rm -rf s2; PATH=$W/s1/bin:$PATH cmake -S src/llvm -B s2 $CONF -DCMAKE_C_COMPILER=$W/s1/bin/clang -DCMAKE_CXX_COMPILER=$W/s1/bin/clang++ -DLLVM_ENABLE_LLD=ON > s2-config.log PATH=$W/s1/bin:$PATH /usr/bin/time -v -o s2-time.txt ninja -C s2 -j8 clang lld > s2-build.log +rm -rf s3; PATH=$W/s2/bin:$PATH cmake -S src/llvm -B s3 $CONF -DCMAKE_C_COMPILER=$W/s2/bin/clang -DCMAKE_CXX_COMPILER=$W/s2/bin/clang++ -DLLVM_ENABLE_LLD=ON > s3-config.log +PATH=$W/s2/bin:$PATH /usr/bin/time -v -o s3-time.txt ninja -C s3 -j8 clang lld > s3-build.log ``` -The timed span's wall over four runs: 44:01.03, 45:08.76, 45:08.77 and -45:08.78; min 44:01.03, median 45:08.765, max 45:08.78. The 44:01.03 run is -the one that started cold. The 45:08.77 run's envelope was not read back. +The valid stage-3 samples: 41:47.28, 41:48.02 and 41:49.40; min 41:47.28, +median 41:48.02, max 41:49.40. For context only, the valid stage-2 samples, +the gcc-built clang+lld building the same source with the `s2` lines under +the same rule: 45:08.76, 45:08.78, 45:12.48 and 45:14.15; min 45:08.76, +median 45:10.63, max 45:14.15. -*Exit*: a ToyOS run on this T14 that matches every element of the recipe -finishes at or under the fastest Linux sample, 44:01.03. Its clang+lld is -`52ed14fc` built by the stage-1 recipe, the span is the stage-2 `ninja` -alone after a fresh configure, the cache is warm, and it runs on AC with the -MSR and MMIO PL1 and PL2 above, read back during the run. +*Exit*: a ToyOS build of the same source with the `s3` configuration, by a +clang+lld built by the recipe above, on this T14, warm, and at the power +envelope above as read back for its whole span, finishes at or under +41:47.28, the fastest valid stage-3 sample. **What M2 must do to delete toyos-ld.** It is frozen and links nothing the host builds; what keeps it is that it is the one linker a ToyOS process can run, From 868378b87dea19be3979653756fca77b3210522c Mon Sep 17 00:00:00 2001 From: japabu Date: Mon, 28 Sep 2026 17:17:08 +0200 Subject: [PATCH 4/6] Pin the T14 LLVM bar's flags and HWP registers, and require a primed start The recipe fixed the building compiler by CMAKE_*_FLAGS_RELEASE and three LLVM keys, so an exported CXXFLAGS would have met it with a different compiler. Its HWP half named Linux policy strings, which a ToyOS kernel that writes IA32_HWP_REQUEST itself has no value to compare against. And "within 8 s of another build" admitted a short build that heats the die on a cold heatsink. - Flags, read off the T14's s2 and s3 caches: CMAKE_C_FLAGS and CMAKE_CXX_FLAGS empty, their _RELEASE forms -O3 -DNDEBUG, every EXE/SHARED/MODULE/STATIC linker flag and its _RELEASE form empty. The driver sets no CFLAGS, CXXFLAGS or LDFLAGS, and neither the ssh nor the login environment carries them. The recipe records s3's compile line for clang/tools/driver/driver.cpp from ninja -t commands clang, which names no -march, -flto or -fprofile in any of its 2670 compiles; the s2 and s3 command lists differ only in the build directory. - HWP, read as root through /dev/cpu/N/msr in the same boot as the spans, at the recipe's governor, EPP and platform_profile: IA32_PM_ENABLE 1, IA32_HWP_REQUEST 0x80002a04 on all eight CPUs (min 4, max 42, desired 0, EPP 128), IA32_HWP_REQUEST_PKG 0x8000ff01, IA32_ENERGY_PERF_BIAS 6, IA32_MISC_ENABLE bit 38 clear. The exit requires a ToyOS run to read these back for its whole span with PL1 and PL2. - Warm: a span starts within 8 s of the end of a complete s2 or s3 ninja line, and a sample is valid only if its first 60 s draw at most 21.2 W from the package energy counter. The three stage-3 samples drew 21.11, 21.14 and 21.06 W; the 46 C primed start drew 24.83 W. - The exit requires ninja to exit 0, names the fork's commits over 52ed14fc as the only allowed source difference and x86_64-unknown-toyos as the cross-built compiler's host triple, says the PL1 windows and the HWP registers are read once rather than per span, and states the no-block-read condition instead of a Linux rusage counter. - Removed: the stage-2 samples and the restated min, median and max. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01W6rME2DoqwjcYFStYHHY4j --- issues/build/toyos-builds-itself.md | 71 +++++++++++++++++++---------- 1 file changed, 47 insertions(+), 24 deletions(-) diff --git a/issues/build/toyos-builds-itself.md b/issues/build/toyos-builds-itself.md index b9c04b3d1c9..7138a7b037c 100644 --- a/issues/build/toyos-builds-itself.md +++ b/issues/build/toyos-builds-itself.md @@ -57,26 +57,51 @@ memory beyond what 2 MiB process pages allow clang+lld.** It is a recipe, and a ToyOS run matches every element of it: - *Source*: `rust-lang/llvm-project` at - `52ed14fcd56afc30f9cccd8ca8ce237c2eef7e04`. -- *Building compiler*: clang+lld at `52ed14fc`, Release (`-O3 -DNDEBUG`), + `52ed14fcd56afc30f9cccd8ca8ce237c2eef7e04`. A ToyOS run's source, for + the building compiler and the build alike, differs from it by nothing + but the fork's commits over that base (`forks.toml`, `[llvm-project]`). +- *Building compiler*: clang+lld at `52ed14fc`, Release, `LLVM_ENABLE_LTO=OFF`, `LLVM_BUILD_INSTRUMENTED=OFF`, no profile data, - built by clang+lld at `52ed14fc` with the same configuration: the `s2` - lines below. Stage 1 is only how the first clang exists: gcc, which has - no ToyOS target, builds it with the `s1` lines - (`gcc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0`). + built by clang+lld at `52ed14fc` with the same configuration and flags: + the `s2` lines below. For ToyOS it is cross-built with those lines for + the host triple `x86_64-unknown-toyos`. Stage 1 is only how the first + clang exists: gcc, which has no ToyOS target, builds it with the `s1` + lines (`gcc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0`). +- *Flags*, of `s2` and `s3` alike: `CMAKE_C_FLAGS` and `CMAKE_CXX_FLAGS` + empty, their `_RELEASE` forms `-O3 -DNDEBUG`, and every + `CMAKE_{EXE,SHARED,MODULE,STATIC}_LINKER_FLAGS`, `_RELEASE` included, + empty; `CFLAGS`, `CXXFLAGS` and `LDFLAGS` unset. The two builds' + `ninja -t commands clang` differ only in the build directory. The + compile line `s3` records for `clang/tools/driver/driver.cpp`, with `$W` + for the work directory: + + ``` + $W/s2/bin/clang++ -DLLVM_BUILD_STATIC -D_GLIBCXX_USE_CXX11_ABI=1 -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -I$W/s3/tools/clang/tools/driver -I$W/src/clang/tools/driver -I$W/src/clang/include -I$W/s3/tools/clang/include -I$W/s3/include -I$W/src/llvm/include -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wc++98-compat-extra-semi -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wsuggest-override -Wstring-conversion -Wno-pass-failed -Wmisleading-indentation -Wctad-maybe-unsupported -fdiagnostics-color -ffunction-sections -fdata-sections -fno-common -Woverloaded-virtual -Wno-nested-anon-types -O3 -DNDEBUG -std=c++17 -fno-exceptions -funwind-tables -fno-rtti -MD -MT tools/clang/tools/driver/CMakeFiles/clang.dir/driver.cpp.o -MF tools/clang/tools/driver/CMakeFiles/clang.dir/driver.cpp.o.d -o tools/clang/tools/driver/CMakeFiles/clang.dir/driver.cpp.o -c $W/src/clang/tools/driver/driver.cpp + ``` - *Build*: that clang+lld builds clang+lld from the same source with the `s3` lines below, configured afresh; cmake 3.28.3, ninja 1.11.1. - *Timed span*: the last line's `ninja` alone. -- *Warm*: each span begins within 8 s of the end of another build of this - source, the package at 68 to 75 °C and `File system inputs` 0. -- *Power envelope*: on AC; `platform_profile` `performance`; intel_pstate - active, governor `powersave`, EPP `balance_performance`. RAPL through the - MSR: PL1 64 W, PL2 64 W, peak 121 W. RAPL through MMIO: PL1 20 W, PL2 - 64 W, peak 121 W. Both PL1 windows are 27983872 µs. All but the - intel_pstate mode and the windows are read back every 60 s and at the - span's start and end. +- *Warm*: each span starts within 8 s of the end of a complete `s2` or + `s3` `ninja` line, the package at 68 to 75 °C, and the build reads no + block from a block device during the span. +- *Power envelope*: on AC; `platform_profile` `performance`. RAPL through + the MSR: PL1 64 W, PL2 64 W, peak 121 W. RAPL through MMIO: PL1 20 W, + PL2 64 W, peak 121 W. Both PL1 windows 27983872 µs. On every CPU: + `IA32_PM_ENABLE` (0x770) 1; `IA32_HWP_REQUEST` (0x774) `0x80002a04`, + that is minimum 4, maximum 42, desired 0, EPP 128, activity window 0, + package control off; `IA32_ENERGY_PERF_BIAS` (0x1B0) 6; + `IA32_MISC_ENABLE` (0x1A0) bit 38 clear, turbo enabled. + `IA32_HWP_REQUEST_PKG` (0x772) `0x8000ff01`. Linux reaches the HWP + values through intel_pstate active, governor `powersave`, EPP + `balance_performance`. AC, `platform_profile`, the RAPL limits, the + governor and EPP are read back every 60 s and at the span's start and + end. The PL1 windows and the MSRs of HWP, EPB and turbo are read once, + not per span: those after the spans, in the same boot, with governor, + EPP and `platform_profile` unchanged in every read-back of every span. - *Validity*: a sample counts only if every read-back during its span - matches the power envelope. + matches the power envelope, and the package power over the span's first + 60 s, from the energy counter (`MSR_PKG_ENERGY_STATUS`, 0x611), is at + most 21.2 W. - *Machine*: i5-1135G7, 8 threads, 16476082176 B RAM; Ubuntu 24.04.4, kernel 6.8.0-142-generic, ext4 on LVM. @@ -96,16 +121,14 @@ rm -rf s3; PATH=$W/s2/bin:$PATH cmake -S src/llvm -B s3 $CONF -DCMAKE_C_COMPILER PATH=$W/s2/bin:$PATH /usr/bin/time -v -o s3-time.txt ninja -C s3 -j8 clang lld > s3-build.log ``` -The valid stage-3 samples: 41:47.28, 41:48.02 and 41:49.40; min 41:47.28, -median 41:48.02, max 41:49.40. For context only, the valid stage-2 samples, -the gcc-built clang+lld building the same source with the `s2` lines under -the same rule: 45:08.76, 45:08.78, 45:12.48 and 45:14.15; min 45:08.76, -median 45:10.63, max 45:14.15. +The valid stage-3 samples: 41:47.28, 41:48.02 and 41:49.40. -*Exit*: a ToyOS build of the same source with the `s3` configuration, by a -clang+lld built by the recipe above, on this T14, warm, and at the power -envelope above as read back for its whole span, finishes at or under -41:47.28, the fastest valid stage-3 sample. +*Exit*: a ToyOS build of the source above with the `s3` configuration and +flags, by a clang+lld built by the recipe above, on this T14, warm and +valid, reading back for its whole span PL1 and PL2, every CPU's HWP +request, the package request, EPB and the turbo bit at the envelope's +values, whose `ninja` exits 0 at or under 41:47.28, the fastest valid +stage-3 sample. **What M2 must do to delete toyos-ld.** It is frozen and links nothing the host builds; what keeps it is that it is the one linker a ToyOS process can run, From 59c096b9f8c9b43d55ebdd146b2422f4bcb8a53b Mon Sep 17 00:00:00 2001 From: japabu Date: Mon, 28 Sep 2026 22:49:51 +0200 Subject: [PATCH 5/6] Re-measure the T14 LLVM bar against libc++ with full read-back; the bar is owed The recipe now builds libc++ in each stage and links s2 and s3 against it statically (the `$LIBCXX` line; the block is the one that ran, sha256 22f864bf...). The power envelope names every element with its raw register value, the read-back covers all of them plus every CPU's microcode revision at start, end and at most 61 s between read starts, and *Machine* gains BIOS N34ET71W (1.71), microcode 0xbe, the kernel command line and Linux's five mitigated vulnerabilities. Removed: the 68-75 C band (the start temperatures' min and max, an OS property at its upper bound), the 21.2 W first-60-s clause (fitted 0.06 W above the samples and rejecting a warm run by itself), and the old bar of 41:47.28 with its samples 41:47.28, 41:48.02 and 41:49.40, which were measured against libstdc++ and without per-span read-back of the HWP, EPB, turbo and window registers. The libstdc++ driver.cpp compile line goes with it. The warm rule is now "the next configure starts as soon as the previous build exits": the measured gaps from one build's last read to the next span's first read were 8.47 s, 8.70 s and 8.44 s, so "within 8 s" was false. Measured on the T14 this round: s1 46:44.23 and s2 47:34.23, exit 0, with s2's clang-22 and lld naming no C++ shared object and no GLIBCXX symbol; s3-0 42:37.88 (the complete line the first span follows); s3-1 42:36.03, valid by the judge (45 rows, 0 off the envelope, max gap 60.428 s, no sectors read). s3-2 started 19:02:49Z; the T14 then went unreachable at about 19:20Z, so s3-2's outcome is unknown and s3-3 did not run. One sample does not set a bar: the issue owes the best of three valid samples, the s2/s3 `ninja -t commands clang lld` comparison, the libc++ driver.cpp compile line, the end-of-run machine read, and stopping the driver and sampler. issues/hardware/a-t14-measurement-has-no-way-back-but-its-wifi.md files the missing return path. Co-Authored-By: Claude Opus 5.5 --- issues/build/toyos-builds-itself.md | 120 +++++++++++------- ...easurement-has-no-way-back-but-its-wifi.md | 17 +++ 2 files changed, 93 insertions(+), 44 deletions(-) create mode 100644 issues/hardware/a-t14-measurement-has-no-way-back-but-its-wifi.md diff --git a/issues/build/toyos-builds-itself.md b/issues/build/toyos-builds-itself.md index 7138a7b037c..21ad7ba0c49 100644 --- a/issues/build/toyos-builds-itself.md +++ b/issues/build/toyos-builds-itself.md @@ -54,7 +54,8 @@ memory beyond what 2 MiB process pages allow (`issues/kernel/process-memory-is-2-mib-pages-and-that-caps-the-process-count.md`). **The bar: Linux on this T14 building the same LLVM with a clang-built -clang+lld.** It is a recipe, and a ToyOS run matches every element of it: +clang+lld against libc++.** It is a recipe, and a ToyOS run matches every +element of it: - *Source*: `rust-lang/llvm-project` at `52ed14fcd56afc30f9cccd8ca8ce237c2eef7e04`. A ToyOS run's source, for @@ -67,68 +68,99 @@ clang+lld.** It is a recipe, and a ToyOS run matches every element of it: the host triple `x86_64-unknown-toyos`. Stage 1 is only how the first clang exists: gcc, which has no ToyOS target, builds it with the `s1` lines (`gcc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0`). +- *C++ library*: libc++ with libc++abi from the same source, built in each + stage's build by that stage's own clang (`runtimes`). The `s2` tools link + it statically: `s2`'s `clang-22` and `lld` name no C++ shared object and + reference no `GLIBCXX` symbol. `s2`'s libc++ headers are + `_LIBCPP_VERSION` 220108. - *Flags*, of `s2` and `s3` alike: `CMAKE_C_FLAGS` and `CMAKE_CXX_FLAGS` - empty, their `_RELEASE` forms `-O3 -DNDEBUG`, and every + empty in the cache, their `_RELEASE` forms `-O3 -DNDEBUG`, and every `CMAKE_{EXE,SHARED,MODULE,STATIC}_LINKER_FLAGS`, `_RELEASE` included, - empty; `CFLAGS`, `CXXFLAGS` and `LDFLAGS` unset. The two builds' - `ninja -t commands clang` differ only in the build directory. The - compile line `s3` records for `clang/tools/driver/driver.cpp`, with `$W` - for the work directory: - - ``` - $W/s2/bin/clang++ -DLLVM_BUILD_STATIC -D_GLIBCXX_USE_CXX11_ABI=1 -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -I$W/s3/tools/clang/tools/driver -I$W/src/clang/tools/driver -I$W/src/clang/include -I$W/s3/tools/clang/include -I$W/s3/include -I$W/src/llvm/include -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wc++98-compat-extra-semi -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wsuggest-override -Wstring-conversion -Wno-pass-failed -Wmisleading-indentation -Wctad-maybe-unsupported -fdiagnostics-color -ffunction-sections -fdata-sections -fno-common -Woverloaded-virtual -Wno-nested-anon-types -O3 -DNDEBUG -std=c++17 -fno-exceptions -funwind-tables -fno-rtti -MD -MT tools/clang/tools/driver/CMakeFiles/clang.dir/driver.cpp.o -MF tools/clang/tools/driver/CMakeFiles/clang.dir/driver.cpp.o.d -o tools/clang/tools/driver/CMakeFiles/clang.dir/driver.cpp.o -c $W/src/clang/tools/driver/driver.cpp - ``` + empty; `CFLAGS`, `CXXFLAGS` and `LDFLAGS` unset. From `$LIBCXX`, + configure adds `-stdlib=libc++` to the C++ compile and link lines and + `-static-libstdc++` to the link lines + (`llvm/cmake/modules/HandleLLVMStdlib.cmake`). - *Build*: that clang+lld builds clang+lld from the same source with the `s3` lines below, configured afresh; cmake 3.28.3, ninja 1.11.1. - *Timed span*: the last line's `ninja` alone. -- *Warm*: each span starts within 8 s of the end of a complete `s2` or - `s3` `ninja` line, the package at 68 to 75 °C, and the build reads no - block from a block device during the span. -- *Power envelope*: on AC; `platform_profile` `performance`. RAPL through - the MSR: PL1 64 W, PL2 64 W, peak 121 W. RAPL through MMIO: PL1 20 W, - PL2 64 W, peak 121 W. Both PL1 windows 27983872 µs. On every CPU: - `IA32_PM_ENABLE` (0x770) 1; `IA32_HWP_REQUEST` (0x774) `0x80002a04`, - that is minimum 4, maximum 42, desired 0, EPP 128, activity window 0, - package control off; `IA32_ENERGY_PERF_BIAS` (0x1B0) 6; - `IA32_MISC_ENABLE` (0x1A0) bit 38 clear, turbo enabled. - `IA32_HWP_REQUEST_PKG` (0x772) `0x8000ff01`. Linux reaches the HWP - values through intel_pstate active, governor `powersave`, EPP - `balance_performance`. AC, `platform_profile`, the RAPL limits, the - governor and EPP are read back every 60 s and at the span's start and - end. The PL1 windows and the MSRs of HWP, EPB and turbo are read once, - not per span: those after the spans, in the same boot, with governor, - EPP and `platform_profile` unchanged in every read-back of every span. -- *Validity*: a sample counts only if every read-back during its span - matches the power envelope, and the package power over the span's first - 60 s, from the energy counter (`MSR_PKG_ENERGY_STATUS`, 0x611), is at - most 21.2 W. -- *Machine*: i5-1135G7, 8 threads, 16476082176 B RAM; Ubuntu 24.04.4, kernel - 6.8.0-142-generic, ext4 on LVM. +- *Warm*: a span's configure line starts as soon as a complete `s3` + `ninja` line exits, and its `ninja` as soon as that configure exits; the + build reads no block from a block device during the span (`time -v`'s + file system inputs 0). +- *Power envelope*, every element and its value: + - AC online; `platform_profile` `performance`. + - `MSR_PKG_POWER_LIMIT` (0x610) `0x0042820000dd8200`: PL1 64 W over + 28 s, PL2 64 W over 2.44 ms, both enabled, unlocked. + - The package limit through MMIO, MCHBAR + 0x59A0, + `0x0042820000dd80a0`: PL1 20 W over 28 s, PL2 64 W over 2.44 ms. + - `MSR_VR_CURRENT_CONFIG` (0x601) `0x3c8`: the peak limit, 121 W. + - On every CPU: `IA32_PM_ENABLE` (0x770) 1; `IA32_HWP_REQUEST` (0x774) + `0x80002a04`, that is minimum 4, maximum 42, desired 0, EPP 128, + activity window 0, package control off; `IA32_ENERGY_PERF_BIAS` + (0x1B0) 6; `IA32_MISC_ENABLE` (0x1A0) bit 38 clear, turbo enabled. + - `IA32_HWP_REQUEST_PKG` (0x772) `0x8000ff01`. + + Linux reaches these through intel_pstate `active` with `no_turbo` 0 and + `hwp_dynamic_boost` 0 and, on every CPU, governor `powersave`, EPP + `balance_performance`, `scaling_min_freq` 400000 and `scaling_max_freq` + 4200000; its read-backs include these. +- *Read-back*: every element of the power envelope and every CPU's + microcode revision are read at the span's start, at its end, and between + them at most 61 s apart from one read's start to the next, the sampler + waiting 60 s after each read. A sample missing a read, or with any read + off its value, is invalid. +- *Machine*: i5-1135G7, 8 threads, 16476082176 B RAM; BIOS `N34ET71W (1.71 )`; + microcode revision (`IA32_BIOS_SIGN_ID`, 0x8B) `0xbe` on every CPU; + Ubuntu 24.04.4, kernel 6.8.0-142-generic, command line + `BOOT_IMAGE=/vmlinuz-6.8.0-142-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro`, + which leaves every mitigation at its default; ext4 on LVM. +- *Mitigations*: `/sys/devices/system/cpu/vulnerabilities` reads + `gather_data_sampling` "Mitigation: Microcode", + `indirect_target_selection` "Mitigation: Aligned branch/return thunks", + `spec_store_bypass` "Mitigation: Speculative Store Bypass disabled via + prctl", `spectre_v1` "Mitigation: usercopy/swapgs barriers and __user + pointer sanitization" and `spectre_v2` "Mitigation: Enhanced / Automatic + IBRS; IBPB: conditional; PBRSB-eIBRS: SW sequence; BHI: SW loop, KVM: SW + loop", and every other entry "Not affected". -The lines, run in bash, which splits `$CONF`: +The lines, run in bash, which splits `$CONF` and `$LIBCXX`: ``` W=$HOME/llvm-baseline; SHA=52ed14fcd56afc30f9cccd8ca8ce237c2eef7e04 mkdir -p $W; cd $W git init -q src && git -C src remote add origin https://github.com/rust-lang/llvm-project.git git -C src fetch -q --depth 1 origin $SHA && git -C src checkout -q FETCH_HEAD -CONF="-G Ninja -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_PROJECTS=clang;lld -DLLVM_TARGETS_TO_BUILD=X86;AArch64 -DLLVM_ENABLE_ASSERTIONS=OFF -DLLVM_INCLUDE_TESTS=OFF -DLLVM_INCLUDE_BENCHMARKS=OFF -DLLVM_INCLUDE_EXAMPLES=OFF" +CONF="-G Ninja -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_PROJECTS=clang;lld -DLLVM_ENABLE_RUNTIMES=libcxx;libcxxabi;libunwind -DLIBCXX_STATICALLY_LINK_ABI_IN_STATIC_LIBRARY=ON -DLLVM_TARGETS_TO_BUILD=X86;AArch64 -DLLVM_ENABLE_ASSERTIONS=OFF -DLLVM_INCLUDE_TESTS=OFF -DLLVM_INCLUDE_BENCHMARKS=OFF -DLLVM_INCLUDE_EXAMPLES=OFF" +LIBCXX="-DLLVM_ENABLE_LIBCXX=ON -DLLVM_STATIC_LINK_CXX_STDLIB=ON" rm -rf s1; cmake -S src/llvm -B s1 $CONF -DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++ > s1-config.log -/usr/bin/time -v -o s1-time.txt ninja -C s1 -j8 clang lld > s1-build.log -rm -rf s2; PATH=$W/s1/bin:$PATH cmake -S src/llvm -B s2 $CONF -DCMAKE_C_COMPILER=$W/s1/bin/clang -DCMAKE_CXX_COMPILER=$W/s1/bin/clang++ -DLLVM_ENABLE_LLD=ON > s2-config.log -PATH=$W/s1/bin:$PATH /usr/bin/time -v -o s2-time.txt ninja -C s2 -j8 clang lld > s2-build.log -rm -rf s3; PATH=$W/s2/bin:$PATH cmake -S src/llvm -B s3 $CONF -DCMAKE_C_COMPILER=$W/s2/bin/clang -DCMAKE_CXX_COMPILER=$W/s2/bin/clang++ -DLLVM_ENABLE_LLD=ON > s3-config.log +/usr/bin/time -v -o s1-time.txt ninja -C s1 -j8 clang lld runtimes > s1-build.log +rm -rf s2; PATH=$W/s1/bin:$PATH cmake -S src/llvm -B s2 $CONF $LIBCXX -DCMAKE_C_COMPILER=$W/s1/bin/clang -DCMAKE_CXX_COMPILER=$W/s1/bin/clang++ -DLLVM_ENABLE_LLD=ON > s2-config.log +PATH=$W/s1/bin:$PATH /usr/bin/time -v -o s2-time.txt ninja -C s2 -j8 clang lld runtimes > s2-build.log +rm -rf s3; PATH=$W/s2/bin:$PATH cmake -S src/llvm -B s3 $CONF $LIBCXX -DCMAKE_C_COMPILER=$W/s2/bin/clang -DCMAKE_CXX_COMPILER=$W/s2/bin/clang++ -DLLVM_ENABLE_LLD=ON > s3-config.log PATH=$W/s2/bin:$PATH /usr/bin/time -v -o s3-time.txt ninja -C s3 -j8 clang lld > s3-build.log ``` -The valid stage-3 samples: 41:47.28, 41:48.02 and 41:49.40. +The stage-3 spans measured: `s3-0`, the complete line the first span +follows, 42:37.88, not a sample; `s3-1` 42:36.03, valid; `s3-2` started, +its outcome unknown; `s3-3` not run. The bar is the best of three valid +samples, and it is not yet set. + +Owed once the T14 answers again +(`issues/hardware/a-t14-measurement-has-no-way-back-but-its-wifi.md`): + +- the `s3-2` and `s3-3` spans, and one more span for each that is invalid; +- `ninja -t commands clang lld` of `s2` and of `s3`, compared: they may + differ only in the build directory; +- the compile line `s3` records for `clang/tools/driver/driver.cpp` under + libc++, into *Flags*; +- the end-of-run machine read; +- the driver (pid 224932) and the root sampler (pid 224450) stopped, if + either still runs. *Exit*: a ToyOS build of the source above with the `s3` configuration and flags, by a clang+lld built by the recipe above, on this T14, warm and -valid, reading back for its whole span PL1 and PL2, every CPU's HWP -request, the package request, EPB and the turbo bit at the envelope's -values, whose `ninja` exits 0 at or under 41:47.28, the fastest valid -stage-3 sample. +valid, with the *Read-back* at the Linux read-back's cadence, whose `ninja` +exits 0 at or under the bar. **What M2 must do to delete toyos-ld.** It is frozen and links nothing the host builds; what keeps it is that it is the one linker a ToyOS process can run, diff --git a/issues/hardware/a-t14-measurement-has-no-way-back-but-its-wifi.md b/issues/hardware/a-t14-measurement-has-no-way-back-but-its-wifi.md new file mode 100644 index 00000000000..dacb01ce611 --- /dev/null +++ b/issues/hardware/a-t14-measurement-has-no-way-back-but-its-wifi.md @@ -0,0 +1,17 @@ +--- +status: open +kind: tooling +opened: 2026-09-28 +--- + +# A T14 measurement has no way back but its WiFi + +A measurement under the T14's Ubuntu, such as the LLVM bar in +`issues/build/toyos-builds-itself.md`, is driven and read over ssh to +`t14@192.168.1.46`, which rides the T14's WiFi. At about 19:20Z on +2026-09-28, during the bar's `s3-2` span, the T14 stopped answering ping and +ssh; nobody could reach it physically, so the run's state, and whether its +driver and root sampler still run, cannot be read. + +Owner: whoever holds the T14 runbook. Exit: a wired or out-of-band path to +the T14 under Ubuntu that the orchestrator can reach while its WiFi is down. From cd711427dacd7ec0b14e85a5755078dfff5f639f Mon Sep 17 00:00:00 2001 From: japabu Date: Tue, 29 Sep 2026 06:44:29 +0200 Subject: [PATCH 6/6] PR #568 round 6: the LLVM bar's judge is a host-tested crate, and every unread value is owed Answers the round-5 SEND BACK and the orchestrator's rulings. toyos-llvmbar is new, a member of the host workspace. It holds the recipe's lines (t14/block.txt, byte-identical to the block the issue carried, sha256 22f864bf...), the two steps the T14 runs (t14/sampler.sh as root, t14/driver.sh as the build's user), the power envelope as constants, and the judge. The judge reads a run directory and S0's capture, prints every stage-3 span's verdict and the bar, and exits 0 only when the bar is set: 1 when it is not, 2 when it cannot judge. BLOCKER F: the sampler tags each row read=start, read=timed or read=end, and the judge refuses a span whose first row is not the start read or whose last is not the end read. Round 5's judge.sh gave VALID, exit 0, on the real samples-s3-1.log with its first or last row deleted; toyos-llvmbar refuses both ("row 1: read=timed, not read=start", "row 44: read=timed, not read=end"), exit 1. A span cut off mid-build has no end read and no time file, and is refused too. BLOCKER D: the BIOS version, command line and mitigation lines were never copied off the T14, so they leave the issue. The driver reads, at every span's start and end, /proc/version, /proc/cmdline, the vulnerabilities and the microcode with S0's own commands, and the BIOS version and boot_id; the judge refuses a span unless both reads exist, name 6.8.0-142-generic, agree with each other on command line, BIOS and boot, and carry S0's version, vulnerabilities and microcode lines. The T14 is captured once, by S0. The exit requires the BIOS version and, for each entry S0 captures as a mitigation, its mechanism read back at the span's start and end, and says what parity means for spec_store_bypass. The NOTE on Warm: the judge requires exit status 0 and file system inputs 0 from time -v, and a warm predecessor, s3-, that exited 0 and whose end read precedes the span's start read. The NOTE on Flags: moved to owed, with the s3 compile and link lines the driver now writes. The NOTE on the cross build: for ToyOS the building compiler is M2's cross build, whose lines M2 owes. The judge is in-tree Rust with an exit code. The unreachable-T14 issue is renamed: that its address rides the WiFi was never read, and the tree says the laptop is on a cable. Its owner is the orchestrator, and the read that settles the link is named. REMOVEs deleted: the "waiting 60 s" clause, s3-0's wall, the pids, "which leaves every mitigation at its default", "nobody could reach it physically". s3-1's 42:36.03 is gone with them: its wall was in no copied file, and its log carries no start or end tag, so no sample is valid and the bar is not set. Co-Authored-By: Claude Opus 5.5 --- Cargo.lock | 4 + Cargo.toml | 1 + issues/build/toyos-builds-itself.md | 135 ++-- ...easurement-has-no-way-back-but-its-wifi.md | 17 - ...o-way-back-when-the-t14-stops-answering.md | 20 + toyos-llvmbar/Cargo.toml | 13 + toyos-llvmbar/src/lib.rs | 594 ++++++++++++++++++ toyos-llvmbar/src/main.rs | 67 ++ toyos-llvmbar/src/tests.rs | 415 ++++++++++++ toyos-llvmbar/t14/block.txt | 12 + toyos-llvmbar/t14/driver.sh | 56 ++ toyos-llvmbar/t14/sampler.sh | 51 ++ 12 files changed, 1287 insertions(+), 98 deletions(-) delete mode 100644 issues/hardware/a-t14-measurement-has-no-way-back-but-its-wifi.md create mode 100644 issues/hardware/a-t14-measurement-has-no-way-back-when-the-t14-stops-answering.md create mode 100644 toyos-llvmbar/Cargo.toml create mode 100644 toyos-llvmbar/src/lib.rs create mode 100644 toyos-llvmbar/src/main.rs create mode 100644 toyos-llvmbar/src/tests.rs create mode 100644 toyos-llvmbar/t14/block.txt create mode 100644 toyos-llvmbar/t14/driver.sh create mode 100644 toyos-llvmbar/t14/sampler.sh diff --git a/Cargo.lock b/Cargo.lock index 1e18b1d023b..7c192d6e676 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1096,6 +1096,10 @@ dependencies = [ name = "toyos-libc-copies" version = "0.1.0" +[[package]] +name = "toyos-llvmbar" +version = "0.1.0" + [[package]] name = "toyos-logstream" version = "0.1.0" diff --git a/Cargo.toml b/Cargo.toml index 661c2ad735b..7a4dda16e5b 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -36,6 +36,7 @@ members = [ "toyos-inspect", "toyos-keymap", "toyos-ld", + "toyos-llvmbar", "toyos-libc-copies", "toyos-logstream", "toyos-manifest", diff --git a/issues/build/toyos-builds-itself.md b/issues/build/toyos-builds-itself.md index 911d9cb7417..df0f0f2700e 100644 --- a/issues/build/toyos-builds-itself.md +++ b/issues/build/toyos-builds-itself.md @@ -55,7 +55,9 @@ memory beyond what 2 MiB process pages allow **The bar: Linux on this T14 building the same LLVM with a clang-built clang+lld against libc++.** It is a recipe, and a ToyOS run matches every -element of it: +element of it. `toyos-llvmbar` holds its lines (`t14/block.txt`), the steps +the T14 runs (`t14/`), the power envelope and the judge, whose module header +is the rule a sample is valid by. - *Source*: `rust-lang/llvm-project` at `52ed14fcd56afc30f9cccd8ca8ce237c2eef7e04`. A ToyOS run's source, for @@ -64,103 +66,74 @@ element of it: - *Building compiler*: clang+lld at `52ed14fc`, Release, `LLVM_ENABLE_LTO=OFF`, `LLVM_BUILD_INSTRUMENTED=OFF`, no profile data, built by clang+lld at `52ed14fc` with the same configuration and flags: - the `s2` lines below. For ToyOS it is cross-built with those lines for - the host triple `x86_64-unknown-toyos`. Stage 1 is only how the first - clang exists: gcc, which has no ToyOS target, builds it with the `s1` - lines (`gcc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0`). + the `s2` lines. Stage 1 is only how the first clang exists: gcc, which + has no ToyOS target, builds it with the `s1` lines + (`gcc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0`). For ToyOS it is M2's + cross build of the same source and configuration, whose lines M2 owes. - *C++ library*: libc++ with libc++abi from the same source, built in each stage's build by that stage's own clang (`runtimes`). The `s2` tools link it statically: `s2`'s `clang-22` and `lld` name no C++ shared object and reference no `GLIBCXX` symbol. `s2`'s libc++ headers are `_LIBCPP_VERSION` 220108. -- *Flags*, of `s2` and `s3` alike: `CMAKE_C_FLAGS` and `CMAKE_CXX_FLAGS` - empty in the cache, their `_RELEASE` forms `-O3 -DNDEBUG`, and every - `CMAKE_{EXE,SHARED,MODULE,STATIC}_LINKER_FLAGS`, `_RELEASE` included, - empty; `CFLAGS`, `CXXFLAGS` and `LDFLAGS` unset. From `$LIBCXX`, - configure adds `-stdlib=libc++` to the C++ compile and link lines and - `-static-libstdc++` to the link lines - (`llvm/cmake/modules/HandleLLVMStdlib.cmake`). - *Build*: that clang+lld builds clang+lld from the same source with the - `s3` lines below, configured afresh; cmake 3.28.3, ninja 1.11.1. + `s3` lines, configured afresh; cmake 3.28.3, ninja 1.11.1. - *Timed span*: the last line's `ninja` alone. - *Warm*: a span's configure line starts as soon as a complete `s3` `ninja` line exits, and its `ninja` as soon as that configure exits; the build reads no block from a block device during the span (`time -v`'s file system inputs 0). -- *Power envelope*, every element and its value: - - AC online; `platform_profile` `performance`. - - `MSR_PKG_POWER_LIMIT` (0x610) `0x0042820000dd8200`: PL1 64 W over - 28 s, PL2 64 W over 2.44 ms, both enabled, unlocked. - - The package limit through MMIO, MCHBAR + 0x59A0, - `0x0042820000dd80a0`: PL1 20 W over 28 s, PL2 64 W over 2.44 ms. - - `MSR_VR_CURRENT_CONFIG` (0x601) `0x3c8`: the peak limit, 121 W. - - On every CPU: `IA32_PM_ENABLE` (0x770) 1; `IA32_HWP_REQUEST` (0x774) - `0x80002a04`, that is minimum 4, maximum 42, desired 0, EPP 128, - activity window 0, package control off; `IA32_ENERGY_PERF_BIAS` - (0x1B0) 6; `IA32_MISC_ENABLE` (0x1A0) bit 38 clear, turbo enabled. - - `IA32_HWP_REQUEST_PKG` (0x772) `0x8000ff01`. +- *Power envelope*: `toyos-llvmbar`'s `ENVELOPE` and `ENVELOPE_EVERY_CPU`, + every element with its value; `LINUX` and `LINUX_EVERY_CPU` are how Linux + reaches them. +- *Machine*: i5-1135G7, 8 threads, 16476082176 B RAM; microcode revision + `0xbe` on every CPU; Ubuntu 24.04.4 with the kernel + `issues/kernel/the-kernel-mitigates-what-linux-mitigates-on-the-t14.md` + pins, 6.8.0-142-generic; ext4 on LVM. +- *Mitigations*: that track's S0 capture is the one read of them. A sample + reads the kernel, the vulnerabilities and the microcode at its start and + its end with S0's own commands, and the judge refuses it unless they are + S0's. - Linux reaches these through intel_pstate `active` with `no_turbo` 0 and - `hwp_dynamic_boost` 0 and, on every CPU, governor `powersave`, EPP - `balance_performance`, `scaling_min_freq` 400000 and `scaling_max_freq` - 4200000; its read-backs include these. -- *Read-back*: every element of the power envelope and every CPU's - microcode revision are read at the span's start, at its end, and between - them at most 61 s apart from one read's start to the next, the sampler - waiting 60 s after each read. A sample missing a read, or with any read - off its value, is invalid. -- *Machine*: i5-1135G7, 8 threads, 16476082176 B RAM; BIOS `N34ET71W (1.71 )`; - microcode revision (`IA32_BIOS_SIGN_ID`, 0x8B) `0xbe` on every CPU; - Ubuntu 24.04.4, kernel 6.8.0-142-generic, command line - `BOOT_IMAGE=/vmlinuz-6.8.0-142-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro`, - which leaves every mitigation at its default; ext4 on LVM. -- *Mitigations*: `/sys/devices/system/cpu/vulnerabilities` reads - `gather_data_sampling` "Mitigation: Microcode", - `indirect_target_selection` "Mitigation: Aligned branch/return thunks", - `spec_store_bypass` "Mitigation: Speculative Store Bypass disabled via - prctl", `spectre_v1` "Mitigation: usercopy/swapgs barriers and __user - pointer sanitization" and `spectre_v2` "Mitigation: Enhanced / Automatic - IBRS; IBPB: conditional; PBRSB-eIBRS: SW sequence; BHI: SW loop, KVM: SW - loop", and every other entry "Not affected". - -The lines, run in bash, which splits `$CONF` and `$LIBCXX`: - -``` -W=$HOME/llvm-baseline; SHA=52ed14fcd56afc30f9cccd8ca8ce237c2eef7e04 -mkdir -p $W; cd $W -git init -q src && git -C src remote add origin https://github.com/rust-lang/llvm-project.git -git -C src fetch -q --depth 1 origin $SHA && git -C src checkout -q FETCH_HEAD -CONF="-G Ninja -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_PROJECTS=clang;lld -DLLVM_ENABLE_RUNTIMES=libcxx;libcxxabi;libunwind -DLIBCXX_STATICALLY_LINK_ABI_IN_STATIC_LIBRARY=ON -DLLVM_TARGETS_TO_BUILD=X86;AArch64 -DLLVM_ENABLE_ASSERTIONS=OFF -DLLVM_INCLUDE_TESTS=OFF -DLLVM_INCLUDE_BENCHMARKS=OFF -DLLVM_INCLUDE_EXAMPLES=OFF" -LIBCXX="-DLLVM_ENABLE_LIBCXX=ON -DLLVM_STATIC_LINK_CXX_STDLIB=ON" -rm -rf s1; cmake -S src/llvm -B s1 $CONF -DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++ > s1-config.log -/usr/bin/time -v -o s1-time.txt ninja -C s1 -j8 clang lld runtimes > s1-build.log -rm -rf s2; PATH=$W/s1/bin:$PATH cmake -S src/llvm -B s2 $CONF $LIBCXX -DCMAKE_C_COMPILER=$W/s1/bin/clang -DCMAKE_CXX_COMPILER=$W/s1/bin/clang++ -DLLVM_ENABLE_LLD=ON > s2-config.log -PATH=$W/s1/bin:$PATH /usr/bin/time -v -o s2-time.txt ninja -C s2 -j8 clang lld runtimes > s2-build.log -rm -rf s3; PATH=$W/s2/bin:$PATH cmake -S src/llvm -B s3 $CONF $LIBCXX -DCMAKE_C_COMPILER=$W/s2/bin/clang -DCMAKE_CXX_COMPILER=$W/s2/bin/clang++ -DLLVM_ENABLE_LLD=ON > s3-config.log -PATH=$W/s2/bin:$PATH /usr/bin/time -v -o s3-time.txt ninja -C s3 -j8 clang lld > s3-build.log -``` - -The stage-3 spans measured: `s3-0`, the complete line the first span -follows, 42:37.88, not a sample; `s3-1` 42:36.03, valid; `s3-2` started, -its outcome unknown; `s3-3` not run. The bar is the best of three valid -samples, and it is not yet set. +The bar is the best of three valid samples, and it is not set: no sample is +valid by the judge. Owed once the T14 answers again -(`issues/hardware/a-t14-measurement-has-no-way-back-but-its-wifi.md`): +(`issues/hardware/a-t14-measurement-has-no-way-back-when-the-t14-stops-answering.md`), +and Ubuntu is wiped only after S0's fixtures and the bar are both committed: -- the `s3-2` and `s3-3` spans, and one more span for each that is invalid; -- `ninja -t commands clang lld` of `s2` and of `s3`, compared: they may - differ only in the build directory; -- the compile line `s3` records for `clang/tools/driver/driver.cpp` under - libc++, into *Flags*; -- the end-of-run machine read; -- the driver (pid 224932) and the root sampler (pid 224450) stopped, if - either still runs. +- S0's capture, whose text is the judge's second argument; +- `t14/driver.sh` under `t14/sampler.sh`, run until `toyos-llvmbar` exits + 0, and the bar, the BIOS version and the kernel command line it prints + recorded here; +- *Flags*, of `s2` and `s3`: the `CMAKE_*_FLAGS` and linker flags the + run's `*-cache.txt` hold, and the compile line `s3-commands.txt` holds + for `clang/tools/driver/driver.cpp` and the link line for `clang`, which + show whether `$LIBCXX` added `-stdlib=libc++` and `-static-libstdc++`; +- `s2-commands.txt` and `s3-commands.txt` compared: they may differ only in + the build directory. *Exit*: a ToyOS build of the source above with the `s3` configuration and -flags, by a clang+lld built by the recipe above, on this T14, warm and -valid, with the *Read-back* at the Linux read-back's cadence, whose `ninja` -exits 0 at or under the bar. +flags, by a clang+lld built by the recipe above, on this T14, warm, whose +`ninja` exits 0 at or under the bar, read back as the judge reads a Linux +sample: the envelope at its cadence from a start read to an end read, and at +the span's start and end the microcode, the BIOS version (SMBIOS type 0) +equal to the Linux sample's, and for each entry S0 captures as a mitigation +that track's *Exit* line with its mechanism in force on every CPU: + +- `gather_data_sampling`: `IA32_MCU_OPT_CTRL` (0x123) `GDS_MITG_DIS` clear + (S2); +- `spectre_v2`: `IA32_SPEC_CTRL` (0x48) at S0's Linux read with SSBD masked + (S2); +- `indirect_target_selection`: the live thunk bodies equal to the body S1 + selects (S5); +- `spectre_v1`: CR4.SMAP set (S4); +- `spec_store_bypass`: where S0 captures Linux's prctl mode, SSBD is set only + in a process that asked by prctl, so parity is SSBD set in exactly the + build's processes that ask Linux, read on the switch into each (S6); which + of them ask is owed. + +An entry S0 captures as a mitigation that this list does not name holds the +exit until it is named here. **What M2 must do to delete toyos-ld.** It is frozen and links nothing the host builds; what keeps it is that it is the one linker a ToyOS process can run, diff --git a/issues/hardware/a-t14-measurement-has-no-way-back-but-its-wifi.md b/issues/hardware/a-t14-measurement-has-no-way-back-but-its-wifi.md deleted file mode 100644 index dacb01ce611..00000000000 --- a/issues/hardware/a-t14-measurement-has-no-way-back-but-its-wifi.md +++ /dev/null @@ -1,17 +0,0 @@ ---- -status: open -kind: tooling -opened: 2026-09-28 ---- - -# A T14 measurement has no way back but its WiFi - -A measurement under the T14's Ubuntu, such as the LLVM bar in -`issues/build/toyos-builds-itself.md`, is driven and read over ssh to -`t14@192.168.1.46`, which rides the T14's WiFi. At about 19:20Z on -2026-09-28, during the bar's `s3-2` span, the T14 stopped answering ping and -ssh; nobody could reach it physically, so the run's state, and whether its -driver and root sampler still run, cannot be read. - -Owner: whoever holds the T14 runbook. Exit: a wired or out-of-band path to -the T14 under Ubuntu that the orchestrator can reach while its WiFi is down. diff --git a/issues/hardware/a-t14-measurement-has-no-way-back-when-the-t14-stops-answering.md b/issues/hardware/a-t14-measurement-has-no-way-back-when-the-t14-stops-answering.md new file mode 100644 index 00000000000..6db17500788 --- /dev/null +++ b/issues/hardware/a-t14-measurement-has-no-way-back-when-the-t14-stops-answering.md @@ -0,0 +1,20 @@ +--- +status: open +kind: tooling +opened: 2026-09-28 +--- + +# A T14 measurement has no way back when the T14 stops answering + +A measurement under the T14's Ubuntu, such as the LLVM bar in +`issues/build/toyos-builds-itself.md`, is driven and read over ssh to +`t14@192.168.1.46`. At about 19:20Z on 2026-09-28, during the bar's `s3-2` +span, the T14 stopped answering ping and ssh, so the run's state, and whether +its driver and root sampler still run, cannot be read. Which of the T14's +links that address is on is unread; `ip route get` to the development Mac, +run on the T14, reads it. + +Owner: the orchestrator, which runs the T14 +(`issues/hardware/the-t14-answers-only-through-a-usb-stick.md`). Exit: a second +path to the T14 under Ubuntu, independent of the link that address is on, that +the orchestrator reaches when that link is down. diff --git a/toyos-llvmbar/Cargo.toml b/toyos-llvmbar/Cargo.toml new file mode 100644 index 00000000000..48daa3ffef9 --- /dev/null +++ b/toyos-llvmbar/Cargo.toml @@ -0,0 +1,13 @@ +# A member of the host workspace (root `Cargo.toml`): the T14 LLVM bar's steps +# and its judge. + +[package] +name = "toyos-llvmbar" +description = "The T14 LLVM bar: the steps its Linux run takes on the T14, and the judge of the files they write." +version = "0.1.0" +edition = "2021" +license = "MIT OR Apache-2.0" +publish = false + +[lints.rust] +warnings = "deny" diff --git a/toyos-llvmbar/src/lib.rs b/toyos-llvmbar/src/lib.rs new file mode 100644 index 00000000000..07f275f26cb --- /dev/null +++ b/toyos-llvmbar/src/lib.rs @@ -0,0 +1,594 @@ +//! The judge of the T14 LLVM bar (`issues/build/toyos-builds-itself.md`). +//! +//! On the T14, `t14/sampler.sh ` runs as root and `t14/driver.sh ` as +//! the build's user, `` holding `t14/block.txt`. The judge reads what they +//! leave there, and the text S0 of +//! `issues/kernel/the-kernel-mitigates-what-linux-mitigates-on-the-t14.md` +//! captured. +//! +//! A stage-3 span `s3-`, `k >= 1`, is a valid sample when: +//! - `samples-s3-.log` opens with a `read=start` row and closes with a +//! `read=end` row, every row between is `read=timed`, and every row is +//! `phase=s3-`; +//! - every row reads every element of [`ENVELOPE`], [`ENVELOPE_EVERY_CPU`], +//! [`LINUX`] and [`LINUX_EVERY_CPU`] at its value, and [`MICROCODE`] on every +//! CPU; +//! - consecutive rows start at most [`MAX_GAP_MS`] apart; +//! - `s3--time.txt`, GNU `time -v`'s, reads exit status 0 and file system +//! inputs 0; +//! - `s3--machine-start.txt` and `s3--machine-end.txt` both read kernel +//! [`KERNEL`], one command line, one BIOS version and one boot, and S0's +//! `/proc/version`, vulnerabilities and microcode lines; +//! - it is warm: `s3-` exited 0, and its end read precedes this span's +//! start read. +//! +//! The bar is the shortest wall of the valid samples once there are +//! [`SAMPLES`] of them. + +use std::collections::BTreeMap; + +/// The T14's logical CPUs, every one of which a per-CPU element is read on. +pub const CPUS: usize = 8; +/// How many valid samples the bar is the best of. +pub const SAMPLES: usize = 3; +/// The longest a span may go between the starts of two reads. +pub const MAX_GAP_MS: u64 = 61_000; +/// The kernel S0 pins. +pub const KERNEL: &str = "6.8.0-142-generic"; +/// The microcode revision, `IA32_BIOS_SIGN_ID` (0x8B) bits 63:32, on every CPU. +pub const MICROCODE: u32 = 0xbe; + +/// The power envelope's package elements, a ToyOS run's as much as Linux's: +/// the sampler's key and the value it reads. +pub const ENVELOPE: &[(&str, &str)] = &[ + ("ac", "1"), + ("platform_profile", "performance"), + // MSR_PKG_POWER_LIMIT: PL1 64 W over 28 s, PL2 64 W over 2.44 ms, both + // enabled, unlocked. + ("msr_610", "0042820000dd8200"), + // The package limit through MMIO, MCHBAR + 0x59A0: PL1 20 W over 28 s, PL2 + // 64 W over 2.44 ms. + ("mmio_59a0", "0042820000dd80a0"), + // MSR_VR_CURRENT_CONFIG: the peak limit, 121 W. + ("msr_601", "00000000000003c8"), + // IA32_HWP_REQUEST_PKG. + ("msr_772", "000000008000ff01"), +]; + +/// The power envelope's per-CPU elements, each read on every CPU. +pub const ENVELOPE_EVERY_CPU: &[(&str, &str)] = &[ + // IA32_PM_ENABLE: HWP on. + ("msr_770", "0000000000000001"), + // IA32_HWP_REQUEST: minimum 4, maximum 42, desired 0, EPP 128, activity + // window 0, package control off. + ("msr_774", "0000000080002a04"), + // IA32_ENERGY_PERF_BIAS. + ("msr_1b0", "0000000000000006"), + // IA32_MISC_ENABLE bit 38 clear: turbo enabled. + ("msr_1a0_b38", "0"), +]; + +/// How Linux reaches the envelope, read back beside it: powercap's limits in +/// µW (PL1, PL2, peak) and windows in µs (PL1, PL2) for the MSR and the MMIO +/// package limit, and intel_pstate. +pub const LINUX: &[(&str, &str)] = &[ + ("rapl_msr", "64000000,64000000,121000000,27983872,2440"), + ("rapl_mmio", "20000000,64000000,121000000,27983872,2440"), + ("pstate", "active"), + ("no_turbo", "0"), + ("hwp_dynamic_boost", "0"), +]; + +/// Linux's per-CPU cpufreq settings, each read on every CPU. +pub const LINUX_EVERY_CPU: &[(&str, &str)] = &[ + ("governor", "powersave"), + ("epp", "balance_performance"), + ("min_khz", "400000"), + ("max_khz", "4200000"), +]; + +const VULNERABILITY: &str = "/sys/devices/system/cpu/vulnerabilities/"; +const BIOS: &str = "/sys/class/dmi/id/bios_version:"; +const BOOT: &str = "/proc/sys/kernel/random/boot_id:"; + +/// Every key a sampler row must carry and the value it must read. +pub fn wanted() -> Vec<(&'static str, String)> { + let every_cpu = |v: &str| [v; CPUS].join(","); + let mut want: Vec<(&'static str, String)> = ENVELOPE + .iter() + .chain(LINUX) + .map(|&(k, v)| (k, v.to_string())) + .collect(); + want.extend( + ENVELOPE_EVERY_CPU + .iter() + .chain(LINUX_EVERY_CPU) + .map(|&(k, v)| (k, every_cpu(v))), + ); + want.push(("msr_8b", every_cpu(&format!("{MICROCODE:08x}00000000")))); + want +} + +/// What S0 captured that a span's machine reads must repeat. +#[derive(Debug)] +pub struct S0 { + version: String, + vulnerabilities: Vec, + microcode: Vec, +} + +impl S0 { + /// Takes the `/proc/version`, vulnerabilities and microcode lines out of + /// S0's text, whatever else it holds. + pub fn parse(text: &str) -> Result { + let versions: Vec<&str> = text + .lines() + .filter(|l| l.starts_with("Linux version ")) + .collect(); + let [version] = versions[..] else { + return Err(format!( + "S0 holds {} /proc/version lines, not 1", + versions.len() + )); + }; + let mut vulnerabilities: Vec = text + .lines() + .filter(|l| l.starts_with(VULNERABILITY)) + .map(String::from) + .collect(); + vulnerabilities.sort(); + if vulnerabilities.is_empty() { + return Err("S0 holds no vulnerabilities line".into()); + } + let microcode: Vec = text.lines().filter_map(microcode).collect(); + if microcode.len() != CPUS { + return Err(format!( + "S0 holds {} microcode lines, not {CPUS}", + microcode.len() + )); + } + Ok(S0 { + version: version.into(), + vulnerabilities, + microcode, + }) + } +} + +/// One machine read, as `driver.sh`'s `machine` writes it. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct Machine { + pub version: String, + pub cmdline: String, + pub bios: String, + boot: String, + vulnerabilities: Vec, + microcode: Vec, +} + +impl Machine { + pub fn parse(text: &str) -> Result { + let mut lines = text.lines(); + let version = lines + .next() + .filter(|l| l.starts_with("Linux version ")) + .ok_or("its first line is not /proc/version")?; + let cmdline = lines + .next() + .filter(|l| !l.is_empty()) + .ok_or("its second line is not /proc/cmdline")?; + let (mut vulnerabilities, mut microcodes, mut bios, mut boot) = + (Vec::new(), Vec::new(), None, None); + for line in lines { + if line.starts_with(VULNERABILITY) { + vulnerabilities.push(line.to_string()); + } else if let Some(m) = microcode(line) { + microcodes.push(m); + } else if let Some(b) = line.strip_prefix(BIOS) { + once(&mut bios, b, "BIOS version")?; + } else if let Some(b) = line.strip_prefix(BOOT) { + once(&mut boot, b, "boot_id")?; + } else { + return Err(format!("it holds a line nothing reads: {line:?}")); + } + } + vulnerabilities.sort(); + if vulnerabilities.is_empty() { + return Err("it holds no vulnerabilities line".into()); + } + if microcodes.len() != CPUS { + return Err(format!( + "it holds {} microcode lines, not {CPUS}", + microcodes.len() + )); + } + Ok(Machine { + version: version.into(), + cmdline: cmdline.into(), + bios: bios.ok_or("it holds no BIOS version")?, + boot: boot.ok_or("it holds no boot_id")?, + vulnerabilities, + microcode: microcodes, + }) + } +} + +fn once(slot: &mut Option, value: &str, what: &str) -> Result<(), String> { + match slot.replace(value.to_string()) { + None if !value.is_empty() => Ok(()), + None => Err(format!("its {what} is empty")), + Some(_) => Err(format!("it holds two {what} lines")), + } +} + +/// The value of a `/proc/cpuinfo` `microcode` line. +fn microcode(line: &str) -> Option { + let value = line + .strip_prefix("microcode")? + .trim_start() + .strip_prefix(':')?; + Some(value.trim().to_string()) +} + +/// Milliseconds since the epoch of a `date -u +%FT%T.%3NZ` stamp. +pub fn utc_ms(stamp: &str) -> Result { + let bad = || format!("{stamp:?} is not a UTC stamp"); + let b = stamp.as_bytes(); + let separators = [ + (4, b'-'), + (7, b'-'), + (10, b'T'), + (13, b':'), + (16, b':'), + (19, b'.'), + (23, b'Z'), + ]; + if b.len() != 24 || separators.iter().any(|&(i, c)| b[i] != c) { + return Err(bad()); + } + let n = |from: usize, to: usize| -> Result { + let digits = &stamp[from..to]; + if digits.bytes().all(|c| c.is_ascii_digit()) { + digits.parse().map_err(|_| bad()) + } else { + Err(bad()) + } + }; + let (y, m, d) = (n(0, 4)?, n(5, 7)?, n(8, 10)?); + let (hh, mm, ss, ms) = (n(11, 13)?, n(14, 16)?, n(17, 19)?, n(20, 23)?); + if !(1..=12).contains(&m) || !(1..=31).contains(&d) || hh > 23 || mm > 59 || ss > 60 { + return Err(bad()); + } + // Days from 1970-01-01 in the proleptic Gregorian calendar, the year + // taken to start in March so the leap day ends it. + let y = if m <= 2 { y - 1 } else { y }; + let era = y.div_euclid(400); + let yoe = y - era * 400; + let doy = (153 * ((m + 9) % 12) + 2) / 5 + d - 1; + let days = era * 146_097 + yoe * 365 + yoe / 4 - yoe / 100 + doy - 719_468; + u64::try_from(((days * 24 + hh) * 60 + mm) * 60 + ss) + .map(|s| s * 1000 + ms as u64) + .map_err(|_| bad()) +} + +/// Centiseconds of GNU time's `Elapsed (wall clock)` value, `m:ss.cc` or +/// `h:mm:ss.cc`. +pub fn wall_cs(value: &str) -> Result { + let bad = || format!("{value:?} is not a GNU time wall clock"); + let (whole, cc) = value.split_once('.').ok_or_else(bad)?; + let numbers: Vec = whole + .split(':') + .chain([cc]) + .map(|p| { + if !p.is_empty() && p.bytes().all(|c| c.is_ascii_digit()) { + p.parse().ok() + } else { + None + } + }) + .collect::>() + .ok_or_else(bad)?; + let (h, m, s, c) = match numbers[..] { + [m, s, c] => (0, m, s, c), + [h, m, s, c] => (h, m, s, c), + _ => return Err(bad()), + }; + if cc.len() != 2 || s > 59 || (h > 0 && m > 59) { + return Err(bad()); + } + Ok(((h * 60 + m) * 60 + s) * 100 + c) +} + +/// `wall_cs`'s inverse, in GNU time's form. +pub fn show_wall(cs: u64) -> String { + let (s, c) = (cs / 100, cs % 100); + match s / 3600 { + 0 => format!("{}:{:02}.{c:02}", s / 60, s % 60), + h => format!("{h}:{:02}:{:02}.{c:02}", s / 60 % 60, s % 60), + } +} + +/// A row's `key=value` fields. +fn fields(row: &str) -> Result, String> { + let mut map = BTreeMap::new(); + for token in row.split_whitespace() { + let (k, v) = token + .split_once('=') + .ok_or_else(|| format!("{token:?} is not key=value"))?; + if map.insert(k, v).is_some() { + return Err(format!("{k} twice")); + } + } + Ok(map) +} + +/// The rows of one span's sampler log against the envelope, its tags and its +/// cadence; the start read's time when the start and end reads are both there. +fn check_log(text: &str, span: &str, refusals: &mut Vec) -> Option { + let want = wanted(); + let rows: Vec<&str> = text.lines().collect(); + if rows.len() < 2 { + refusals.push(format!( + "its sampler log holds {} rows, and a span is read at its start and its end", + rows.len() + )); + } + let (mut first, mut prev, mut ends): (Option, Option, _) = + (None, None, (false, false)); + for (i, row) in rows.iter().enumerate() { + let n = i + 1; + let f = match fields(row) { + Ok(f) => f, + Err(e) => { + refusals.push(format!("row {n}: {e}")); + continue; + } + }; + let tag = match i { + 0 => "start", + _ if n == rows.len() => "end", + _ => "timed", + }; + match f.get("read") { + Some(&t) if t == tag => { + ends.0 |= tag == "start"; + ends.1 |= tag == "end"; + } + got => refusals.push(format!( + "row {n}: read={}, not read={tag}", + got.unwrap_or(&"") + )), + } + if f.get("phase") != Some(&span) { + refusals.push(format!( + "row {n}: phase={}, not phase={span}", + f.get("phase").unwrap_or(&"") + )); + } + for (k, v) in &want { + if f.get(k) != Some(&v.as_str()) { + refusals.push(format!( + "row {n}: {k}={}, not {v}", + f.get(k).unwrap_or(&"") + )); + } + } + match f.get("utc").map(|u| utc_ms(u)) { + Some(Ok(t)) => { + if let Some(p) = prev { + match t.checked_sub(p) { + Some(gap) if gap <= MAX_GAP_MS => {} + Some(gap) => { + refusals.push(format!("row {n}: {gap} ms after the read before it")) + } + None => refusals.push(format!("row {n}: earlier than the read before it")), + } + } + first.get_or_insert(t); + prev = Some(t); + } + Some(Err(e)) => refusals.push(format!("row {n}: {e}")), + None => refusals.push(format!("row {n}: no utc")), + } + } + first.filter(|_| ends == (true, true)) +} + +/// GNU `time -v`'s `key: value` lines. +fn gnu_time(text: &str) -> BTreeMap<&str, &str> { + text.lines() + .filter_map(|l| l.rsplit_once(": ")) + .map(|(k, v)| (k.trim(), v.trim())) + .collect() +} + +/// Exit status 0 and file system inputs 0, or why not; the wall if both hold. +fn check_time(text: Option<&str>, name: &str, refusals: &mut Vec) -> Option { + let Some(text) = text else { + refusals.push(format!("{name} has no time file: its ninja did not finish")); + return None; + }; + let time = gnu_time(text); + let mut ok = true; + for (key, want) in [("Exit status", "0"), ("File system inputs", "0")] { + if time.get(key) != Some(&want) { + refusals.push(format!( + "{name}: {key} {}, not {want}", + time.get(key).unwrap_or(&"") + )); + ok = false; + } + } + match time + .get("Elapsed (wall clock) time (h:mm:ss or m:ss)") + .map(|w| wall_cs(w)) + { + Some(Ok(wall)) if ok => Some(wall), + Some(Ok(_)) => None, + Some(Err(e)) => { + refusals.push(format!("{name}: {e}")); + None + } + None => { + refusals.push(format!("{name}: no wall clock")); + None + } + } +} + +/// One stage-3 span's judgement. +#[derive(Debug)] +pub struct Verdict { + pub span: String, + pub wall_cs: Option, + pub machine: Option, + pub refusals: Vec, +} + +impl Verdict { + pub fn valid(&self) -> bool { + self.refusals.is_empty() && self.wall_cs.is_some() + } +} + +/// Judges `s3-`, `k >= 1`, reading the run directory through `read`, which +/// answers `None` for a file that is not there. +pub fn judge_span(read: &dyn Fn(&str) -> Option, s0: &S0, k: u32) -> Verdict { + assert!( + k >= 1, + "s3-0 is the line the first span follows, never a sample" + ); + let span = format!("s3-{k}"); + let prev = format!("s3-{}", k - 1); + let mut refusals = Vec::new(); + + let start = match read(&format!("samples-{span}.log")) { + Some(log) => check_log(&log, &span, &mut refusals), + None => { + refusals.push("no sampler log".into()); + None + } + }; + let wall_cs = check_time( + read(&format!("{span}-time.txt")).as_deref(), + &span, + &mut refusals, + ); + + let mut reads = Vec::new(); + for end in ["start", "end"] { + let name = format!("{span}-machine-{end}.txt"); + match read(&name).map(|t| Machine::parse(&t)) { + Some(Ok(m)) => reads.push(m), + Some(Err(e)) => refusals.push(format!("{name}: {e}")), + None => refusals.push(format!("no {name}")), + } + } + for m in &reads { + if m.version.split_whitespace().nth(2) != Some(KERNEL) { + refusals.push(format!("the kernel is not {KERNEL}: {:?}", m.version)); + } + let each = format!("{MICROCODE:#x}"); + let checks = [ + (m.version == s0.version, "its /proc/version is not S0's"), + ( + m.vulnerabilities == s0.vulnerabilities, + "its vulnerabilities lines are not S0's", + ), + ( + m.microcode == s0.microcode, + "its microcode lines are not S0's", + ), + ( + m.microcode.iter().all(|r| *r == each), + "a CPU's microcode is not the recipe's", + ), + ]; + refusals.extend( + checks + .iter() + .filter(|(ok, _)| !ok) + .map(|(_, why)| why.to_string()), + ); + } + if let [first, last] = &reads[..] { + for (same, what) in [ + (first.cmdline == last.cmdline, "command line"), + (first.bios == last.bios, "BIOS version"), + (first.boot == last.boot, "boot"), + ] { + if !same { + refusals.push(format!( + "the {what} changed between the start and end machine reads" + )); + } + } + } + + let prev_exited = check_time( + read(&format!("{prev}-time.txt")).as_deref(), + &prev, + &mut refusals, + ) + .is_some(); + let prev_end = read(&format!("samples-{prev}.log")).and_then(|log| { + let f = fields(log.lines().last()?).ok()?; + if f.get("read") != Some(&"end") { + return None; + } + utc_ms(f.get("utc")?).ok() + }); + let prev_end = prev_end.filter(|_| prev_exited); + match (prev_end, start) { + (Some(p), Some(start)) if p < start => {} + (Some(_), Some(_)) => refusals.push(format!( + "{prev}'s end read does not precede this span's start read" + )), + (None, _) => refusals.push(format!( + "not warm: {prev} has no end read after a build that exited 0" + )), + (Some(_), None) => {} + } + + Verdict { + span, + wall_cs, + machine: reads.into_iter().next(), + refusals, + } +} + +/// Every stage-3 span a run left, judged, and the bar if it is set. +pub struct Run { + pub verdicts: Vec, + pub bar_cs: Option, +} + +/// Judges `s3-1` onwards until a span that left no file at all. +pub fn judge(read: &dyn Fn(&str) -> Option, s0: &S0) -> Run { + let left_any = |k: u32| { + [ + format!("samples-s3-{k}.log"), + format!("s3-{k}-time.txt"), + format!("s3-{k}-machine-start.txt"), + ] + .iter() + .any(|f| read(f).is_some()) + }; + let verdicts: Vec = (1..) + .take_while(|&k| left_any(k)) + .map(|k| judge_span(read, s0, k)) + .collect(); + let walls: Vec = verdicts + .iter() + .filter(|v| v.valid()) + .filter_map(|v| v.wall_cs) + .collect(); + let bar_cs = (walls.len() >= SAMPLES) + .then(|| walls.iter().copied().min()) + .flatten(); + Run { verdicts, bar_cs } +} + +#[cfg(test)] +mod tests; diff --git a/toyos-llvmbar/src/main.rs b/toyos-llvmbar/src/main.rs new file mode 100644 index 00000000000..3a875889ace --- /dev/null +++ b/toyos-llvmbar/src/main.rs @@ -0,0 +1,67 @@ +//! `toyos-llvmbar `: every stage-3 span's verdict +//! and the bar. Exits 0 when the bar is set, 1 when it is not, 2 when it +//! cannot judge. + +use std::path::Path; +use std::process::exit; + +use toyos_llvmbar::{judge, show_wall, S0, SAMPLES}; + +fn refuse(why: String) -> ! { + eprintln!("toyos-llvmbar: {why}\nusage: toyos-llvmbar "); + exit(2) +} + +fn main() { + let args: Vec = std::env::args().skip(1).collect(); + let [dir, s0] = &args[..] else { + refuse(format!("two arguments, not {}", args.len())) + }; + let dir = Path::new(dir); + if !dir.is_dir() { + refuse(format!("{} is not a directory", dir.display())); + } + let s0 = std::fs::read_to_string(s0) + .map_err(|e| format!("read {s0}: {e}")) + .and_then(|t| S0::parse(&t)) + .unwrap_or_else(|why| refuse(why)); + let read = |name: &str| match std::fs::read_to_string(dir.join(name)) { + Ok(text) => Some(text), + Err(e) if e.kind() == std::io::ErrorKind::NotFound => None, + Err(e) => panic!("read {}: {e}", dir.join(name).display()), + }; + let run = judge(&read, &s0); + for v in &run.verdicts { + if v.valid() { + println!( + "{} VALID {}", + v.span, + show_wall(v.wall_cs.expect("a valid span has a wall")) + ); + } else { + println!("{} INVALID", v.span); + for why in &v.refusals { + println!(" {why}"); + } + } + } + let valid = run.verdicts.iter().filter(|v| v.valid()).count(); + if let Some(m) = run + .verdicts + .iter() + .find(|v| v.valid()) + .and_then(|v| v.machine.as_ref()) + { + println!( + "machine: {}\n cmdline {}\n BIOS {}", + m.version, m.cmdline, m.bios + ); + } + match run.bar_cs { + Some(bar) => println!("bar: {}, the best of {valid} valid samples", show_wall(bar)), + None => { + println!("bar: not set, {valid} of {SAMPLES} valid samples"); + exit(1); + } + } +} diff --git a/toyos-llvmbar/src/tests.rs b/toyos-llvmbar/src/tests.rs new file mode 100644 index 00000000000..491c4a4b730 --- /dev/null +++ b/toyos-llvmbar/src/tests.rs @@ -0,0 +1,415 @@ +use std::collections::BTreeMap; + +use super::*; + +/// A row the T14's sampler wrote during a stage-3 build, before rows carried +/// a `read` tag. +const T14_ROW: &str = concat!( + "utc=2026-09-28T18:20:04.802Z ", + "epoch=1790619604 ", + "phase=s3-1 ", + "temp_mc=70000 ", + "disk_rd_sectors=nvme0n1:1918110,sda:78852 ", + "ac=1 ", + "platform_profile=performance ", + "rapl_msr=64000000,64000000,121000000,27983872,2440 ", + "rapl_mmio=20000000,64000000,121000000,27983872,2440 ", + "msr_610=0042820000dd8200 ", + "mmio_59a0=0042820000dd80a0 ", + "msr_601=00000000000003c8 ", + "msr_770=0000000000000001,0000000000000001,0000000000000001,0000000000000001,0000000000000001,0000000000000001,0000000000000001,0000000000000001 ", + "msr_774=0000000080002a04,0000000080002a04,0000000080002a04,0000000080002a04,0000000080002a04,0000000080002a04,0000000080002a04,0000000080002a04 ", + "msr_1b0=0000000000000006,0000000000000006,0000000000000006,0000000000000006,0000000000000006,0000000000000006,0000000000000006,0000000000000006 ", + "msr_1a0_b38=0,0,0,0,0,0,0,0 ", + "msr_772=000000008000ff01 ", + "msr_8b=000000be00000000,000000be00000000,000000be00000000,000000be00000000,000000be00000000,000000be00000000,000000be00000000,000000be00000000 ", + "governor=powersave,powersave,powersave,powersave,powersave,powersave,powersave,powersave ", + "epp=balance_performance,balance_performance,balance_performance,balance_performance,balance_performance,balance_performance,balance_performance,balance_performance ", + "min_khz=400000,400000,400000,400000,400000,400000,400000,400000 ", + "max_khz=4200000,4200000,4200000,4200000,4200000,4200000,4200000,4200000 ", + "pstate=active ", + "no_turbo=0 ", + "hwp_dynamic_boost=0 ", + "energy_uj=107418320833", +); +const T14_UTC: &str = "utc=2026-09-28T18:20:04.802Z"; + +const VERSION: &str = "Linux version 6.8.0-142-generic (fixture) #142-Ubuntu SMP"; + +/// GNU `time -v`'s output for a ninja that ran `wall`. +fn time_v(wall: &str, exit: u32, inputs: u32) -> String { + format!( + "\tCommand being timed: \"ninja -C s3 -j8 clang lld\"\n\ + \tUser time (seconds): 19403.20\n\ + \tPercent of CPU this job got: 795%\n\ + \tElapsed (wall clock) time (h:mm:ss or m:ss): {wall}\n\ + \tMajor (requiring I/O) page faults: 417\n\ + \tFile system inputs: {inputs}\n\ + \tFile system outputs: 4145776\n\ + \tPage size (bytes): 4096\n\ + \tExit status: {exit}\n" + ) +} + +fn vulnerabilities() -> String { + [ + "gather_data_sampling:Mitigation: fixture", + "spectre_v1:Mitigation: fixture", + "meltdown:Not affected", + ] + .iter() + .map(|l| format!("{VULNERABILITY}{l}\n")) + .collect() +} + +fn microcodes() -> String { + format!("microcode\t: {MICROCODE:#x}\n").repeat(CPUS) +} + +fn machine(bios: &str, boot: &str) -> String { + format!( + "{VERSION}\nBOOT_IMAGE=/vmlinuz-{KERNEL} ro\n{}{}{BIOS}{bios}\n{BOOT}{boot}\n", + vulnerabilities(), + microcodes() + ) +} + +/// S0's text: its lines among others it prints. +fn s0_text() -> String { + format!("Reading package lists...\n{VERSION}\nBOOT_IMAGE=/vmlinuz-{KERNEL} ro\n{}{}0x0000000000000000\n", vulnerabilities(), microcodes()) +} + +fn s0() -> S0 { + S0::parse(&s0_text()).expect("the fixture's S0 parses") +} + +/// The T14's row at `t` seconds into its day, for `span`, tagged `read`. +fn row(span: &str, t: u64, read: &str) -> String { + let stamp = format!( + "utc=2026-09-28T{:02}:{:02}:{:02}.500Z", + t / 3600, + t / 60 % 60, + t % 60 + ); + T14_ROW.replacen(T14_UTC, &stamp, 1).replacen( + "phase=s3-1 ", + &format!("phase={span} read={read} "), + 1, + ) +} + +/// A span's sampler log: a start read at `t`, timed reads 60 s apart, and an +/// end read 60 s after the last. +fn log(span: &str, t: u64, timed: u64) -> String { + let mut rows = vec![row(span, t, "start")]; + rows.extend((1..=timed).map(|i| row(span, t + 60 * i, "timed"))); + rows.push(row(span, t + 60 * (timed + 1), "end")); + rows.join("\n") + "\n" +} + +/// A run of s3-0 to s3-3 in which every span is valid. +fn run() -> BTreeMap { + let mut files = BTreeMap::new(); + for (k, wall) in ["42:37.88", "42:40.00", "42:36.03", "42:38.50"] + .iter() + .enumerate() + { + let span = format!("s3-{k}"); + files.insert( + format!("samples-{span}.log"), + log(&span, 3600 + 1000 * k as u64, 4), + ); + files.insert(format!("{span}-time.txt"), time_v(wall, 0, 0)); + files.insert( + format!("{span}-machine-start.txt"), + machine("BIOS-A", "boot-a"), + ); + files.insert( + format!("{span}-machine-end.txt"), + machine("BIOS-A", "boot-a"), + ); + } + files +} + +fn judged(files: &BTreeMap) -> Run { + judge(&|name: &str| files.get(name).cloned(), &s0()) +} + +fn edit(files: &mut BTreeMap, name: &str, f: impl FnOnce(&str) -> String) { + let text = files + .get(name) + .unwrap_or_else(|| panic!("the fixture has no {name}")); + let edited = f(text); + assert_ne!(&edited, text, "the edit to {name} changed nothing"); + files.insert(name.into(), edited); +} + +fn without_line(text: &str, which: fn(usize) -> usize) -> String { + let mut lines: Vec<&str> = text.lines().collect(); + lines.remove(which(lines.len())); + lines.join("\n") + "\n" +} + +/// The one span `edit` turned invalid, and what the judge said. +fn only_refused(files: &BTreeMap) -> (String, Vec) { + let run = judged(files); + let refused: Vec<&Verdict> = run.verdicts.iter().filter(|v| !v.valid()).collect(); + let [v] = refused[..] else { + panic!( + "{} spans refused, not 1: {:#?}", + refused.len(), + run.verdicts + ) + }; + assert_eq!(run.bar_cs, None, "two valid samples set no bar"); + (v.span.clone(), v.refusals.clone()) +} + +fn says(refusals: &[String], words: &str) -> bool { + refusals.iter().any(|r| r.contains(words)) +} + +#[test] +fn a_complete_run_sets_the_bar_to_its_shortest_valid_wall() { + let run = judged(&run()); + let spans: Vec<(&str, bool)> = run + .verdicts + .iter() + .map(|v| (v.span.as_str(), v.valid())) + .collect(); + assert_eq!( + spans, + [("s3-1", true), ("s3-2", true), ("s3-3", true)], + "{:#?}", + run.verdicts + ); + assert_eq!(run.bar_cs.map(show_wall).as_deref(), Some("42:36.03")); +} + +#[test] +fn a_span_without_its_start_read_is_refused() { + let mut files = run(); + edit(&mut files, "samples-s3-2.log", |t| without_line(t, |_| 0)); + let (span, why) = only_refused(&files); + assert_eq!(span, "s3-2"); + assert!(says(&why, "row 1: read=timed, not read=start"), "{why:#?}"); +} + +#[test] +fn a_span_without_its_end_read_is_refused() { + let mut files = run(); + edit(&mut files, "samples-s3-3.log", |t| { + without_line(t, |n| n - 1) + }); + let (span, why) = only_refused(&files); + assert_eq!(span, "s3-3"); + assert!(says(&why, "read=timed, not read=end"), "{why:#?}"); +} + +#[test] +fn a_span_cut_off_mid_build_is_refused() { + let mut files = run(); + edit(&mut files, "samples-s3-3.log", |t| { + without_line(t, |n| n - 1) + }); + files.remove("s3-3-time.txt"); + files.remove("s3-3-machine-end.txt"); + let (span, why) = only_refused(&files); + assert_eq!(span, "s3-3"); + assert!( + says(&why, "did not finish") && says(&why, "no s3-3-machine-end.txt"), + "{why:#?}" + ); +} + +#[test] +fn a_missing_timed_read_is_a_gap() { + let mut files = run(); + edit(&mut files, "samples-s3-1.log", |t| without_line(t, |_| 2)); + let (span, why) = only_refused(&files); + assert_eq!(span, "s3-1"); + assert!(says(&why, "120000 ms after the read before it"), "{why:#?}"); +} + +#[test] +fn one_element_off_on_one_cpu_is_refused() { + let mut files = run(); + edit(&mut files, "samples-s3-1.log", |t| { + t.replacen( + "0000000080002a04,0000000080002a04,0000000080002a04,0000000080002a04", + "0000000080002a04,0000000080002a04,0000000080002a04,0000000080002a05", + 1, + ) + }); + let (_, why) = only_refused(&files); + assert!(says(&why, "row 1: msr_774="), "{why:#?}"); +} + +#[test] +fn another_microcode_in_one_row_is_refused() { + let mut files = run(); + edit(&mut files, "samples-s3-2.log", |t| { + t.replacen("msr_8b=000000be00000000", "msr_8b=000000bc00000000", 1) + }); + let (_, why) = only_refused(&files); + assert!(says(&why, "row 1: msr_8b="), "{why:#?}"); +} + +#[test] +fn a_build_that_failed_or_read_a_block_is_refused() { + for (exit, inputs, words) in [(1, 0, "Exit status 1"), (0, 8, "File system inputs 8")] { + let mut files = run(); + edit(&mut files, "s3-3-time.txt", |_| { + time_v("42:36.03", exit, inputs) + }); + let (span, why) = only_refused(&files); + assert_eq!(span, "s3-3"); + assert!(says(&why, words), "{why:#?}"); + } +} + +#[test] +fn a_missing_machine_read_is_refused() { + for end in ["start", "end"] { + let mut files = run(); + files.remove(&format!("s3-2-machine-{end}.txt")); + let (_, why) = only_refused(&files); + assert!( + says(&why, &format!("no s3-2-machine-{end}.txt")), + "{why:#?}" + ); + } +} + +#[test] +fn the_machine_reads_agree_with_each_other_and_with_s0() { + type Change = fn(&str) -> String; + let cases: [(&str, Change, &str); 5] = [ + ( + "s3-1-machine-end.txt", + |_| machine("BIOS-B", "boot-a"), + "the BIOS version changed", + ), + ( + "s3-1-machine-end.txt", + |_| machine("BIOS-A", "boot-b"), + "the boot changed", + ), + ( + "s3-1-machine-start.txt", + |t| t.replacen("spectre_v1:Mitigation: fixture", "spectre_v1:Vulnerable", 1), + "vulnerabilities lines are not S0's", + ), + ( + "s3-1-machine-start.txt", + |t| { + t.replacen( + "6.8.0-142-generic (fixture)", + "6.8.0-143-generic (fixture)", + 1, + ) + }, + "the kernel is not 6.8.0-142-generic", + ), + ( + "s3-1-machine-end.txt", + |t| t.replacen(": 0xbe", ": 0xbc", 1), + "microcode lines are not S0's", + ), + ]; + for (name, change, words) in cases { + let mut files = run(); + edit(&mut files, name, change); + let (span, why) = only_refused(&files); + assert_eq!(span, "s3-1"); + assert!(says(&why, words), "{name}: {why:#?}"); + } +} + +#[test] +fn a_span_after_a_failed_build_is_not_warm() { + let mut files = run(); + edit(&mut files, "s3-0-time.txt", |_| time_v("42:37.88", 1, 0)); + let (span, why) = only_refused(&files); + assert_eq!(span, "s3-1"); + assert!(says(&why, "not warm: s3-0"), "{why:#?}"); +} + +#[test] +fn a_span_after_a_span_without_its_end_read_is_not_warm() { + let mut files = run(); + edit(&mut files, "samples-s3-1.log", |t| { + without_line(t, |n| n - 1) + }); + let run = judged(&files); + let refused: Vec<&str> = run + .verdicts + .iter() + .filter(|v| !v.valid()) + .map(|v| v.span.as_str()) + .collect(); + assert_eq!(refused, ["s3-1", "s3-2"]); + assert!( + says(&run.verdicts[1].refusals, "not warm: s3-1"), + "{:#?}", + run.verdicts[1] + ); +} + +#[test] +fn the_t14s_own_rows_read_the_envelope() { + let tagged = |t, read| row("s3-1", t, read); + let log = [tagged(3600, "start"), tagged(3660, "end")].join("\n"); + let mut refusals = Vec::new(); + assert!(check_log(&log, "s3-1", &mut refusals).is_some()); + assert_eq!(refusals, Vec::::new()); + + let untagged = check_log(&[T14_ROW, T14_ROW].join("\n"), "s3-1", &mut refusals); + assert!( + untagged.is_none() && says(&refusals, "read=, not read=start"), + "{refusals:#?}" + ); +} + +#[test] +fn a_utc_stamp_reads_as_date_does() { + let f = fields(T14_ROW).expect("the T14's row is key=value"); + let epoch: u64 = f["epoch"].parse().expect("epoch is a number"); + assert_eq!(utc_ms(f["utc"]).map(|ms| ms / 1000), Ok(epoch)); + assert_eq!(utc_ms("1970-01-01T00:00:00.000Z"), Ok(0)); + assert_eq!(utc_ms("2000-03-01T00:00:00.001Z"), Ok(951_868_800_001)); + assert!(utc_ms("2026-09-28 18:20:04.802Z").is_err()); +} + +#[test] +fn a_wall_clock_reads_both_of_gnu_times_forms() { + assert_eq!(wall_cs("42:36.03"), Ok(255_603)); + assert_eq!(wall_cs("1:02:03.45"), Ok(372_345)); + assert_eq!(show_wall(372_345), "1:02:03.45"); + for bad in ["42:36", "42:61.00", "42:36.3", "x:36.03"] { + assert!(wall_cs(bad).is_err(), "{bad}"); + } +} + +#[test] +fn s0_is_refused_without_one_version_or_every_cpus_microcode() { + assert!(S0::parse(&format!("{}{VERSION}\n", s0_text())).is_err()); + assert!(S0::parse(&s0_text().replacen("microcode\t: 0xbe\n", "", 1)).is_err()); + assert!(S0::parse(&s0_text().replace(VULNERABILITY, "/elsewhere/")).is_err()); +} + +#[test] +fn the_sampler_writes_every_key_the_judge_reads() { + let sampler = include_str!("../t14/sampler.sh"); + for key in wanted() + .iter() + .map(|(k, _)| *k) + .chain(["utc", "phase", "read"]) + { + assert!( + sampler.contains(&format!(" {key}=$")) || sampler.contains(&format!("\"{key}=$")), + "sampler.sh writes no {key}" + ); + } +} diff --git a/toyos-llvmbar/t14/block.txt b/toyos-llvmbar/t14/block.txt new file mode 100644 index 00000000000..21bfee412b2 --- /dev/null +++ b/toyos-llvmbar/t14/block.txt @@ -0,0 +1,12 @@ +W=$HOME/llvm-baseline; SHA=52ed14fcd56afc30f9cccd8ca8ce237c2eef7e04 +mkdir -p $W; cd $W +git init -q src && git -C src remote add origin https://github.com/rust-lang/llvm-project.git +git -C src fetch -q --depth 1 origin $SHA && git -C src checkout -q FETCH_HEAD +CONF="-G Ninja -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_PROJECTS=clang;lld -DLLVM_ENABLE_RUNTIMES=libcxx;libcxxabi;libunwind -DLIBCXX_STATICALLY_LINK_ABI_IN_STATIC_LIBRARY=ON -DLLVM_TARGETS_TO_BUILD=X86;AArch64 -DLLVM_ENABLE_ASSERTIONS=OFF -DLLVM_INCLUDE_TESTS=OFF -DLLVM_INCLUDE_BENCHMARKS=OFF -DLLVM_INCLUDE_EXAMPLES=OFF" +LIBCXX="-DLLVM_ENABLE_LIBCXX=ON -DLLVM_STATIC_LINK_CXX_STDLIB=ON" +rm -rf s1; cmake -S src/llvm -B s1 $CONF -DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++ > s1-config.log +/usr/bin/time -v -o s1-time.txt ninja -C s1 -j8 clang lld runtimes > s1-build.log +rm -rf s2; PATH=$W/s1/bin:$PATH cmake -S src/llvm -B s2 $CONF $LIBCXX -DCMAKE_C_COMPILER=$W/s1/bin/clang -DCMAKE_CXX_COMPILER=$W/s1/bin/clang++ -DLLVM_ENABLE_LLD=ON > s2-config.log +PATH=$W/s1/bin:$PATH /usr/bin/time -v -o s2-time.txt ninja -C s2 -j8 clang lld runtimes > s2-build.log +rm -rf s3; PATH=$W/s2/bin:$PATH cmake -S src/llvm -B s3 $CONF $LIBCXX -DCMAKE_C_COMPILER=$W/s2/bin/clang -DCMAKE_CXX_COMPILER=$W/s2/bin/clang++ -DLLVM_ENABLE_LLD=ON > s3-config.log +PATH=$W/s2/bin:$PATH /usr/bin/time -v -o s3-time.txt ninja -C s3 -j8 clang lld > s3-build.log diff --git a/toyos-llvmbar/t14/driver.sh b/toyos-llvmbar/t14/driver.sh new file mode 100644 index 00000000000..024ed777412 --- /dev/null +++ b/toyos-llvmbar/t14/driver.sh @@ -0,0 +1,56 @@ +#!/bin/bash +# As the build's user, with sampler.sh running on the same run directory, which holds +# block.txt: s1, s2, one complete s3 line, then the stage-3 spans s3-1 to s3-3 back to +# back. Every configure and ninja line is block.txt's own line, eval'd. +set -eu +R=$1; B=$R/block.txt +line() { + local l n + l=$(awk -v p="$1" 'index($0, p) == 1' $B); n=$(printf '%s\n' "$l" | grep -c .) + [ "$n" = 1 ] || { echo "block: $n lines start with '$1'" >&2; exit 1; } + printf '%s\n' "$l" +} +L_W=$(line 'W=$HOME/llvm-baseline;'); L_CONF=$(line 'CONF='); L_LIBCXX=$(line 'LIBCXX=') +L_C1=$(line 'rm -rf s1;'); L_B1=$(line '/usr/bin/time') +L_C2=$(line 'rm -rf s2;'); L_B2=$(line 'PATH=$W/s1/bin:$PATH /usr/bin/time') +L_C3=$(line 'rm -rf s3;'); L_B3=$(line 'PATH=$W/s2/bin:$PATH /usr/bin/time') +eval "$L_W"; cd $W; eval "$L_CONF"; eval "$L_LIBCXX" +[ "$(git -C src rev-parse HEAD)" = "$SHA" ] || { echo "src is not at $SHA"; exit 1; } +[ -z "$(git -C src status --porcelain)" ] || { echo "src is not clean"; exit 1; } +SPID=$(cat $R/sampler.pid) +exec 3<>$R/phase.fifo +phase() { + [ -d /proc/$SPID ] || { echo "sampler $SPID gone"; exit 1; } + echo "$1" >&3 + echo "== $1 $(date -u +%FT%T.%3NZ)" +} +trap 'echo stop >&3' EXIT +# The kernel, command line, mitigations and microcode by S0's own commands +# (issues/kernel/the-kernel-mitigates-what-linux-mitigates-on-the-t14.md), then the BIOS and the boot. +machine() { + { cat /proc/version /proc/cmdline; grep . /sys/devices/system/cpu/vulnerabilities/*; grep microcode /proc/cpuinfo + grep -H . /sys/class/dmi/id/bios_version /proc/sys/kernel/random/boot_id; } > $R/$1.txt +} +KEYS='^(CMAKE_BUILD_TYPE|CMAKE_C_COMPILER|CMAKE_CXX_COMPILER|CMAKE_C_FLAGS|CMAKE_CXX_FLAGS|CMAKE_C_FLAGS_RELEASE|CMAKE_CXX_FLAGS_RELEASE|CMAKE_(EXE|SHARED|MODULE|STATIC)_LINKER_FLAGS(_RELEASE)?|LLVM_ENABLE_LTO|LLVM_BUILD_INSTRUMENTED|LLVM_PROFDATA_FILE|LLVM_ENABLE_LLD|LLVM_ENABLE_PROJECTS|LLVM_ENABLE_RUNTIMES|LLVM_ENABLE_LIBCXX|LLVM_STATIC_LINK_CXX_STDLIB|LIBCXX_STATICALLY_LINK_ABI_IN_STATIC_LIBRARY|LLVM_TARGETS_TO_BUILD|LLVM_ENABLE_ASSERTIONS|CXX_(COMPILER|LINKER)_SUPPORTS_(STATIC_)?STDLIB)[:=]' +run() { # name stage configure-line ninja-line + local name=$1 s=$2 + eval "$3" + grep -E "$KEYS" $s/CMakeCache.txt > $R/$name-cache.txt + grep -E "compiler identification|Linker detection|lld project|libc\+\+|WARNING" $s-config.log >> $R/$name-cache.txt || true + mv $s-config.log $R/$name-config.log + sync + machine $name-machine-start + phase $name-start + eval "$4" + phase $name-end + machine $name-machine-end + mv $s-time.txt $R/$name-time.txt; mv $s-build.log $R/$name-build.log + echo "== $name done: $(grep -E 'Elapsed|Exit status' $R/$name-time.txt | tr '\n' ' ')" +} +run s1 s1 "$L_C1" "$L_B1" +run s2 s2 "$L_C2" "$L_B2" +for f in clang clang++ ld.lld; do [ -x s2/bin/$f ] || { echo "s2/bin/$f missing"; exit 1; }; done +for N in 0 1 2 3; do run s3-$N s3 "$L_C3" "$L_B3"; done +ninja -C s2 -t commands clang lld > $R/s2-commands.txt +ninja -C s3 -t commands clang lld > $R/s3-commands.txt +echo "== end $(date -u +%FT%T.%3NZ)" diff --git a/toyos-llvmbar/t14/sampler.sh b/toyos-llvmbar/t14/sampler.sh new file mode 100644 index 00000000000..d2d898f596c --- /dev/null +++ b/toyos-llvmbar/t14/sampler.sh @@ -0,0 +1,51 @@ +#!/bin/bash +# As root, beside driver.sh: reads every envelope element into samples-.log at each +# phase the driver writes to phase.fifo, tagged read=start or read=end, and every 60 s +# between, tagged read=timed. +set -eu +R=$1 +P=/sys/class/powercap +CPUS="0 1 2 3 4 5 6 7" +[ "$(ls -d /dev/cpu/[0-9]* | wc -l)" = 8 ] || { echo "expected 8 CPUs" >&2; exit 1; } +[ -e /dev/cpu/0/msr ] || { echo "no /dev/cpu/0/msr: modprobe msr" >&2; exit 1; } +T= +for h in /sys/class/hwmon/hwmon*; do [ "$(cat $h/name)" = coretemp ] && T=$h; done +[ "$(cat $T/temp1_label)" = "Package id 0" ] || { echo "no package temp" >&2; exit 1; } +mch=$(dd if=/sys/bus/pci/devices/0000:00:00.0/config bs=1 skip=$((0x48)) count=8 status=none | od -An -tx8 | tr -d ' ') +[ $((0x$mch & 1)) = 1 ] || { echo "MCHBAR disabled: $mch" >&2; exit 1; } +MMIO_PL=$(( (0x$mch & ~0x7fff) + 0x59a0 )) +rd() { dd if=/dev/cpu/$1/msr bs=8 count=1 skip=$(($2)) iflag=skip_bytes status=none | od -An -tx8 | tr -d ' '; } +percpu() { local v= c; for c in $CPUS; do v=$v${v:+,}$(rd $c $1); done; echo $v; } +b38() { local v= c x; for c in $CPUS; do x=$(rd $c 0x1a0); v=$v${v:+,}$(( (0x$x >> 38) & 1 )); done; echo $v; } +sysall() { local v= c; for c in $CPUS; do v=$v${v:+,}$(cat /sys/devices/system/cpu/cpu$c/cpufreq/$1); done; echo $v; } +rapl() { echo $(cat $P/$1/constraint_{0,1,2}_power_limit_uw) $(cat $P/$1/constraint_{0,1}_time_window_us) | tr ' ' ,; } +disks() { awk '$3 ~ /^(nvme[0-9]+n[0-9]+|sd[a-z]+)$/ {printf "%s%s:%s", s, $3, $6; s=","}' /proc/diskstats; } +[ -p $R/phase.fifo ] || mkfifo $R/phase.fifo +echo $$ > $R/sampler.pid +exec 3<>$R/phase.fifo +phase=idle; out=$R/samples-idle.log +sample() { + echo "utc=$(date -u +%FT%T.%3NZ) epoch=$(date +%s) phase=$phase read=$1 temp_mc=$(cat $T/temp1_input) disk_rd_sectors=$(disks)" \ + "ac=$(cat /sys/class/power_supply/AC/online) platform_profile=$(cat /sys/firmware/acpi/platform_profile)" \ + "rapl_msr=$(rapl intel-rapl:0) rapl_mmio=$(rapl intel-rapl-mmio:0)" \ + "msr_610=$(rd 0 0x610) mmio_59a0=$(dd if=/dev/mem bs=8 count=1 skip=$MMIO_PL iflag=skip_bytes status=none | od -An -tx8 | tr -d ' ') msr_601=$(rd 0 0x601)" \ + "msr_770=$(percpu 0x770) msr_774=$(percpu 0x774) msr_1b0=$(percpu 0x1b0) msr_1a0_b38=$(b38) msr_772=$(rd 0 0x772) msr_8b=$(percpu 0x8b)" \ + "governor=$(sysall scaling_governor) epp=$(sysall energy_performance_preference) min_khz=$(sysall scaling_min_freq) max_khz=$(sysall scaling_max_freq)" \ + "pstate=$(cat /sys/devices/system/cpu/intel_pstate/status) no_turbo=$(cat /sys/devices/system/cpu/intel_pstate/no_turbo) hwp_dynamic_boost=$(cat /sys/devices/system/cpu/intel_pstate/hwp_dynamic_boost)" \ + "energy_uj=$(cat $P/intel-rapl:0/energy_uj)" >> "$out" +} +sample timed; next=$(($(date +%s) + 60)) +while :; do + wait_s=$((next - $(date +%s))); [ $wait_s -ge 1 ] || wait_s=1 + if read -t $wait_s -u 3 line; then + case $line in + *-start) phase=${line%-start}; out=$R/samples-$phase.log; sample start ;; + *-end) sample end; phase=idle; out=$R/samples-idle.log ;; + stop) sample timed; exit 0 ;; + *) echo "unknown phase: $line" >&2; exit 1 ;; + esac + else + rc=$?; [ $rc -gt 128 ] || { echo "fifo read failed: $rc" >&2; exit 1; } + sample timed; next=$(($(date +%s) + 60)) + fi +done