From 89013de9fd91e51509eda12e25ad89939d11a583 Mon Sep 17 00:00:00 2001 From: japabu Date: Sun, 4 Oct 2026 02:47:13 +0200 Subject: [PATCH 1/2] File the sysroot's debug assertions and the allocators' spinlock; repoint a moved fixture path The sysroot's core, alloc and std inherit rust.debug-assertions from the bootstrap profile "compiler" that src/sysroot.rs's std_config names, so a link-time no-panic proof fails against ToyOS's libraries in every arm and passes against upstream's (a scout's matrix, posted on the pull request). std's and libc's global allocators spin on one AtomicI32 with no futex, so a real-time waiter on a fair holder's CPU spins in its place. #703 moved toyos-mixer under userland/soundserver/mixer; the one issue line naming its fixture by the old path is repointed. Every other hit of the moved crates' names in issues/ is a package name, a branch name or a transcript recorded at a named run, and stays. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01WcU2Dsw6mDYtwYfzVHPzM8 --- ...allocate-under-a-spinlock-with-no-futex.md | 41 +++++++++++++ ...-the-compiler-profiles-debug-assertions.md | 61 +++++++++++++++++++ .../design-debt/what-is-owed-on-file-size.md | 2 +- 3 files changed, 103 insertions(+), 1 deletion(-) create mode 100644 issues/build/std-and-libc-allocate-under-a-spinlock-with-no-futex.md create mode 100644 issues/build/the-sysroots-core-and-alloc-carry-the-compiler-profiles-debug-assertions.md diff --git a/issues/build/std-and-libc-allocate-under-a-spinlock-with-no-futex.md b/issues/build/std-and-libc-allocate-under-a-spinlock-with-no-futex.md new file mode 100644 index 00000000000..84037d5ead3 --- /dev/null +++ b/issues/build/std-and-libc-allocate-under-a-spinlock-with-no-futex.md @@ -0,0 +1,41 @@ +--- +status: open +kind: defect +opened: 2026-10-04 +--- + +# std's and libc's allocators spin on one flag, so a waiter that outranks the holder spins in its place + +Every allocation a ToyOS program makes goes through one `dlmalloc` behind a +process-wide `AtomicI32` that a waiter takes with `swap(1, Acquire)` in a +`spin_loop()` loop, never sleeping on a futex: + +- std's: `library/std/src/sys/alloc/toyos.rs` in the `rust/` fork at + `a0d44493`, lines 57–72 (`LOCKED`, `lock`, `DropLock`), taken by `alloc`, + `alloc_zeroed`, `dealloc` and `realloc`, lines 74–96. +- libc's, in a program linked without std: `userland/libc/src/lib.rs`, lines + 174–188, taken by `LibcAllocator`'s `alloc`, `dealloc` and `realloc`, lines + 192–207. Beside std (`std-runtime`), libc's C allocator calls std's + (`userland/libc/src/memory.rs`). + +A holder that is preempted, or that is inside the `mmap` or `munmap` syscall +`dlmalloc` makes under the lock through its backing allocator, leaves every +other thread of its process that allocates spinning through its slice. A +real-time thread outranks a fair one by right +(`issues/kernel/cpu-time-is-a-band-and-not-a-reservation.md`), so a real-time +waiter on the holder's CPU keeps a fair holder off that CPU while it spins: +priority inversion. How long such a spin lasts, and whether it ends before the +holder is run elsewhere, is unmeasured; nothing in the tree contends this lock. + +The futex both need is there: libc's `futex_lock` and `futex_unlock` +(`userland/libc/src/pthread.rs`, lines 116–131), and std's `sync::Mutex`, which +is the futex mutex on ToyOS (`library/std/src/sys/sync/mutex/mod.rs`, over +`library/std/src/sys/pal/toyos/futex.rs`). + +Owner track: `issues/kernel/toyos-has-its-own-allocator.md`, whose std front +replaces std's allocator; it does not rule whether this lock is fixed on +`dlmalloc` first. Owner: the orchestrator. + +**Exit**: neither file's allocator spins: a contended allocation waits on a +futex and a release with waiters wakes one, or std's front of ToyOS's +allocator has replaced both. diff --git a/issues/build/the-sysroots-core-and-alloc-carry-the-compiler-profiles-debug-assertions.md b/issues/build/the-sysroots-core-and-alloc-carry-the-compiler-profiles-debug-assertions.md new file mode 100644 index 00000000000..70932ef9438 --- /dev/null +++ b/issues/build/the-sysroots-core-and-alloc-carry-the-compiler-profiles-debug-assertions.md @@ -0,0 +1,61 @@ +--- +status: open +kind: defect +opened: 2026-10-04 +--- + +# The sysroot's `core` and `alloc` carry the debug assertions of a profile written for compiler contributors + +`std_config` (`src/sysroot.rs`) writes `profile = "compiler"` into the +bootstrap configuration every ToyOS library is built under: the kernel's and +the loader's `core` and `alloc` (`Libraries::Freestanding`) as well as every +program's `std`. In the `rust/` fork at `a0d44493`, that profile +(`src/bootstrap/defaults/bootstrap.compiler.toml`) sets +`rust.debug-assertions = true`; bootstrap's `std_debug_assertions` falls back +to it when `rust.debug-assertions-std` is unset +(`src/bootstrap/src/core/config/config.rs`), which `std_config` leaves it, and +passes it to the library crates' cargo profile +(`src/bootstrap/src/core/builder/cargo.rs`). Upstream's distributed libraries +are built without. The libraries' assertions were never chosen: `std_config`'s +doc comment takes the profile so the libraries are built as the compiler was. +The guest profile keeps debug assertions on in ToyOS's own code "because +fail-fast beats speed here" (`kernel/Cargo.toml`, `userland/Cargo.toml`, the +root `Cargo.toml`); whether the libraries should too is the decision nobody +made. + +So every kernel and program ships `alloc`'s and `core`'s `debug_assert!`s, +`RawVecInner::finish_grow`'s alignment check among them +(`library/alloc/src/raw_vec/mod.rs`), and the precondition checks `ub_checks` +gates in the code compiled into those crates. The same profile's +`frame-pointers = true` reaches them as `-Cforce-frame-pointers=true`. + +**Measured.** A link-time no-panic proof: a `no_std` binary for +`x86_64-unknown-none` whose panic handler calls a symbol nobody defines, +linking a fallible parser. Its controls hold in every arm: the binary with no +parser links, and each of a `Vec::push`, a `handle_alloc_error`, an over-wide +shift and an out-of-range index fails to link. The parser, built with the +binary's own debug assertions off, links against stable 1.98.1's and nightly's +upstream libraries and fails against a ToyOS sysroot, at `opt-level = 2` and +under fat LTO alike. In the parser's arm that writes into a vector's spare +capacity and sets its length in `unsafe`, lld's `--why-live` names `core::panicking::panic` and `panic_fmt`, +live through `RawVecInner::finish_grow` alone; in its default arm +`alloc::raw_vec::handle_error` and `handle_alloc_error` are live beside them, +and against upstream's libraries nothing is. Under the guest +profile with `-Zub-checks=no` it links against stable's and fails against +ToyOS's. The matrix, the `--why-live` logs and the spike's source are a +comment on the pull request that added this file. So stage 2 of +`issues/kernel/a-panic-is-never-an-accident.md` can prove a parser panic-free +against an upstream library on the host, and against none the kernel links. + +**Not measured**: the `ub_checks` share of what the proof found, which no log +attributes; what the libraries' assertions, checks and frame pointers cost a +kernel or a program in time or in image bytes, on QEMU or the T14. + +Owner track: `issues/kernel/a-panic-is-never-an-accident.md`, stage 2. +Owner: the orchestrator. + +**Exit**: `std_config` sets `rust.debug-assertions-std` by a line of its own, +so the compiler profile decides nothing about the libraries' assertions; and +either it is `false` and the proof's parser, built with its own debug +assertions off, links against the sysroot that configuration builds, or a +ruling keeps it `true` and this file becomes `rejected`, citing the ruling. diff --git a/issues/design-debt/what-is-owed-on-file-size.md b/issues/design-debt/what-is-owed-on-file-size.md index c4dfc1547be..f9501fda2e5 100644 --- a/issues/design-debt/what-is-owed-on-file-size.md +++ b/issues/design-debt/what-is-owed-on-file-size.md @@ -23,7 +23,7 @@ what survives it is a different question. `userland/soundserver/mixer/`), 2026-08-20; the effects shell is `userland/soundd/` over eight files with a 261-line `main.rs`, from a 2,366-line one. What the note asked for is what it got: **the mixing is - sample-exact and proven so.** `toyos-mixer/fixtures/mix-corpus.txt` was + sample-exact and proven so.** `userland/soundserver/mixer/fixtures/mix-corpus.txt` was written by `soundd/src/main.rs` before a line of it moved, and `the_corpus_is_reproduced_bit_for_bit` holds the crate to it byte for byte. The transcript is compact because exhausted domains are digested rather than From cfb1cc1fd98eb2ee65fa7d56ec8e583b4b15b140 Mon Sep 17 00:00:00 2001 From: japabu Date: Sun, 4 Oct 2026 03:28:49 +0200 Subject: [PATCH 2/2] Issues: name which controls ran in which arm, leave the spinlock's contention unmeasured, and own the assertions file by the orchestrator alone Review round 1 of #707: the -Zub-checks=no rows ran only control-oom on stable and nightly and no failing control on the ToyOS sysroot; nothing measured whether two threads meet on the allocator lock; and stage 2 of a-panic-is-never-an-accident exits on upstream libraries, so it cannot own a defect in ToyOS's sysroot. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01WcU2Dsw6mDYtwYfzVHPzM8 --- ...libc-allocate-under-a-spinlock-with-no-futex.md | 5 ++++- ...carry-the-compiler-profiles-debug-assertions.md | 14 +++++++++----- 2 files changed, 13 insertions(+), 6 deletions(-) diff --git a/issues/build/std-and-libc-allocate-under-a-spinlock-with-no-futex.md b/issues/build/std-and-libc-allocate-under-a-spinlock-with-no-futex.md index 84037d5ead3..c4d7dd4c81f 100644 --- a/issues/build/std-and-libc-allocate-under-a-spinlock-with-no-futex.md +++ b/issues/build/std-and-libc-allocate-under-a-spinlock-with-no-futex.md @@ -25,7 +25,10 @@ real-time thread outranks a fair one by right (`issues/kernel/cpu-time-is-a-band-and-not-a-reservation.md`), so a real-time waiter on the holder's CPU keeps a fair holder off that CPU while it spins: priority inversion. How long such a spin lasts, and whether it ends before the -holder is run elsewhere, is unmeasured; nothing in the tree contends this lock. +holder is run elsewhere, is unmeasured, and so is whether two threads ever meet +on this lock: `thread::spawn` allocates, and several programs allocate from more +than one thread, among them `userland/sshserver`, `userland/supervisor` and +`userland/soundserver`. The futex both need is there: libc's `futex_lock` and `futex_unlock` (`userland/libc/src/pthread.rs`, lines 116–131), and std's `sync::Mutex`, which diff --git a/issues/build/the-sysroots-core-and-alloc-carry-the-compiler-profiles-debug-assertions.md b/issues/build/the-sysroots-core-and-alloc-carry-the-compiler-profiles-debug-assertions.md index 70932ef9438..d1ec40c37a7 100644 --- a/issues/build/the-sysroots-core-and-alloc-carry-the-compiler-profiles-debug-assertions.md +++ b/issues/build/the-sysroots-core-and-alloc-carry-the-compiler-profiles-debug-assertions.md @@ -31,9 +31,12 @@ gates in the code compiled into those crates. The same profile's **Measured.** A link-time no-panic proof: a `no_std` binary for `x86_64-unknown-none` whose panic handler calls a symbol nobody defines, -linking a fallible parser. Its controls hold in every arm: the binary with no -parser links, and each of a `Vec::push`, a `handle_alloc_error`, an over-wide -shift and an out-of-range index fails to link. The parser, built with the +linking a fallible parser. The binary with no parser links in every arm. Each +of a `Vec::push`, a `handle_alloc_error`, an over-wide shift and an +out-of-range index fails to link in every arm without `-Zub-checks=no`; with +it, only the `handle_alloc_error` control ran, against stable's and nightly's +libraries, and failed, and the ToyOS sysroot's rows ran no failing control. +The parser, built with the binary's own debug assertions off, links against stable 1.98.1's and nightly's upstream libraries and fails against a ToyOS sysroot, at `opt-level = 2` and under fat LTO alike. In the parser's arm that writes into a vector's spare @@ -51,8 +54,9 @@ against an upstream library on the host, and against none the kernel links. attributes; what the libraries' assertions, checks and frame pointers cost a kernel or a program in time or in image bytes, on QEMU or the T14. -Owner track: `issues/kernel/a-panic-is-never-an-accident.md`, stage 2. -Owner: the orchestrator. +Owner: the orchestrator. Stage 2 of +`issues/kernel/a-panic-is-never-an-accident.md` exits on a host step against +upstream libraries, so it does not wait on this file. **Exit**: `std_config` sets `rust.debug-assertions-std` by a line of its own, so the compiler profile decides nothing about the libraries' assertions; and