diff --git a/issues/audio/disk-wait-pins-a-cpu.md b/issues/audio/disk-wait-pins-a-cpu.md deleted file mode 100644 index a02bb9b617f..00000000000 --- a/issues/audio/disk-wait-pins-a-cpu.md +++ /dev/null @@ -1,29 +0,0 @@ ---- -status: open -kind: defect -opened: 2026-08-08 ---- - -# The audio pops are four spinlocks, not one spinning driver - -Measured 2026-08-08 on `wt/toyos-asyncusb` at `87835d1`: at the moment a disk -transfer is waited for, an ordinary guest is **four ticket spinlocks deep** — -`log_file::SINK`, `vfs::VFS`, `fat32_adapter::VOLUMES` and `xhci::XHCI` — and -every one of them disables preemption for its whole life. `io-depth-probe` -(kernel feature) reports the depth with the backtrace that got there: 4 from the -idle loop, 5 from a syscall. So **making `xhci::wait_transfer` park cannot work -and would not help**: `prepare_wait` asserts the depth equals the context's -baseline, so a park there is a named panic on the first flush — and the three -locks above `XHCI` would still hold the CPU if it were not. -`object/ops.rs:619` (`fsync`, formerly `fd.rs:644` before the handle-table -rewrite) puts every userland file write-back at the same depth, so this is not a property of -the log sink; it is that this kernel cannot touch a disk without pinning a CPU -for the whole device round trip. The log sink is only the writer that runs -continuously, which is why it is the one gate A sees. - -The log-flush deferral fix — whose affordability heuristic left the kernel with -the log architecture, so no CPU takes a flush now — and -[`stop-the-device-voice-keep-the-wake.md`](stop-the-device-voice-keep-the-wake.md) -are the fixes that came before. Each was right about its own defect; both chose -which CPU absorbs a stall whose duration neither touched. The pinning disk wait -is logd's `fsync` through the three spinlocks above. diff --git a/issues/hardware/the-t14s-firmware-interrupts-every-cpu-every-2-2-s-under-toyos.md b/issues/hardware/the-t14s-firmware-interrupts-every-cpu-every-2-2-s-under-toyos.md new file mode 100644 index 00000000000..04874b9c582 --- /dev/null +++ b/issues/hardware/the-t14s-firmware-interrupts-every-cpu-every-2-2-s-under-toyos.md @@ -0,0 +1,85 @@ +--- +status: open +kind: defect +opened: 2026-10-03 +--- + +# The T14's firmware interrupts every CPU every 2.2 s under ToyOS + +On LENOVO 20W0003AMZ, BIOS N34ET71W (1.71), one boot of `main` at `c59e09ed6` +with a throwaway instrument that reads `MSR_SMI_COUNT` (MSR 0x34) on each CPU +as each interrupts-off window opens and closes read the count from 1.20 s to +38.79 s into the boot, `test_rs_ring_park_herd` running forty times in it. Its +lines are quoted on #681 (comment 5962619169; the boot itself in comment +5962618149). + +- **It moved 17 times, from 4824 to 4841, on all eight CPUs alike**: at each of + the boot's 42 reports the eight read the same value. +- **About every 2.2 s.** Fifteen of the steps fall inside a window the + instrument printed, and the windows that caught consecutive steps close + 2,215,327,897 to 2,221,576,672 ns apart, median 2,216,534,952. +- **Each step stops every CPU for about 4.5 ms.** The 25 interrupts-off + windows a step falls in read 4,529,514 to 4,764,935 ns: sixteen open in a + job's syscall or interrupt, thirteen of them `SYS_SYSINFO`, and nine in a + pass leaving idle. None of those sections reads 2 ms where no step falls. At + 9.925 s all eight CPUs carry one at once. That the CPU stops is shown by two + CPUs waiting on a lock across a step, which went 4,503,694 and 4,554,675 ns + between two turns of their own spin. +- **Linux, in one condition, read none.** On the same machine under Ubuntu's + `6.8.0-142-generic`, `perf stat -a -A -e msr/smi/` read 0 on every CPU over + 120.378 s beside `rtla timerlat top -q -d 2m --dma-latency 0`, which holds + every CPU's C-states shallow and runs a 1 kHz timer on each. Linux idle has + not been read. + +What differs, as far as anything has read: + +- ToyOS performs no ACPI enable handshake: `rg -n -i + 'smi_cmd|acpi_enable|sci_en|smi_en|0xb2\b' kernel/src bootloader/src` finds + only the xHCI legacy handoff's own `USBLEGCTLSTS` enables + (`kernel/src/drivers/xhci/legacy.rs`), and nothing reads `PM1a_CNT.SCI_EN` + or the chipset's `SMI_EN`. Both controllers' `USBLEGCTLSTS` read + `0xe0000000`, no enable set, before the handoff wrote it. +- Under that Linux, read as root from `/sys/firmware/acpi/tables/FACP` and + `/dev/port` with nothing written (comment 5962768253): the FADT's + `SMI_CMD=0xb2` and `ACPI_ENABLE=0xf0`; `PM1a_CNT` (port 0x1804) `0x0001`, + `SCI_EN` set; `SMI_EN` (port 0x1830) `0x10002033`, bits 0 (`GBL_SMI_EN`), 1 + (`EOS`), 4 (`SLP_SMI_EN`), 5 (`APMC_EN`), 13 (`TCO_EN`) and 28. + +**The cause, measured, is the firmware's legacy mode.** A scout on `main` at +`73282fa93` (comment 5966817345) booted the T14 twice, each boot 30 s idle and +then 30 s with a spinning thread on every CPU: + +- With nothing written, `PM1a_CNT` read `0x0000`, `SCI_EN` clear, from 0.156 s + to 61.226 s, and `SMI_EN` read `0x10002033`, Linux's value. The count rose + by 14 in the idle 30 s and 13 in the busy 30 s, on every CPU alike. +- With `ACPI_ENABLE` written to `SMI_CMD` at 0.156 s, `SCI_EN` read set + 2,129,279 ns later. The write moved the writing CPU's count from 4817 to + 4818, and every CPU then read 4818 at every report through 61.233 s. The + SCI's line, GSI 9, stayed masked: nothing in ToyOS handles an SCI. Two GPEs + moved after the switch, in that boot's `kernel.log` lines quoted in comment + 5966982752 (the patch, comment 5966982939, prints each GPE register byte by + byte, lowest address first): + - the firmware's enable set GPE 9's enable bit: `gpe0_en` byte 1 reads `00` + at line 121, before the write, and `02` at line 123, after it; + - GPE 110's status rose after the switch and stayed set, served by nothing: + `gpe0_sts` byte 13 reads `00` at line 123 and `40` at every report from + line 298 through line 759. The boot with nothing written shows it in none + of its 25 reports (comment 5966982609). + +The `mask-windows` kernel prints such a window as its CPU's own: cpu4's +4,720,498 ns in the sixth of #649's `mask_windows` boots at `72f16e39a`, and cpu0's 4,725,822 in `649-r6/3-idle-halt-counted` +(`issues/kernel/a-process-lengthens-an-interrupts-off-walk-by-the-threads-it-parks-on-one-ring.md`), +are this by reading, those kernels not reading the count. + +**Owner**: the fix is stage 1 of +`issues/kernel/toyos-runs-the-machine-in-acpi-mode-and-interprets-its-aml.md`. +The T14 loop (`issues/hardware/the-t14-boots-toyos-unattended.md`) owns only +the reading: its jobs are the measurements owed on hardware, and it builds the +row. + +**Exit**: a T14 row reads `MSR_SMI_COUNT` on every CPU after init is spawned +and after the boot's last write to `SMI_CMD`, whoever makes it, and again at +the stop's report at least 4.444 s later, two of the longest period read, and +on every CPU the two agree. The interval opens after that write because the +ACPI enable is one, a write of `ACPI_ENABLE` to `SMI_CMD`, and raises one +firmware interrupt where `APMC_EN` is set. diff --git a/issues/hardware/xhci-waits-are-spins.md b/issues/hardware/xhci-waits-are-spins.md index 7688c20d68d..d13c2c735a4 100644 --- a/issues/hardware/xhci-waits-are-spins.md +++ b/issues/hardware/xhci-waits-are-spins.md @@ -4,44 +4,108 @@ kind: defect opened: 2026-08-03 --- -# The xHCI driver's waits are spins with preemption disabled, wherever they run - -`bdf2596` moved the *boundary* — an input read no longer drives the driver — so -the only thread that runs enumeration and recovery now is the one inside -`drain_irqs`. That fixes who pays; it does not change what is paid. - -Every wait in this driver is a spin against a wall-clock deadline, taken while -holding `XHCI`, which is a ticket spinlock and therefore preemption off for its -whole life: - -- `settles()` — controller halt, HCRST, CNR, R/S, and the port reset. Bound - `USB_TIMEOUT_NS`, 2 s. -- `wait_command()` and `wait_transfer()` — every command and every transfer. - Same bound. - -**The port machine took the two that ran inside a scheduler pass out of that -list.** A teardown's Disable Slot and an endpoint recovery's three-in-a-row -(Reset or Stop Endpoint, Set TR Dequeue, CLEAR_FEATURE(HALT)) are -submit-and-return now, so the six seconds above are reachable only from the boot -path and from `storage_read`/`storage_write` — the first has no scheduler to -give a pass back to, and the second is the case named below that this conversion -does not fix. - -So a worst case is a CPU that does not reschedule for **six seconds**, and an -ordinary hot-plug enumeration on the T14 is ~14 ms of it (`hotplug-blocks-a-scheduler-pass`). -Nothing in the suite can measure the bad case: QEMU answers every one of these -in microseconds, which is why a driver built entirely out of them passed -everything here for a season. - -**The hot-plug half of that conversion has landed**: enumeration is -`device::begin`/`stepped`, submit-and-return, driven by the port machine — so -no runtime path blocks a scheduler pass on it any more. `restart_endpoint`'s -half is done the same way: the route is `toyos_xhci::recovery`'s, driven twice -— a blocking loop for a disk's bulk pair, which runs on the thread that -faulted, and a stepped one for HID. - -One case is *not* fixed by that and needs its own answer: `storage_read` and -`storage_write` are called by the page cache on a faulting thread, so a thread -touching a file on a USB disk drives a SCSI command under the same lock. The -input poll was gratuitous and could simply be deleted; this one is inherent, and -the choice is between an I/O thread and making the block layer asynchronous. +# The xHCI driver's waits are spins, and a USB disk call spins with interrupts masked + +Every wait in the kernel's xHCI driver spins against a wall-clock deadline +while holding `XHCI` (`kernel/src/drivers/xhci/mod.rs`), a ticket spinlock and +so preemption off for its whole life: `settles()` for the controller's halt, +reset and port reset, and `wait_command()` and `wait_transfer()` for every +command and transfer, each bounded by `USB_TIMEOUT_NS`, 2 s. The port +machine's enumeration and HID recovery submit and return; what still spins is +the boot path's bring-up, the stop, a disk's bind after boot, inside a +scheduler pass +(`issues/hardware/a-disk-plugged-in-after-boot-is-bound-inside-a-scheduling-pass.md`), +and a disk call. + +A call on a USB disk holds `XHCI` from its first command (`with_disk`, +`kernel/src/drivers/xhci/wait/msc.rs`) inside a syscall, which runs with +interrupts masked from entry to exit +(`issues/kernel/syscall-preemption-is-incidental.md`): a partition claim's +`SYS_PARTITION_READ`, `SYS_PARTITION_WRITE` and `SYS_FSYNC`, the last a cache +flush through `xhci::storage_flush` (`partition_fsync`, +`kernel/src/object/ops.rs`), and the table `SYS_DEVICE_CLAIM` reads for one +(`gpt::claimable`, `kernel/src/gpt.rs`). logd's `fsync` is one of these: the +LOG fsd answers each with `SYS_FSYNC` on its claim. So its CPU holds +interrupts and preemption off for as long as the device takes, and a CPU whose +TLB shootdown waits on that CPU's acknowledgement spins as long, masked. + +**Its bound outruns the TLB-ack tripwire.** `time::DEAF_CPU` (5 s), past which +a CPU waiting on an acknowledgement panics, is held above `CALL_AFTER_BREAK` +(4.75 s), the longest a disk call spins once its transport has broken. But that +bound opens at the wait that broke, and `transfer_blocks` starts a batch while +the operation's 2 s `block::OPERATION` has any left: a batch that starts at +1.99 s and breaks runs its ladder to 6.74 s after the operation began, all of +it with `IF` clear. That is arithmetic on the declared constants; no boot has +been seen to do it. + +## On the T14 + +On LENOVO 20W0003AMZ, BIOS N34ET71W (1.71), `/log` is a FAT32 partition of the +USB stick the machine boots from, which `/system/bin/fsd` in the LOG role +writes with `SYS_PARTITION_WRITE`. One boot of `main` at `c59e09ed6` with a +throwaway instrument that records every window of 2 ms or more with the +addresses that opened and closed it, its CPU's last events, and samples of +that CPU every 2 ms ran `test_rs_ring_park_herd` forty times. Its lines are +quoted on #681 (comments 5962618149, 5962618492 and 5962618868). Pid 4 is that +fsd. + +- **Eleven writes of 7.8 ms or more in 37 s**, 2.0 to 4.6 s apart, each window + opened at the syscall's entry and closed at its return. Nine are on one CPU + alone: 8,289,211 to 10,669,042 ns. 36 of their 37 samples are in + `wait_transfer < bulk < bot < scsi < transfer_blocks` under `storage_write`; + the 37th is the same syscall waiting for the process table's lock. +- **All eight CPUs at once, twice**, at 17.416 s and 21.116 s: the write on one + CPU (7,848,573 and 7,927,725 ns) while a herd thread on each of the other + seven is in `SYS_THREAD_EXIT`, whose unmap shoots down every CPU's TLB. The + fourteen shootdowns span 7,703,612 to 7,852,515 ns, and their initiators + spin: none goes more than 10,819 ns between two turns of its wait. The other + targets acknowledge within 24 µs; the writing CPU acknowledges by interrupt + once its syscall has returned, 130 to 264 ns before each shootdown ends. Each + event is followed at once by a shorter one of the same shape: the next + write, 2,824,878 and 2,846,782 ns, and fourteen more shootdowns. The + report's `tlb:` line reads `max=7846us` after the first, where it read + `max=1146us` before it. +- **Shorter**: sixteen of the same server's writes read 2,368,371 to + 2,905,716 ns, those two among them, and two of its `SYS_PARTITION_READ` + 2,596,003 and 2,954,780. At the stop, the black box keeps a write opened at + 38.795 s whose CPU's next event comes 7,953,409 ns later. +- **init's partition claims**: on cpu0, 6,002,063 ns at 1.171 s and + 6,623,907 ns at 1.179 s, each `SYS_DEVICE_CLAIM` from entry to return, all + four samples in `wait_transfer` under `storage_read`. +- **Its `SYS_FSYNC`**: 15 windows' trails carry one, and in 7 it spins on a + contended `XHCI` at `with_disk_by` (comment 5966523482). None held + interrupts off for 2 ms. +- **Not the firmware**: `MSR_SMI_COUNT` does not move across any of them. The + windows it does move across are + `issues/hardware/the-t14s-firmware-interrupts-every-cpu-every-2-2-s-under-toyos.md`'s. + +The `mask-windows` kernel (`kernel/src/windows.rs`) prints a span and no +opener, and #649's T14 boots printed these by reading: + +- cpu0's line before the first job's exit, init's partition claims: + 6,540,326 to 6,590,095 ns in seven of the eight `mask_windows` boots at + `72f16e39a`, and 6,533,222, 2 × 3,273,547 and 6,508,165 in the first report + of the three at `8b73eba69` (comment 5959415453). +- Every CPU at once: 9,603,617 to 9,651,846 ns beside `tlb: … max=9575us`, the + fourth boot at `72f16e39a`; 2 × 5,014,552 to 2 × 5,075,290 beside + `max=10038us`, `649-r6/2-report-halved` (comment 5960575031). +- One CPU alone: 9,260,193, the seventh boot at `72f16e39a`; cpu7, where pid 4 + started, at 11,492,028 and 8,456,658 before the first job's exit + (`649-r6/1-head`, `649-r6/3-idle-halt-counted`). +- The stop's report: one CPU at 10,214,820 to 11,267,384 in four boots at + `72f16e39a`, and two in `649-r5/1-head`, cpu7 at 11,522,834 and cpu4 at + 11,645,411 (`loader.log:57`, `:63`). + +**Owner**: `issues/kernel/the-kernel-is-small-interrupts-post-and-threads-wait.md`, +whose step 10 moves the whole xHCI to usbd and deletes the kernel's driver. It +owes both exits. + +**Exit**, two: +- **The tripwire's, short of step 10**: the whole of one operation's `IF`-clear + spin is under `DEAF_CPU` by construction, because the call's bound opens + where the operation opens, or a later batch starts only while a whole call's + bound is still inside it; and a staged boot whose first batch spends most of + the budget and whose next batch breaks shows the operation ending inside + `DEAF_CPU`. +- **The spin's**: the tree has no `kernel/src/drivers/xhci`, so no kernel wait + is a USB device's. Step 10 meets both. diff --git a/issues/kernel/a-disk-operation-can-spin-past-the-tlb-ack-tripwire-before-its-break.md b/issues/kernel/a-disk-operation-can-spin-past-the-tlb-ack-tripwire-before-its-break.md deleted file mode 100644 index ee0f09fa33f..00000000000 --- a/issues/kernel/a-disk-operation-can-spin-past-the-tlb-ack-tripwire-before-its-break.md +++ /dev/null @@ -1,26 +0,0 @@ ---- -status: open -kind: defect -opened: 2026-09-22 ---- - -# A disk operation can spin past the TLB-ack tripwire before its break - -`time::DEAF_CPU` (5 s), the TLB-ack tripwire, is held above xHCI's `CALL_AFTER_BREAK` -(4.75 s), "the longest a disk call spins with `IF` clear once its transport has -broken". But the call's bound is measured from the wait that broke, and a USB -disk operation holds the controller lock — `IF` clear — from its first command: -`transfer_blocks` sends one command per `MSC_MAX_BLOCKS` batch, and starts a -batch while the operation's 2 s `block::OPERATION` budget has any left. A batch -that starts at 1.99 s and breaks runs its ladder to 1.99 + 4.75 = 6.74 s after -the operation began, all of it with `IF` clear: past the tripwire on any CPU -waiting for this one's TLB acknowledgement. That is arithmetic on the declared -constants; no boot has been seen to do it. - -## Exit condition - -The whole of one operation's `IF`-clear spin is under `DEAF_CPU` by -construction — the call bound opens where the operation does, or a later batch -starts only with a whole call's bound still inside it — and a staged boot in -which the first batch spends most of the budget and the next one breaks shows -the operation ending inside it. diff --git a/issues/kernel/a-process-lengthens-an-interrupts-off-walk-by-the-threads-it-parks-on-one-ring.md b/issues/kernel/a-process-lengthens-an-interrupts-off-walk-by-the-threads-it-parks-on-one-ring.md index e10367acbf8..d0dc5d77598 100644 --- a/issues/kernel/a-process-lengthens-an-interrupts-off-walk-by-the-threads-it-parks-on-one-ring.md +++ b/issues/kernel/a-process-lengthens-an-interrupts-off-walk-by-the-threads-it-parks-on-one-ring.md @@ -57,13 +57,15 @@ mutated kernels) read the same report of the same load at 1980239 969922 to 1044783. - 2 × 5075290 is one of eight: every CPU reads 2 × 5014552 to 2 × 5075290 (`:416` to `:430`), beside `tlb: shootdowns=282 wait=131633us max=10038us` - (`:431`). It is the first reading of - `issues/kernel/some-t14-boots-carry-an-interrupts-off-window-of-9-ms-or-more.md`. -- 4725822 is cpu0's. The other seven read 544283 to 1018355. + (`:431`). It is a log write's all-CPU shape + (`issues/hardware/xhci-waits-are-spins.md`) by reading. +- 4725822 is cpu0's, a firmware step's length by reading + (`issues/hardware/the-t14s-firmware-interrupts-every-cpu-every-2-2-s-under-toyos.md`). + The other seven read 544283 to 1018355. -So one size reads 1.03 to 10.15 ms over six boots, and which boot carried -one of the machine's own events decides a single reading. None of the six -took an interrupt a handler #634 changed serves +So one size reads 1.03 to 1.98 ms in the four of those reports none of the +machine's own events reached, and which boot carried one decides a single +reading. None of the six took an interrupt a handler #634 changed serves (`userdev=0 sound=0 dmafault=0 hda=0` on every CPU). **Exit**: the interrupts-off window step 2's instrument reads on the T14 diff --git a/issues/kernel/a-shutdown-on-a-held-usb-disk-left-a-cpu-deaf-to-a-tlb-shootdown.md b/issues/kernel/a-shutdown-on-a-held-usb-disk-left-a-cpu-deaf-to-a-tlb-shootdown.md index 1baf4cb4031..79cca8589e3 100644 --- a/issues/kernel/a-shutdown-on-a-held-usb-disk-left-a-cpu-deaf-to-a-tlb-shootdown.md +++ b/issues/kernel/a-shutdown-on-a-held-usb-disk-left-a-cpu-deaf-to-a-tlb-shootdown.md @@ -29,8 +29,8 @@ accord is logd's network thread, on a boot that gives logd no netd; the logd branch no longer starts that thread there, which would remove this boot's trigger and not the deaf CPU. -`a-disk-operation-can-spin-past-the-tlb-ack-tripwire-before-its-break.md` is -the arithmetic for one disk operation outrunning `time::DEAF_CPU`; this is a boot +`issues/hardware/xhci-waits-are-spins.md` carries the arithmetic for one disk +operation outrunning `time::DEAF_CPU`; this is a boot that did outrun it, in `quiesce`, across several operations each inside its own budget. Whether `quiesce` holds `IF` clear between them is not measured. diff --git a/issues/kernel/cpu0-holds-interrupts-and-preemption-off-for-6-5-ms-on-the-t14.md b/issues/kernel/cpu0-holds-interrupts-and-preemption-off-for-6-5-ms-on-the-t14.md deleted file mode 100644 index 19ebc5402e4..00000000000 --- a/issues/kernel/cpu0-holds-interrupts-and-preemption-off-for-6-5-ms-on-the-t14.md +++ /dev/null @@ -1,58 +0,0 @@ ---- -status: open -kind: defect -opened: 2026-10-02 ---- - -# cpu0 holds interrupts and preemption off for 6.5 ms on the T14 - -Read by the `mask-windows` kernel (`kernel/src/windows.rs`) on LENOVO -20W0003AMZ, BIOS N34ET71W (1.71), in the eight `mask_windows` boots of #649 -at `72f16e39a`: four as that head stands and four with stage 6 step 1 (#634) -reverted on it. Each boot printed one report, at `test_rs_ring_park_herd`'s -exit, so a line is its CPU's longest window from the moment it joined the -scheduler to that exit. - -cpu0's line, in the seven boots where no longer window covers it: - -| boot | `irqs_off_ns` | `preempt_off_ns` | -|---|---|---| -| 1, reverted | 6551791 | 6551707 | -| 2 | 6540754 | 6540671 | -| 3, reverted | 6549549 | 6549484 | -| 5, reverted | 6540326 | 6540262 | -| 6 | 6589053 | 6588992 | -| 7, reverted | 6590095 | 6590031 | -| 8 | 6587342 | 6587281 | - -- The two kinds differ by 61 to 84 ns in every row, so one section holds - both. -- It is cpu0's alone. In the six of those boots where no CPU carries a - longer one, the other seven CPUs' interrupts-off lines read 251,452 to - 4,720,498 ns. -- It is there with step 1 and without it. -- In all eight logs cpu0 says `sched: cpu=0 ready=0 … current=None trips=1` - at 1.155 or 1.156 s, as the CPUs join the scheduler, and nothing more - before init's first line at 1.161 or 1.162 s. - -The fourth boot's cpu0 reads 9644415 and 9643939, the event -`issues/kernel/some-t14-boots-carry-an-interrupts-off-window-of-9-ms-or-more.md` -records. - -Nothing names what opens it. A report carries a span and no address, and one -report a boot cannot say whether the window recurs. What will: the record -keeping, beside each CPU's longest span, the address its opening hook was -called from, printed in the report and resolved through the kernel's symbols. - -It opens before the boot's first job ends. The three boots of #649 at -`8b73eba69` (comment 5959415453, readbacks `649-r5/1-head`, -`649-r5/2-report-halved`, whose kernel prints half of every span, and -`649-r5/3-idle-halt-counted`) print a report at each of four exits. cpu0's -`irqs_off_ns` in the first, at `test_rs_idle_span`'s exit 1.746 s into the -boot (`windowscase/kernel.log:348` in each), reads 6533222, 2 × 3273547 and -6508165. In the three reports after it cpu0 reads at most 943904, 2 × 472938 -and 345229. - -**Exit**: the section that opens the window is named by that reading on the -T14, and cpu0's longest window there no longer includes it, or this file is -replaced by the bound the section is held to and the derivation of it. diff --git a/issues/kernel/idle-cpus-enter-the-c-states-cst-names.md b/issues/kernel/idle-cpus-enter-the-c-states-cst-names.md index ea53ca33f83..fda05c16c34 100644 --- a/issues/kernel/idle-cpus-enter-the-c-states-cst-names.md +++ b/issues/kernel/idle-cpus-enter-the-c-states-cst-names.md @@ -7,8 +7,9 @@ opened: 2026-09-29 # Idle CPUs enter the C-states `_CST` names The kernel runs no `mwait` and reads no `_CST`. The states come from `_CST`, -as Linux takes the T14's, after the AML interpreter that -`issues/kernel/the-scheduler-and-the-clocks-never-talk.md` waits on. +as Linux takes the T14's, after the AML interpreter +`issues/kernel/toyos-runs-the-machine-in-acpi-mode-and-interprets-its-aml.md` +records ToyOS lacks. **Exit**: on the T14, since a KVM guest's idle is its host's: the idle minute's package energy, and timer lateness no worse than `hlt`'s by more than diff --git a/issues/kernel/inits-claim-of-a-pci-function-the-t14-lacks-holds-interrupts-off-for-3-8-ms.md b/issues/kernel/inits-claim-of-a-pci-function-the-t14-lacks-holds-interrupts-off-for-3-8-ms.md new file mode 100644 index 00000000000..b1e84d61685 --- /dev/null +++ b/issues/kernel/inits-claim-of-a-pci-function-the-t14-lacks-holds-interrupts-off-for-3-8-ms.md @@ -0,0 +1,32 @@ +--- +status: open +kind: defect +opened: 2026-10-02 +--- + +# init's claim of a PCI function the T14 lacks holds interrupts off for 3.8 ms + +On LENOVO 20W0003AMZ, BIOS N34ET71W (1.71), one boot of `main` at `c59e09ed6` +with a throwaway instrument that names a window's opener and samples its CPU +every 2 ms (#681, comment 5962618149) read init's `SYS_DEVICE_CLAIM` for +blockd's `pci:1b36:0010`, which this machine does not have, holding cpu0's +interrupts off for 3,817,265 ns at 1.162 s, from the syscall's entry to its +return, with `MSR_SMI_COUNT` unmoved. Its one sample, 1.7 µs before it closed, +is in `PciDevice::is_id` under `pcidev::claim`. By reading, the claim reads the +vendor ID of each of the 24 functions the kernel enumerated +(`kernel/src/pcidev/mod.rs`); nothing has said what it spends 3.8 ms on. + +Before the first job's exit cpu0 also carries init's partition claims, 6.0 +and 6.6 ms in that boot (`issues/hardware/xhci-waits-are-spins.md`), and the +`mask-windows` kernel (`kernel/src/windows.rs`) prints each CPU's longest +window and no opener, so its cpu0 line there reads this claim only once those +have left the kernel. + +**Owner**: `issues/kernel/the-kernel-is-small-interrupts-post-and-threads-wait.md`, +whose stage 5 exits on no interrupts-off window longer than a register access, +and whose step 10 takes the partition claims off cpu0. + +**Exit**: a T14 boot reads the longest interrupts-off window inside that +claim no longer than one register access, with `MSR_SMI_COUNT` unmoved across +that window. A figure for that access comes from the claim's own reads timed +on the T14, and nobody has timed them yet. diff --git a/issues/kernel/scheduler-pass-blocks-in-xhci.md b/issues/kernel/scheduler-pass-blocks-in-xhci.md index 8fc7d5c0f3b..ac3cd70c74e 100644 --- a/issues/kernel/scheduler-pass-blocks-in-xhci.md +++ b/issues/kernel/scheduler-pass-blocks-in-xhci.md @@ -36,7 +36,7 @@ handing it on, gated by simulator invariant I14. This half is not. The second instance of the same shape used to be the idle loop's log flush; the idle loop touches no filesystem now — the log is logd's file — and the disk wait that survives, logd's `fsync` pinning a CPU for the device round -trip, is `issues/audio/disk-wait-pins-a-cpu.md`'s subject. +trip, is `issues/hardware/xhci-waits-are-spins.md`'s subject. Closing this means making xHCI enumeration and endpoint recovery asynchronous, so that `drain_irqs` only ever does work it can finish: drain the event ring, diff --git a/issues/kernel/some-t14-boots-carry-an-interrupts-off-window-of-9-ms-or-more.md b/issues/kernel/some-t14-boots-carry-an-interrupts-off-window-of-9-ms-or-more.md deleted file mode 100644 index 8d70322a2f2..00000000000 --- a/issues/kernel/some-t14-boots-carry-an-interrupts-off-window-of-9-ms-or-more.md +++ /dev/null @@ -1,94 +0,0 @@ ---- -status: open -kind: defect -opened: 2026-10-02 ---- - -# Some T14 boots carry an interrupts-off window of 9 ms or more - -Read by the `mask-windows` kernel (`kernel/src/windows.rs`) on LENOVO -20W0003AMZ, BIOS N34ET71W (1.71), in the eight `mask_windows` boots of #649 -at `72f16e39a`: four as that head stands and four with stage 6 step 1 (#634) -reverted on it, one job each, `test_rs_ring_park_herd`. The longest window -that recurs in every boot is cpu0's 6.5 ms -(`issues/kernel/cpu0-holds-interrupts-and-preemption-off-for-6-5-ms-on-the-t14.md`). -Three readings are longer, each in some boots and not in others, on both -arms: - -- **Every CPU at once, the fourth boot** (step 1 in). The herd's report: - - | CPU | `irqs_off_ns` | `preempt_off_ns` | - |---|---|---| - | cpu0 | 9644415 | 9643939 | - | cpu1 | 9603617 | 9588515 | - | cpu2 | 9627971 | 9625674 | - | cpu3 | 9641999 | 9620791 | - | cpu4 | 9651846 | 9635137 | - | cpu5 | 9623545 | 9612295 | - | cpu6 | 9624969 | 9621345 | - | cpu7 | 9614836 | 9614716 | - - The same report's shootdown line reads - `tlb: shootdowns=280 wait=125911us max=9575us`, where the other seven - boots read at most `wait=49119us` and at most `max=1421us`, and cpu0's - census reads `nmi=2`, where the other seven read `nmi=1`. -- **One CPU, the seventh boot** (step 1 reverted). cpu7 reads 9260193 and - 9260124. cpu0 reads its 6590095, the other six 920,455 to 1,404,001, and - the shootdown line `max=1386us`. -- **One CPU, after the herd's report, in four boots of the eight.** The stop - prints a report too, of which the black box keeps the lines of cpu2 to - cpu7. In it one CPU reads - - | boot | CPU | `irqs_off_ns` | `preempt_off_ns` | - |---|---|---|---| - | 1, reverted | cpu7 | 10677997 | 10677829 | - | 2 | cpu7 | 11267384 | 11267320 | - | 5, reverted | cpu2 | 10214820 | 10214656 | - | 6 | cpu7 | 10874427 | 10874364 | - - and in the other four cpu7 reads 1,424,134 to 1,532,702 ns and no kept - line more. What that report spans is the herd's teardown, which its own - report precedes, the spawn of `reboot` on cpu7 and the stop. - -By reading, not measured: every CPU waiting out one CPU's masked window -inside a shootdown would read as the fourth boot does. - -Nothing names what opens any of them. A report carries a span and no address, -and no time: eight equal spans are not shown to be one event. What will: the -record keeping, beside each CPU's longest span, the address its opening hook -was called from and the counter it closed at, printed in the report. - -The third reading falls on the `reboot` and stop side of the herd's -teardown. The three boots of #649 at `8b73eba69` (comment 5959415453, -readbacks `649-r5/1-head`, `649-r5/2-report-halved`, whose kernel prints half -of every span, and `649-r5/3-idle-halt-counted`) took a report at `echo`'s -exit, after the herd's and before `reboot` was spawned. Its longest -`irqs_off_ns` is cpu7's in each: 1499808, 2 × 748872 and 1479558 -(`windowscase/kernel.log:456`). The stop's report after it, in the six lines -the black box keeps, reads cpu7 11522834 and cpu4 11645411 in the first boot -(`649-r5/1-head/windowscase/loader.log:57`, `:63`), and at most cpu7's -2 × 3799327 in the second and cpu7's 1305656 in the third. - -The three boots at `0aa8d4c88` (comment 5960575031, readbacks -`649-r6/1-head`, `649-r6/2-report-halved` and `649-r6/3-idle-halt-counted`, -the same two mutated kernels) print a report at three exits, `idle_span`'s, -`pwd`'s and the herd's, and at the stop: - -- **The first reading recurred, inside the herd's own report**, which opens - at `pwd`'s exit. In `2-report-halved` every CPU reads 2 × 5014552 to - 2 × 5075290 (`windowscase/kernel.log:416` to `:430`), beside - `tlb: shootdowns=282 wait=131633us max=10038us` (`:431`): the shootdown's - `max` is beside it again, 9575us then and 10038us now, and cpu0's second - NMI is not: `nmi=1` (`:415`). That is two boots of #649's fourteen. -- **One CPU, before the first job's exit.** cpu7's line of the first report, - `idle_span`'s (`:362`), reads 11492028 in `1-head`, 2 × 1308074 in - `2-report-halved` and 8456658 in `3-idle-halt-counted`, beside - `max=1655us`, `max=1451us` and `max=1353us` (`:363`). The same line read - 2327848, 2 × 1831727 and 3630933 at `8b73eba69`. -- **Nothing after the herd's report.** The stop's report, in the six lines - the black box keeps, reads at most cpu7's 1396887, 2 × 758960 and 1470910 - (`windowscase/loader.log:57`). - -**Exit**: each of the three, and the one-CPU reading before the first job's -exit, is named by that reading on the T14, and is removed, or this file is -replaced by the bound it is held to and the derivation of it. diff --git a/issues/kernel/the-global-pipe-lock-spans-a-user-copy.md b/issues/kernel/the-global-pipe-lock-spans-a-user-copy.md index a4ab14f071b..ae961f8815f 100644 --- a/issues/kernel/the-global-pipe-lock-spans-a-user-copy.md +++ b/issues/kernel/the-global-pipe-lock-spans-a-user-copy.md @@ -28,7 +28,7 @@ Everything pipe-shaped in the kernel goes through the same static: `create` (`pi This is **not** the device-I/O class. The copy is memory-speed and takes no device round trip, so it does not approach `sync::Lock`'s 500,000,000-spin `DEADLOCK` panic (`sync.rs:72-75`) the way a `block::Handle`'s device lock and `vfs::VFS` do — that class is tracked in `issues/kernel/every-wait-in-this-kernel-is-a-spin.md` and is not what this file is about. -What it is: a global lock held with preemption disabled for a stretch that scales with a userland-chosen length up to 2 MiB, on the path that carries every IPC message in the system. The scheduler's own bar for an uninterruptible stretch is `MAX_PASS_NS = 200_000` ns (`toyos-sched/src/cpu.rs:1235`); a 2 MiB copy plus, on first write, a 2 MiB zeroing and a bitmap scan is on the wrong side of that at any plausible memory bandwidth. **The hold time is not measured and that is the first thing owed** — `io-depth-probe` and the `audio_tone` worst-wake instrument are the tools that would price it, the same pair `issues/audio/disk-wait-pins-a-cpu.md` used for the device path. +What it is: a global lock held with preemption disabled for a stretch that scales with a userland-chosen length up to 2 MiB, on the path that carries every IPC message in the system. The scheduler's own bar for an uninterruptible stretch is `MAX_PASS_NS = 200_000` ns (`toyos-sched/src/cpu.rs:1235`); a 2 MiB copy plus, on first write, a 2 MiB zeroing and a bitmap scan is on the wrong side of that at any plausible memory bandwidth. **The hold time is not measured and that is the first thing owed.** Two things make it worth closing rather than noting. The convoy is machine-wide, not per-pipe: one process reading 2 MiB out of a full pipe stalls an unrelated process's `SYS_CONNECT` and an unrelated inbox's readiness poll. And it is directly userland-driven — the length is the caller's, so a program that wants the stall only has to ask for it. diff --git a/issues/kernel/the-scheduler-and-the-clocks-never-talk.md b/issues/kernel/the-scheduler-and-the-clocks-never-talk.md index 3e3e4cd0d92..c9a9165e011 100644 --- a/issues/kernel/the-scheduler-and-the-clocks-never-talk.md +++ b/issues/kernel/the-scheduler-and-the-clocks-never-talk.md @@ -19,8 +19,8 @@ Two couplings every production scheduler has and ours lacks: wake. A latency-sensitive wake should prefer a shallow-idle core; a throughput task can pay to wake a deep one. -Both wait for the ACPI/power track (task #136 — own AML interpreter, battery -first) because today the kernel neither reads nor sets P-states or C-states on -metal. Filed so the placement track (see -`placement-is-blind-to-caches-and-topology.md`) leaves room for an idle-state -input when it designs wake placement. +Both wait for an AML interpreter +(`issues/kernel/toyos-runs-the-machine-in-acpi-mode-and-interprets-its-aml.md`), +because today the kernel neither reads nor sets P-states or C-states on metal. +Filed so the placement track (see `placement-is-blind-to-caches-and-topology.md`) +leaves room for an idle-state input when it designs wake placement. diff --git a/issues/kernel/toyos-runs-the-machine-in-acpi-mode-and-interprets-its-aml.md b/issues/kernel/toyos-runs-the-machine-in-acpi-mode-and-interprets-its-aml.md new file mode 100644 index 00000000000..75a57722416 --- /dev/null +++ b/issues/kernel/toyos-runs-the-machine-in-acpi-mode-and-interprets-its-aml.md @@ -0,0 +1,33 @@ +--- +status: open +kind: track +opened: 2026-10-03 +--- + +# ToyOS runs the machine in ACPI mode and interprets its AML + +ToyOS leaves the machine in the mode its firmware hands over: nothing writes +`ACPI_ENABLE` to `SMI_CMD`, and nothing handles an SCI. The T14's firmware +hands it over in legacy mode, which interrupts every CPU every 2.2 s +(`issues/hardware/the-t14s-firmware-interrupts-every-cpu-every-2-2-s-under-toyos.md`). + +Nothing in the tree evaluates the AML in a machine's DSDT or SSDTs. What +ToyOS takes from them it takes another way: `toyos-acpi/src/dsdt.rs` finds +`\_S5_` by a byte scan, and the loader asks UEFI for the root bridges' +windows `_CRS` would name (`bootloader/src/rootbridge.rs`). Nothing reads +`_CST`, which names a CPU's C-states. + +**Ruled** (owner, 2026-10-03, on the scout in #681's comment 5966817345): the +switch to ACPI mode lands as stage 1, together with a userland server that +claims the SCI and handles the power button, so nothing the firmware does in +legacy mode today is lost; not the switch alone with the events masked, and +not after the AML work. The AML interpreter follows as later stages: ToyOS +writes its own, and the battery comes first (his direction of `0ee814f5a`). + +**Stage 1: ACPI mode, its SCI served in userland.** The switch to ACPI mode, +and a userland server that claims the SCI and handles the power button, a +fixed event that needs no AML. **Exit**: on the T14, `MSR_SMI_COUNT` stays +flat on every CPU over the interval the firmware issue's exit defines, and a +press of the power button stops the machine cleanly, through ToyOS's own +power-off path (`SYS_SHUTDOWN`), and the boot's log records the press and that +stop: a T14 row reads both there.