diff --git a/issues/boot-media/the-loader-does-only-what-must-precede-the-handover.md b/issues/boot-media/the-loader-does-only-what-must-precede-the-handover.md index 1c3aec6e7e..9718d54d13 100644 --- a/issues/boot-media/the-loader-does-only-what-must-precede-the-handover.md +++ b/issues/boot-media/the-loader-does-only-what-must-precede-the-handover.md @@ -224,8 +224,9 @@ Each stage lands on its own, in this order. label `ready`, and the service writes one byte to it once it serves. A read end that closes with no byte is a service that never came up. - Init runs `update --good` once every service `[boot] up` names has - written its byte; `update` holds the `slots` claim's table and writes the - running slot's good flag and nothing else. + written its byte, and never for an image that names no `[boot] up`; + `update` holds the `slots` claim's table and writes the running slot's + good flag and nothing else. - An image whose `system.toml` grants no program the `slots` claim is never good, so each boot spends a try and it boots three times. Today only `system.toml` and `tests/updatecase/system.toml` grant it. @@ -309,13 +310,35 @@ Each stage lands on its own, in this order. boot manager booting the entry the loader wrote. 6. **The T14 bench.** + - It opens with a timed `ssh t14 update < image` of a session image of + `issues/hardware/the-t14-reboots-through-ubuntu-for-every-test.md`, since + an image switch costs that write and a reset: ToyOS has written this + stick at 0.2 s to 9.5 s per MiB. - It is built from #539's pieces that stage 6 takes above, and the bench path of `src/metal.rs`. `--via-ubuntu` stays as the old path. + - Every T14 image is signed with the bench key the T14 trusts, and a loader + reaches the T14 as an image does: ToyOS receives it over ssh and writes it + to the stick, and the running loader tries it once and keeps the old one + as the fallback (owner rulings). A stick that boots neither loader costs + a hand and `diag/flash.sh`. + - The host's first exec on an image is `update --good`, and a session image + names no `[boot] up`. A death goes by `update --once`, the kept slot boots + after it, and the host fetches the record over `sftp`. - The bench reads the loader's file off the ESP. - The tested pass's `loader.log` is kept as `loader-previous.log` by a rename (`SetInfo`), not by #539's copy. - It runs over the cable that - `issues/hardware/the-t14-answers-only-through-a-usb-stick.md` owns. + `issues/hardware/the-t14-answers-only-through-a-usb-stick.md` owns, and + waits on + `issues/isolation/a-reset-stops-xhci-and-leaves-every-claimed-pci-function-armed.md`, + `issues/hardware/the-t14-hung-after-rebooting-with-its-i219-faulted.md`, + `issues/panic-path/a-fatal-event-stands-down-both-bounds-and-may-leave-nothing-to-end-the-machine.md`, + `issues/hardware/most-t14-leases-land-one-dhcp-retry-late.md`, + `issues/diagnostics/a-swaps-redial-asks-again-with-no-event-to-wait-on.md`, + `issues/hardware/the-t14-redial-re-asks-mdns-after-every-refusal.md`, + `issues/diagnostics/a-first-dial-turned-away-before-a-line-is-waited-on-to-its-callers-bound.md` + and + `issues/diagnostics/a-netd-that-dies-while-serving-leaves-the-hosts-stream-silent.md`. - #539's issues `a-loader-change-reaches-a-machine-only-by-writing-its-stick`, `the-bench-reads-no-quiescent-log-volume`, `the-bench-runs-with-no-bound-on-its-own-boot`, @@ -324,13 +347,26 @@ Each stage lands on its own, in this order. as far as it is true of what lands. **Exit**: `bench_loop_drives_a_toyos_machine` passes in QEMU. On the T14, - with Ubuntu never started, three things hold: a kernel change boots; a slot - with a flipped byte, no signature or a lower security version is refused - and the other boots; and a slot that dies falls back on its own. + with Ubuntu never started: a whole run, its sessions and every boot that + goes by `--once` (`deadlinewedge`, `hardlockup`, `usbload` and + `foreignrecord` among them), gives the verdicts a per-boot run gave at the + commit this stage branches from; a kernel change boots; a slot with a + flipped byte, no signature or a lower security version is refused and the + other boots; a slot that dies falls back on its own, and one whose `sshd` + refuses the host's key is never marked good; a `--once` image that panics + returns the machine to its session with its record judged; and a loader + sent over ssh boots once, and one that brings no slot to good leaves the + old one booting. 7. **Ubuntu leaves the loop.** Delete `toyos-metal`'s `--via-ubuntu` path, - `--metal-via-ubuntu` and `bootloader/src/bootnext.rs`. After a reset the - firmware comes back to the loader because ToyOS's entry is first (stage 5). + `--metal-via-ubuntu`, `bootloader/src/bootnext.rs` and test-runner's + job-list mode. After a reset the firmware comes back to the loader because + ToyOS's entry is first (stage 5). + - `issues/build/a-hung-boots-log-partition-is-wiped-by-the-next-runs-flash.md`, + `issues/build/the-sudoers-rendering-has-no-host-side-judge.md`, + `issues/build/the-metal-loop-writes-the-readback-volume-into-a-directory-it-has-not-made.md` + and `issues/hardware/the-t14-stopped-answering-ssh-between-two-lan-boots.md` + close here, each only as far as it is true of what lands. **Exit**: no path in `src/metal*.rs` reaches Ubuntu. On the T14, a panic's reset reaches the loader with no `BootNext` set. @@ -358,11 +394,13 @@ Each stage lands on its own, in this order. `watchdog_quiet`'s loader half. `watchdog_armed` judges the kernel's read-back, and `loader_watchdog_arms` becomes the kernel's row. - **Exit**: `bootloader/src` holds no TCO access. On q35, `watchdog_armed` - passes on the kernel's lines (counting, `no_reboot=0`, `timeout=0`), and - `watchdog_resets` passes. A boot that hangs right after the arm, before - `mm::init`, is reset by the TCO; moving the arm back after `pci::enumerate` - makes that test fail. + **Exit**: `bootloader/src` holds no TCO access. On the T14, `watchdog_armed` + passes on the kernel's lines (counting, `no_reboot=0`, `timeout=0`). Once + the reading of + `issues/hardware/a-frozen-toyos-waits-for-a-hand-on-the-power-button.md` has + seen the TCO reset the T14, a boot there that hangs right after the arm, + before `mm::init`, is reset by the TCO; moving the arm back after + `pci::enumerate` makes that row fail. 9. **The crash report belongs to the kernel.** The loader hands the last boot's record to the kernel instead of decoding it. The kernel logs it, and @@ -373,8 +411,7 @@ Each stage lands on its own, in this order. `DROPPED_OPENS_WITH`'s count. - The report pass goes, taking with it `end_this_pass`, the chain constants, `loaderlog`'s chain lines, `armed_at`'s `GetTime`, everything - else in `bootloader/src/blackbox.rs` except the claim and the arm, and the - chained-pass item of `issues/hardware/the-t14-boots-toyos-unattended.md`. + else in `bootloader/src/blackbox.rs` except the claim and the arm. - `ENDS_AT_CHAIN` goes, so this stage rewrites the exit of `issues/diagnostics/a-wedged-reports-newest-records-were-cut-by-the-loaders-own-log-file.md`: a `deadlinewedge` rerun whose `/log` carries the `WEDGED` report with its diff --git a/issues/build/hard-lockup-bound-ms-is-read-by-nothing-but-its-own-assertion.md b/issues/build/hard-lockup-bound-ms-is-read-by-nothing-but-its-own-assertion.md index b3b0e5edb0..ed43976940 100644 --- a/issues/build/hard-lockup-bound-ms-is-read-by-nothing-but-its-own-assertion.md +++ b/issues/build/hard-lockup-bound-ms-is-read-by-nothing-but-its-own-assertion.md @@ -19,10 +19,12 @@ alone. ## Owner -`issues/hardware/the-t14-boots-toyos-unattended.md`, the track that holds the -hard-lockup detector and its bound. +`issues/hardware/a-frozen-toyos-waits-for-a-hand-on-the-power-button.md`, the +track that arms the hard-lockup detector on every boot. ## What would close it -The constant and its assertion deleted, with whatever `toyos-tco`'s version -owes for it; `hard_lockup_bound_ms`'s own test already holds the half. +That track's first step: a boot that names no `boot-deadline=` arms the +detector at `HARD_LOCKUP_BOUND_MS`, whose doc then says so in place of "the one +a T14 boot runs under". The constant is not deleted, since that step would +declare it again. diff --git a/issues/build/the-update-rig-the-guest-cut-deleted-is-still-cited.md b/issues/build/the-update-rig-the-guest-cut-deleted-is-still-cited.md new file mode 100644 index 0000000000..52745ef69b --- /dev/null +++ b/issues/build/the-update-rig-the-guest-cut-deleted-is-still-cited.md @@ -0,0 +1,37 @@ +--- +status: open +kind: tooling +opened: 2026-10-03 +--- + +# The update rig the guest cut deleted is still cited + +#660 (`06788146b`) deleted `tests/common/update.rs` with its `Rig`, +`tests/common/fwvars.rs` and `tests/updatecase/system.toml`. `git grep -n +'fwvars\|updatecase\|common/update\.rs\|Rig::\|vars::plant\|vars::live'` finds +nine lines in three files, each planning or describing a test on them: + +- `issues/boot-media/the-loader-does-only-what-must-precede-the-handover.md`, + six lines. Stage 3's exit takes `fwvars::live` as + `update_floor_is_the_images_own`'s oracle. Stage 5 says + `tests/updatecase/system.toml` grants `slots` and that two tests run + `updatecase`; one of its negative controls signs with `Rig::update` and + reads the floor with `fwvars::live`, which its oracles name too. +- `issues/boot-media/the-loader-never-sets-the-firmwares-memory-overwrite-request.md`, + two lines: its exit's `mor_is_set_where_defined` plants and reads the vars + store with `vars::plant` and `vars::live`. +- `issues/build/the-kernel-console-split-does-not-re-arm-across-a-guest-reset.md`, + one line: the reset in place it describes is `Rig::boot`'s. + +What each should name is the tier its test has now, and stage A of +`issues/build/the-guest-suite-runs-only-what-no-cheaper-tier-reaches.md` gives +it: `update_floor_is_the_images_own`, `update_refusals_boot_the_other_slot` +and `update_grant_refuses_a_stray_partition` are host tests there and guest +tests in the loader track, and the tree holds no `update_*` test today. + +**Owner**: that stage A, whose pull request lands those tests and with them +the oracle and the rig a sentence can name. + +**Exit**: the `git grep` above finds this file alone: each sentence names the +tier that holds its test and an oracle, a rig or a config the tree has, or has +gone with the test it described. diff --git a/issues/hardware/a-frozen-toyos-waits-for-a-hand-on-the-power-button.md b/issues/hardware/a-frozen-toyos-waits-for-a-hand-on-the-power-button.md new file mode 100644 index 0000000000..aeda14821e --- /dev/null +++ b/issues/hardware/a-frozen-toyos-waits-for-a-hand-on-the-power-button.md @@ -0,0 +1,96 @@ +--- +status: open +kind: track +opened: 2026-09-30 +--- + +# A frozen ToyOS waits for a hand on the power button + +No shipped image arms a hardware watchdog or the hard-lockup detector: the +kernel arms the chipset's TCO only on the `watchdog` parameter +(`arch::watchdog::init`), feeds it from the scheduler pass +(`kernel/src/sched/driver.rs:512`), and arms the detector only through +`boot-deadline=` (`kernel/src/deadline.rs:168-183`). + +**The owner's rulings:** + +- A userland `watchdogd` feeds the watchdog on every machine, with no new + syscall; the detector is armed on every boot; nothing ships for tests alone. +- **A panic's panel holds as it does today** and feeds the watchdog while it + holds, after a key too (`panic_console::hold_the_panel`). +- **`watchdogd` reaches the chipset's timer through the general port claim + #592 brings, its `isa` claim, as the ACPI server will**, and not through a + device class of its own: one way of doing it and not two (2026-10-03). That + ruling does not settle two things, and nothing here decides them: + - #592's grant covers ports below `0x100`, and the T14's TCO is at `0x400` + (`issues/hardware/an-armed-tco-has-never-reset-the-t14.md`). + - #592 refuses a claim on ports the kernel drives, and under this track the + kernel still arms the timer, feeds it from the panel and disarms it. + +**The track's own lines, which no ruling carries:** + +- **The scheduler pass feeds from the kernel's arm until the timer is first + claimed, and never after**, so a `watchdogd` that ends with no successor + holding the claim resets the machine. After that first claim the panel's + hold is the one feed the kernel keeps. A successor's claim that meets its + predecessor's release still in flight + (`issues/kernel/deferred-release-outlives-its-syscall.md`) is refused, so + `watchdogd` waits on that defect. +- A job holding `device` can mint the claim where no `watchdogd` holds it, + which + `issues/isolation/every-job-test-runner-starts-holds-its-whole-system-capability.md` + fences. +- `watchdogd` feeds from a thread under `rt`, until + `issues/kernel/cpu-time-is-a-band-and-not-a-reservation.md` gives it a + reservation. +- **Every guest runs with `-action watchdog=none`.** q35's TCO counts + `QEMU_CLOCK_VIRTUAL`, which a loaded host advances while it starves a guest, + and its second expiry does what `-action watchdog=` says (QEMU 11.1.1, + `hw/acpi/ich9_tco.c:61-70,244`). A TCO reset, and a feeding kernel that + `watchdog_fed` finds not reset, are judged on the T14. +- The T14's PCH TCO is its one reset that needs nothing of the kernel (no BMC, + no AMT, a battery no switch cuts), and no armed TCO has reset it yet + (`issues/hardware/an-armed-tco-has-never-reset-the-t14.md`). + +**Next: the detector on every boot**, at the half of `boot-deadline=` it takes +today or at `toyos_tco::HARD_LOCKUP_BOUND_MS` where a boot names none. The +constant then has a reader, which closes +`issues/build/hard-lockup-bound-ms-is-read-by-nothing-but-its-own-assertion.md`. + +**Exit**: a host test holds the bound of a boot that names no `boot-deadline=` +to `HARD_LOCKUP_BOUND_MS`, and putting the arm back behind `deadline::start` +fails to build. `Armed`'s `(0, _)` arm, which nothing reaches, is gone. + +**Then, once loader stage 7 has taken Ubuntu**, whose `iTCO_wdt_probe` clears +`SECOND_TO_STS` (Linux 6.12, `drivers/watchdog/iTCO_wdt.c:545-560`), out of the +loop: **the reading**, `watchdog_resets`' metal row in +`issues/build/the-guest-suite-runs-only-what-no-cheaper-tier-reaches.md`. The +owner's ruling (2026-10-03): one T14 boot may stop feeding the chipset's +watchdog on purpose, "only if its quick i dont want tests doing nothing for a +long time". So the row's wait is bounded by the watchdog's own bound and fails +loudly past it, with no long idle wait. One T14 boot is armed with `watchdog` +and `wedge-before-reset`, under a deadline past its job list and +`toyos_tco::BOUND_MS`, which `toyos_tco::STAGED_BOUND_MS` is not. The fed +control is `watchdog_fed`, once +`issues/hardware/watchdog-fed-ends-its-boot-before-the-bound-it-judges.md` +closes. + +**Exit**: the next armed kernel's `TCO2_STS` read, in `arch::watchdog`'s +`arm`, says the last boot ended in a TCO reset, and the page still reads +`ARMED` where the deadline would have sealed a record; the armed boot after +that says no TCO reset. That closes +`issues/hardware/tco2-sts-clearing-is-verified-on-qemu-only.md`. Otherwise the +T14 exit below waits on the TCO defect. + +**Last: `watchdogd` ships**, after loader stage 8 +(`issues/boot-media/the-loader-does-only-what-must-precede-the-handover.md`) +has made the kernel's arm the only one. `toyos_tco::PARAM` goes, and with it +`watchdog_quiet`, `loader_watchdog_arms`' control arm and the +`testcases-watchdog` boot: the kernel arms every boot whose chipset has a +`toyos_tco` row, q35 included, and one with none says it is unwatched. + +**Exit**, on the T14: a boot whose `watchdogd` ends with no successor, and one +wedged by `wedge-before-reset` with `watchdogd` feeding, each end in the TCO's +reset, and the boot after each says so; a kernel whose scheduler pass feeds on +once the claim has gone keeps the first up. The boot after a held panic says +no TCO reset. diff --git a/issues/hardware/most-t14-leases-land-one-dhcp-retry-late.md b/issues/hardware/most-t14-leases-land-one-dhcp-retry-late.md new file mode 100644 index 0000000000..67fd30a095 --- /dev/null +++ b/issues/hardware/most-t14-leases-land-one-dhcp-retry-late.md @@ -0,0 +1,32 @@ +--- +status: open +kind: defect +opened: 2026-09-29 +--- + +# Most T14 leases land one DHCP retry late + +Over the LAN boots in the readbacks of three T14 runs, the I219's link came up +2.72–2.82 s after its driver every time. Of the 16 leases, 5 landed 3.2–3.6 s +after netd came up and 11 at 13.1–13.4 s, so the first lease of a boot, and +with it `sshd`, arrived at 9.1–9.3 s of boot on 4 of the 13 boots and at +19.0–19.1 s on the other 9. + +The ten-second step is smoltcp 0.12's default `RetryConfig::discover_timeout`, +which netd keeps. netd restarts +discovery when the link comes up (`userland/netd/src/dhcp.rs:42-61`), and on +the late boots the lease came from the DISCOVER sent one timeout after that +one. What became of the first, whether it left the machine and whether an +OFFER came back, is unmeasured. RFC 2131 §4.1 puts the first retransmission +at 4 s, randomized by ±1 s. + +The network track owns it: netd leaves smoltcp in stage 5 of +`issues/design-debt/toyos-has-its-own-network-stack.md`, whose stage 3 is +`toyos-dhcp`. Stage 6 of +`issues/boot-media/the-loader-does-only-what-must-precede-the-handover.md` +waits on it. + +**Exit**: netd logs each DISCOVER it sends and each OFFER it receives, and a +T14 boot's log shows what became of the DISCOVER sent as the link came up; on +every boot of a T14 run an unanswered DISCOVER is sent again within RFC 2131 +§4.1's 4 ± 1 s. diff --git a/issues/hardware/the-t14-boots-toyos-unattended.md b/issues/hardware/the-t14-boots-toyos-unattended.md deleted file mode 100644 index 71d1d21ef7..0000000000 --- a/issues/hardware/the-t14-boots-toyos-unattended.md +++ /dev/null @@ -1,121 +0,0 @@ ---- -status: open -kind: track -opened: 2026-09-03 ---- - -# The T14 boots ToyOS unattended and reports through a log partition - -The T14 stopped being a GitHub Actions runner (owner ruling, 2026-09-03), so -what the tracker owes "on hardware" is owed to this loop: a driver on this Mac -flashes the stick left plugged into the machine, sets one boot, reboots, and -reads the verdict off a log partition. **There is no serial channel** — the -16550 loopback reads `0xFF` -(`issues/hardware/a-metal-session-runs-a-pre-flash-gate-first.md`, which is also -the loop's admission check: no image is flashed that has not passed it) — so the -log partition and the screen are the only two channels there are. - -What is left to build: - -- **The chipset watchdog is armed and does not count**, so nothing this tree - arms can end a wedged boot; a panicking kernel ends one by its own bound - instead, and a kernel that wedges without panicking after `clock::init` is now - ended by `kernel/src/deadline.rs` (below) — before it, still nothing. - `issues/hardware/an-armed-tco-has-never-reset-the-t14.md` carries the - registers and the datasheet. **Exit**: a T14 boot that resets itself on an - armed TCO. - - **The software answer was designed and is not built, and the first reason is - a number this loop does not have yet.** The design evaluated was a sentinel: - an AP started early becomes a CPU spinning on the TSC, the BSP publishes each - boot phase it reaches, and a phase missed within a bound seals a record into - the black box and writes the FADT reset register. Four findings, taken by - reading the code: - - 1. **The bound cannot be derived.** It has to be wider than the slowest - healthy phase on this machine and narrower than a wait for a hand, and - nothing in this tree has measured a T14 phase duration. A bound guessed - wrong resets a healthy laptop in a loop, which is worse than the present - state. This loop's boot-fact run is what publishes those numbers, so the - design is sequenced behind it rather than blocked on a ruling. - 2. **A sentinel is a CPU outside the roster, and the roster is modelled.** - `Roster::begin_attempt`/`commit` hands out dense ids that `boot_aps` fills - in attempt order, `smp_failed_ap_leaves_no_hole` is the gate that exists - because a hole in them is a defect, and `kernel-loom/tests/smp_bringup.rs` - is what decides whether a second committer is sound. A CPU taking an id - before `boot_aps` runs is a change to that protocol and needs the model - extended first. - 3. **It cannot start before the machine has a clock without a second - bring-up path.** `boot_aps`' SDM §8.4.4.1 delays and its 100 ms start - budget are spun on `clock::nanos_since_boot`, which answers zero until - `clock::init` — so an AP started before that never leaves the delay. - `clock::cpuid_tsc_hz` exists for the panic path's version of this problem - and would serve, at the cost of a second decider for how long an - INIT-SIPI wait is. - 4. **The reset it would take is not lock-free.** `acpi::reboot` opens with - `serial::flush_final`, and a `BackendGuard` masks interrupts for its whole - life — so a BSP wedged inside one holds the lock a sentinel would spin on, - on exactly the boots the sentinel exists for. A `reset_now` that writes the - decoded port and nothing else is small and separable from the rest. - - What no design of this shape can cover is the span before `apic::init`: no AP - can be started before the BSP's own LAPIC is enabled, and `acpi::init_reset` - — which decodes the register any of this would write — runs inside it. That - floor is inherent rather than an argument against the design. - - **The half of it that does not need an AP is built** (`kernel/src/deadline.rs`): - a bound armed off the parameter line as `boot-deadline=` and polled from - the timer interrupt entry, in both rings, on every CPU — two atomics and a - `rdtsc`, no lock, no allocation. On expiry it seals a `WEDGED` record naming - the bound, the uptime, the boot phase and **the tail of the log ring** into - the black box, and writes the reset register. That closes findings 1 and 4 - for everything after `clock::init`: the bound is derived rather than guessed - (`toyos_tco::WEDGE_BOUND_MS`, twice the runner's own, so a job the runner is - about to end is not a wedge), and `acpi::reset_now` is the lock-free write - finding 4 asked for — `reboot` now goes through it, so there is one writer of - that port. It ends a wedge whose every CPU has stopped taking scheduler - passes: measured on QEMU as `boot_deadline_ends_a_wedge`, 18 s with the arm - against a machine that never came back without it, twice at 66 s. - - **The second of the two seams that were left is closed, and not by a - sentinel.** A machine on which no CPU takes an interrupt is now - `kernel/src/hardlockup`: the local APIC's performance-counter LVT in NMI - delivery mode, armed on every CPU whose CPUID states an architectural PMU, - sampling that CPU's own interrupt count about once a second and sealing a - record — the CPU, its `rip` and `rsp` off the NMI frame, the lock it is - spinning on, a line for every CPU — from the stuck CPU's own NMI, then - `reset_now`. Its bound is half the deadline's and off the same parameter - (`toyos_tco::hard_lockup_bound_ms`), so the two compose rather than race. That - is the state run 22 sat in past 420 s with the deadline armed and unfired, and - it wanted a *sample* rather than a poll — no CPU outside the roster would have - helped, since a sentinel spinning on the TSC still has to be given a CPU that - runs, and the CPUs that were running were the deaf ones. - - **What is left for a sentinel is finding 3's span alone**: before - `clock::init` there is no TSC period to convert either bound with, and before - `apic::init` there is no LVT to arm — so a kernel that dies in early bring-up - still needs a hand, and only a CPU started that early reaches it. Finding 2 - (the roster is modelled and a CPU taking an id outside it is a protocol - change) stands unchanged and is what that would cost. -- **An AP loads its IDT before its control registers**, so a fault in that span - triple-faults the machine — - `issues/kernel/an-ap-loads-the-idt-before-its-control-registers.md`, whose - exit is one of the two orderings it names, judged by `smp_bringup` and the - SMP suite. -- **A loader pass overwrites the pass before it.** Every pass writes - `loader.log`, so on a chained boot the second pass's three lines replace the - first pass's record of the boot it is reporting on — measured on run 10, where - the surviving file was 228 bytes and the boot's own watchdog read-back was - gone. **Exit**: a chained run whose stick carries every pass, each named by - the pass that wrote it. -- **The measurements owed on hardware are this loop's jobs**, by record: - `issues/kernel/the-split-window-tlb-cost-is-unpriced.md`, - `issues/kernel/ap-control-registers-inherit-init.md`, - `issues/kernel/ap-tsc-trail-is-assumed-and-never-checked.md`, - `issues/audio/hda-ring-fix-unverified-on-metal.md`, and the - IOMMU track's three hardware-only answers — isolation scopes and reserved - regions, the 2× cost bar, and the compatibility-format question in - `issues/kernel/qemu-passes-compatibility-format-interrupts.md` - (`issues/kernel/the-iommu-refuses-nothing-yet.md` states the first two). - -`smp_failed_ap_leaves_no_hole` is deleted; `issues/build/smp-ap-hole-and-log-reserve-window-red-under-a-loaded-host.md` records the commit that restores it. diff --git a/issues/hardware/the-t14-reboots-through-ubuntu-for-every-test.md b/issues/hardware/the-t14-reboots-through-ubuntu-for-every-test.md new file mode 100644 index 0000000000..66dae43deb --- /dev/null +++ b/issues/hardware/the-t14-reboots-through-ubuntu-for-every-test.md @@ -0,0 +1,82 @@ +--- +status: open +kind: track +opened: 2026-09-29 +--- + +# The T14 reboots through Ubuntu for every test + +The owner's direction: the tests run one after another in a ToyOS that stays +booted, driven by the host over ssh, with no reboot between them and no Ubuntu. +The T14 is the integration tier: the shared boot runs only as `shared_metal` +and `c_corpus_metal`, and +`issues/build/the-guest-suite-runs-only-what-no-cheaper-tier-reaches.md` adds +metal rows. + +- **A metal row is a member** of the session its kernel build, parameter line + and config name, and costs no boot, unless it ends the machine (a panic, a + wedge, a reset) or judges a boot (the loader, an update, a file read back + after a reboot). Its registration says which. +- **One exec channel**, to `test-runner`'s stdin loop, since `sshd` aliases a + second channel's input (`issues/isolation/sshd-holds-one-channel-and-does-not-say-so.md`). + `tests/ssh-client-host`'s `pipe` relays it once it writes output as it + arrives (`tests/ssh-client-host/src/main.rs:251-263`). +- **A test's window** is what crosses between its markers: the job's output, + which never reaches `/log` (`userland/sshd/src/main.rs:234-239`), and the + kernel's records, which a runner the exec starts reads on its own + `toyos::log::LogTail` and writes before `===TEST_END===`. A record the ring + dropped reds the test. QEMU's serial runner writes no copy: its console + already carries the ring. +- **A leftover reds its test.** After each member the host sends `list`, and + the runner prints the processes alive and every root `/` has, whole + (`kernel/src/vfs.rs:83-86`). The host holds both to the session's first, + `/log` less `bootlog::split_listing`'s files. Claims join once + `issues/kernel/deferred-release-outlives-its-syscall.md` closes; until then a + member that mints one runs last in its session, as `endowment_denied` does. +- **A stuck test** is killed at the bound the host's `run` names, as + `--bound-ms=` names the job list's (`userland/test-runner/src/main.rs:78-83`): + 14.1 s, twice `mutual_kill`'s 7.041 s. It reds by name and the next member + runs. QEMU's harness names no bound and sends no `list`; its own ceiling ends + the guest. + +**Next: two sessions in place of six boots.** `shared`, `shared-2`, `ccorpus`, +`testcases`, `testcases-mkdir` and `testcases-readdir` share a config, a +parameter line and the shipping kernel. The session image is `tests/testcases` +with `tests/lantalkcase`'s netd, sshd and streaming `logd`, under the same +120 s `boot-deadline=`. A session holds 74.8 s of members, priced at +`toyos_tco::RUST_MEMBER_MS` a Rust member, `toyos_tco::C_MEMBER_MS` a C case +and a list at twice its slowest on the T14: the bound less a tenth, less a +lease as late as 19.1 s +(`issues/hardware/most-t14-leases-land-one-dhcp-retry-late.md`), less one +per-job bound. Every judge reads the stick as today but `syscall_cost`, which +reads the window. `loader_watchdog_arms`' control arm rides a session, +`mkdir_cap` and `readdir_bound` remove what they made, and `audio_idle_suspend` +waits for soundd's `inspect` to read `suspended` +(`userland/soundd/src/inspect.rs:21-33`), not for a boot no client has reached. + +**Exit**, on the T14: every registration those boots carried gives the +per-boot verdict at the same head, each window's judgement agrees, and the +sessions in reverse order, claims last, give the same verdicts. A fault, by the +kernel's record in its window, and a job spinning past its bound, each staged +in one member, red it alone and the next runs. A host test over a recorded +window reds only the member that left a child alive, a file in `/home` or a +`NEVER_CLEAN` line. + +**Then: Ubuntu leaves the loop** in stages 6 and 7 of +`issues/boot-media/the-loader-does-only-what-must-precede-the-handover.md`: +a session image comes by `update`, a row that boots on its own by +`update --once`, and each death's record over `sftp`. + +**Last: sessions outlive the boot deadline**, once +`issues/hardware/a-frozen-toyos-waits-for-a-hand-on-the-power-button.md` has +armed the hard-lockup detector on every boot and shipped `watchdogd`. An image +names no `boot-deadline=`, `metal::judge_arms` takes `watchdogd`'s row as its +bound, and `pipe` bounds each window rather than its whole run +(`tests/ssh-client-host/src/main.rs:64`). A ToyOS the host cannot reach keeps +its watchdog fed, so the run says it waits for a hand once nothing has answered +within the watchdog's bound and a POST allowance. A swap then leaves a boot's +bound (`issues/hardware/a-swap-on-the-t14-lives-inside-a-metal-boots-bound.md`). + +**Exit**: the host's plan names every reset in a run by what forces it, and a +host test whose transport goes quiet under a session finds the run stopped, +saying it waits for a hand. diff --git a/issues/hardware/the-t14s-firmware-interrupts-every-cpu-every-2-2-s-under-toyos.md b/issues/hardware/the-t14s-firmware-interrupts-every-cpu-every-2-2-s-under-toyos.md index 04874b9c58..10c7f60878 100644 --- a/issues/hardware/the-t14s-firmware-interrupts-every-cpu-every-2-2-s-under-toyos.md +++ b/issues/hardware/the-t14s-firmware-interrupts-every-cpu-every-2-2-s-under-toyos.md @@ -71,11 +71,10 @@ The `mask-windows` kernel prints such a window as its CPU's own: cpu4's (`issues/kernel/a-process-lengthens-an-interrupts-off-walk-by-the-threads-it-parks-on-one-ring.md`), are this by reading, those kernels not reading the count. -**Owner**: the fix is stage 1 of -`issues/kernel/toyos-runs-the-machine-in-acpi-mode-and-interprets-its-aml.md`. -The T14 loop (`issues/hardware/the-t14-boots-toyos-unattended.md`) owns only -the reading: its jobs are the measurements owed on hardware, and it builds the -row. +**Owner**: stage 1 of +`issues/kernel/toyos-runs-the-machine-in-acpi-mode-and-interprets-its-aml.md`, +for the fix and for the reading: its exit holds the count flat on the T14 over +this exit's interval, so the stage builds the row. **Exit**: a T14 row reads `MSR_SMI_COUNT` on every CPU after init is spawned and after the boot's last write to `SMI_CMD`, whoever makes it, and again at diff --git a/issues/hardware/watchdog-fed-ends-its-boot-before-the-bound-it-judges.md b/issues/hardware/watchdog-fed-ends-its-boot-before-the-bound-it-judges.md new file mode 100644 index 0000000000..3d8acda7a5 --- /dev/null +++ b/issues/hardware/watchdog-fed-ends-its-boot-before-the-bound-it-judges.md @@ -0,0 +1,20 @@ +--- +status: open +kind: tooling +opened: 2026-09-30 +--- + +# `watchdog_fed` ends its boot before the bound it judges + +`watchdog_fed` holds that an armed T14 boot runs its list to its own stop, +which a chipset reset anywhere in it would have cut short. Its boot, +`testcases-watchdog`, carries no job of its own and neither does +`loader_watchdog_arms`' arm on it, so its list is `reboot` alone: on two T14 +runs the runner spawned `reboot` 1.196 s and 1.489 s into the kernel's clock, +and the shutdown disarms the timer (`arch::watchdog::disarm`), inside one +`toyos_tco::BOUND_MS` of 9,600 ms. A kernel that never fed the timer passes it. + +**Exit**: `watchdog_fed`'s boot holds past `toyos_tco::BOUND_MS` before its +`reboot`, and the same boot on a kernel whose scheduler pass never feeds reds +it once `issues/hardware/a-frozen-toyos-waits-for-a-hand-on-the-power-button.md` +has seen the TCO reset the T14. diff --git a/issues/isolation/every-job-test-runner-starts-holds-its-whole-system-capability.md b/issues/isolation/every-job-test-runner-starts-holds-its-whole-system-capability.md new file mode 100644 index 0000000000..4fdc5f270a --- /dev/null +++ b/issues/isolation/every-job-test-runner-starts-holds-its-whole-system-capability.md @@ -0,0 +1,27 @@ +--- +status: open +kind: defect +opened: 2026-09-29 +--- + +# Every job test-runner starts holds test-runner's whole system capability + +`run_one` endows each job it spawns a `SysCap::duplicate` of test-runner's own +capability (`userland/test-runner/src/main.rs:248-250`), and a duplicate +carries every right the original does (`toyos/src/syscap.rs:63-70`). The job +also inherits test-runner's whole namespace +(`userland/test-runner/src/main.rs:62-68`). On `tests/testcases` the row grants +`device`, `dup`, `logread`, `power` and `roster` +(`tests/testcases/system.toml:41`), so every test binary may mint a device +claim, read every kernel record, reset or power off the machine, list every +process, and hand all of it to a child of its own. A right or a name added to +that row reaches every job the same way. + +The binaries have no `[programs]` rows, so nothing declares what any of them +needs: `test_rs_audio_idle_suspend` reads `roster` off the duplicate +(`tests/toyos-rust-tests/src/bin/audio_idle_suspend.rs:7-11`), and +`endowment_denied` narrows `power` away from it. + +**Exit**: a job holds only the rights and names its test is declared to need, +narrowed by test-runner (`SysCap::narrowed`, `toyos/src/syscap.rs:133-139`), and +a job that asks for anything else is refused.