diff --git a/issues/a-firmware-call-does-what-its-handler-chooses-and-the-kernel-bounds-only-the-call.md b/issues/a-firmware-call-does-what-its-handler-chooses-and-the-kernel-bounds-only-the-call.md new file mode 100644 index 00000000000..dd80ed8abab --- /dev/null +++ b/issues/a-firmware-call-does-what-its-handler-chooses-and-the-kernel-bounds-only-the-call.md @@ -0,0 +1,130 @@ +--- +status: open +kind: defect +opened: 2026-10-08 +--- + +# A firmware call does what its handler chooses, and the kernel bounds only the call + +A byte the `acpi` claim's holder stores to the FADT's `SMI_CMD` is a call into +the firmware, which the kernel makes for it +(`call`, `kernel/src/arch/x86_64/acpi_mode.rs`; decided by +`toyos_userbound::firmware::port`). The byte selects a handler that runs in +system management mode, above the kernel, and reads whatever the holder wrote +to firmware's memory before the call, which the mediated access passes both +ways (`issues/the-acpi-claims-holder-reaches-every-port-and-firmware-range-the-kernel-did-not-declare.md`). +So a bug in `/system/bin/acpiserver`, or AML it runs, reaches whatever the +machine's firmware does on any byte with any argument block. + +What the kernel bounds: + +- **Who**: a process that holds the claim, bound to it. +- **When**: never once the stop has begun. +- **Where**: the boot processor, read beside the `out` (ACPI 6.5 Table 5.9, + as `kernel/src/arch/x86_64/smi_cmd.rs` quotes it). +- **Which byte**: none the FADT gives a meaning, `ACPI_ENABLE`, + `ACPI_DISABLE`, `S4BIOS_REQ`, `PSTATE_CNT` and `CST_CNT`, each the kernel's + own command or nobody's. ACPI 6.5 Table 5.9 names no other value for the + port. A zero names none in four of them; `S4BIOS_REQ` names the byte it + holds, zero too, where the FACS's `S4BIOS_F` is set, and none where it is + clear (`SmiCmd::named`, `toyos-acpi/src/fadt.rs`, which quotes each). A + write wider than a byte that reaches the port is refused whole. +- **How often a byte is written to the command port**: eight in any second + (`firmware::CALLS`, `toyos-userbound/src/firmware.rs`), counted for every + holder there has been. The ninth is refused to the caller by name and + written nowhere. That is a bound on writes to `SMI_CMD` and on nothing + else that raises a firmware interrupt: the chipset's own SMI enable + register sits at a port no table names and nothing declared, so the holder + writes it as it writes any port + (`issues/the-acpi-claims-holder-reaches-every-port-and-firmware-range-the-kernel-did-not-declare.md`). + Eight a second is no bound on the firmware interrupts a holder can ask + for. + +What it does not: + +- **What the handler does.** Nothing reads the argument block, and nothing + could hold it to a meaning: the handlers are the machine's maker's. +- **How long a call holds the machine.** The write returns when the handler + does, with the boot processor's interrupts closed in `smi_cmd::answer` and + the asker, where it is another CPU, spinning under the mediation's lock + with preemption off; on the T14 a software interrupt to the firmware stops + every CPU. The kernel counts and times each + (`toyos_abi::counters::Counter::FirmwareCalls` and `FirmwareNanos`, and a + line for the first call of each byte) and can end none: system management + mode takes no interrupt of this kernel's, the NMI among them, which is + held pending until the handler returns (Intel SDM Vol. 3C, "NMI Handling + While in SMM"). + +What a handler that outlasts each of the kernel's bounds meets +(`kernel/src/arch/x86_64/smi_cmd.rs`, decided by `kernel::bootwrite`). The +decision is host-tested; the rest is by reading and staged nowhere, since no +guest's call runs a handler: + +- **The kick's bound has no handler in it.** The asker gives the boot + processor `DEAF_CPU`, 5 s, to say it has the write, which it says before + the `out`. A handler's time is never read as a boot processor that takes no + interrupt. +- **A handler that holds the boot processor 5 s is a panic that says so**, + whichever CPU asked: `smi_cmd: the firmware has held the boot processor in + its handler for the 0x.. written to SMI_CMD and has not returned` from an + asker on another CPU that ran meanwhile, at 5 s from the take; and + `smi_cmd: the firmware held the boot processor ..ns in its handler` from + the boot processor itself once the write retires, which is the one that + speaks where the asker was the boot processor. Where the interrupt stopped + an asker on another CPU too, either may speak: the asker resumes with the + clock past the span and the round not yet published, and may panic first + with "has not returned" of a handler that has. Either message names the + firmware and the byte. So a handler that returns after 5 s still ends the + machine: the kernel gives up no CPU for that long, as it gives up none to + a TLB shootdown. +- **The hard-lockup bound, on an image that names a boot deadline** + (`kernel/src/hardlockup/mod.rs`; half the deadline). Its sample is an NMI, + delivered to the boot processor when the handler returns and before the + kernel's own judgement above. It finds `IF` clear in `smi_cmd::answer` and + no interrupt taken, and where that has lasted its bound it seals a `WEDGED` + record that names cpu0 and that `pc` and resets the machine, unless an + asker's panic stood it down first. Whether the counter that raises the + sample counts in system management mode on the T14 is unread. +- **The boot deadline, on such an image** (`kernel/src/deadline.rs`), is + polled from a timer entry, and none is taken while every CPU is stopped: + the first tick after a handler that outlasted it seals `the boot deadline + expired`, where nothing above took the seal. +- **A handler that never returns.** Where its interrupt stopped every CPU, + no instruction of this kernel runs again and it says nothing: the machine + is the firmware's, and a hand on the power button ends it. Where it + stopped the boot processor alone, an asker on another CPU panics at 5 s + with the words above. Asked on the boot processor itself, nothing waits on + the write, and the next wait on cpu0 speaks: a TLB shootdown's `DEAF_CPU` + panic, or the boot deadline on an image that names one. A machine on which + neither comes is not ended by this kernel. +- **A machine whose FADT names no `SMI_CMD`.** Nothing is declared there, so + its chipset's command port is a port like any other and the holder writes + it with no bound at all. + +What is measured. Eight a second is this kernel's own number and no +measurement: the model of the T14's AML that was read makes two calls in one +evaluation at the most, and retries a call its handler has not answered once +a millisecond, ten thousand times; eight holds that storm to eight calls a +second and refuses the rest, and whether it refuses a call a healthy machine +needs is unread. No call but the kernel's own enable and disable has been +made on the T14: the server passes no write its AML asks for to the kernel +yet (`userland/acpiserver/src/host.rs`), so nothing there asks for one. On QEMU's q35 the chipset model keeps the byte and its +`SMI_EN` reads 0, so no guest's call interrupts a firmware, and what a guest +reads of one is that it was written, where, and how often. + +The server holds no bound of its own on the calls one evaluation makes: that +is the slice's that evaluates the methods which call. + +Owned by `issues/toyos-runs-the-machine-in-acpi-mode-and-interprets-its-aml.md`, +beside `issues/the-acpi-servers-holder-drives-the-embedded-controller-unfiltered.md`. + +**Exit**: a call is made only for a byte the machine's loaded tables store +to the port, checked by something other than the holder, or the owner rules +the bound above is the one ToyOS keeps; and the rate is held against the +calls a T14 row reads the machine's own AML making, with the time each held +the boot processor; and the span is ruled. The slice that passes the first +write its AML asks for to the kernel reads on the T14 the time each call its +AML makes holds cpu0, and the owner rules, against those readings, whether a +handler that returns after `DEAF_CPU` ends the machine. Where the interrupt +stops every CPU the kernel such a handler returns to is whole, so the panic +there is this kernel's choice. That slice does not land without the ruling. diff --git a/issues/the-acpi-claims-holder-reaches-every-port-and-firmware-range-the-kernel-did-not-declare.md b/issues/the-acpi-claims-holder-reaches-every-port-and-firmware-range-the-kernel-did-not-declare.md index 57047f0f1ac..9d8443d2ae1 100644 --- a/issues/the-acpi-claims-holder-reaches-every-port-and-firmware-range-the-kernel-did-not-declare.md +++ b/issues/the-acpi-claims-holder-reaches-every-port-and-firmware-range-the-kernel-did-not-declare.md @@ -43,7 +43,9 @@ What the kernel's declarations do not follow: where it counts the SMI its own `SMI_CMD` write raises. - **A machine whose FADT names no `SMI_CMD`.** Nothing is declared there, so the chipset's software-SMI port is a port like any other and the holder - writes it; the write is refused by name only where the FADT names the port. + writes it; a byte for it is the kernel's to write, on the boot processor + and under its bounds, only where the FADT names the port + (`issues/a-firmware-call-does-what-its-handler-chooses-and-the-kernel-bounds-only-the-call.md`). So a bug in `/system/bin/acpiserver`, or AML it runs, can reach those; the kernel bounds where, and not what. The owner's ruling on the server reading diff --git a/issues/the-t14s-firmware-interrupts-every-cpu-every-2-2-s-under-toyos.md b/issues/the-t14s-firmware-interrupts-every-cpu-every-2-2-s-under-toyos.md index 8e019a7e61d..dbf3c30c928 100644 --- a/issues/the-t14s-firmware-interrupts-every-cpu-every-2-2-s-under-toyos.md +++ b/issues/the-t14s-firmware-interrupts-every-cpu-every-2-2-s-under-toyos.md @@ -83,14 +83,33 @@ are this by reading, those kernels not reading the count. **Owner**: stage 1 of `issues/toyos-runs-the-machine-in-acpi-mode-and-interprets-its-aml.md`, -for the fix and for the reading: its exit holds the count flat on the T14 over -this exit's interval, so the stage builds the row, and it reads the count -through the general counters ("General counters", owner, 2026-10-03, +for the fix and for the reading: its exit holds the count on the T14 to what +ToyOS asked for over this exit's interval, so the stage builds the row, and +it reads the count through the general counters ("General counters", owner, 2026-10-03, `issues/toyos-explains-itself.md`). -**Exit**: a T14 row reads `MSR_SMI_COUNT` on every CPU after init is spawned -and after the boot's last write to `SMI_CMD`, whoever makes it, and again at -the stop's report at least 4.444 s later, two of the longest period read, and -on every CPU the two agree. The interval opens after that write because the -ACPI enable is one, a write of `ACPI_ENABLE` to `SMI_CMD`, and raises one -firmware interrupt where `APMC_EN` is set. +**Exit**: every SMI in the interval is one ToyOS asked for. A T14 row reads +`MSR_SMI_COUNT` on every CPU and the boot processor's `firmware_calls`, the +kernel's count of its writes to `SMI_CMD`, after init is spawned and after +the ACPI enable, and again at the stop's report at least 4.444 s later, two +of the longest period read; and on every CPU the SMI count's delta equals the +boot processor's `firmware_calls` delta over that interval. A write to +`SMI_CMD` raises one firmware interrupt where `APMC_EN` is set, so the +interval may hold such writes, and each is counted on both sides. + +Two things the equality rests on, which the row answers before it is read +over an interval that holds a call: + +- **A round of the counters is not one instant.** A call that lands between + one CPU's sample and the boot processor's leaves that CPU's SMI delta one + short of `firmware_calls` with no unasked interrupt in it. The row takes + its two reads where no call is in flight, or bounds the difference by the + calls in flight. +- **"One command moves every CPU's count by exactly one" has been read for + the enable alone**: every CPU read the count the writer read after it. The + disable's line reads the boot processor's count only, and no other byte has + been written on the T14. + +The measure holds unasked interrupts to none and says nothing of how many +ToyOS asks for: that is the rate's +(`issues/a-firmware-call-does-what-its-handler-chooses-and-the-kernel-bounds-only-the-call.md`). diff --git a/issues/toyos-runs-the-machine-in-acpi-mode-and-interprets-its-aml.md b/issues/toyos-runs-the-machine-in-acpi-mode-and-interprets-its-aml.md index b768251212e..4e10af18689 100644 --- a/issues/toyos-runs-the-machine-in-acpi-mode-and-interprets-its-aml.md +++ b/issues/toyos-runs-the-machine-in-acpi-mode-and-interprets-its-aml.md @@ -105,8 +105,9 @@ fixed event that needs no AML, and takes the EC's events. **Exit**: QEMU's cleanly, through ToyOS's own power-off path (`SYS_SHUTDOWN`), with the press and that stop in the boot's log. On the T14, `counters` reads `MSR_SMI_COUNT`, through the general counters and not by a check of its own, -flat on every CPU over the interval the firmware issue's exit defines, and the -machine still in ACPI mode; `acpi_server_events` reads each EC query number +and holds the count to what ToyOS asked for over the interval the firmware +issue's exit defines, every CPU's delta equal to the boot processor's +`firmware_calls` delta, and the machine still in ACPI mode; `acpi_server_events` reads each EC query number once with its count; and `acpi_server_death` kills the server and reads `SCI_EN` clear in `PM1_CNT` afterwards, the kernel having written `ACPI_DISABLE` to `SMI_CMD`. @@ -322,6 +323,66 @@ mediated access, leaves open: no second reading; it goes when the owner rules one in, and a guest test then supplies a sleep type that is not the machine's and reads it refused. +What the firmware call, which the kernel makes for the server where its AML +stores a byte to `SMI_CMD`, leaves open. The server's AML makes none yet: its +host denies every write AML asks for and passes none to the kernel +(`userland/acpiserver/src/host.rs`). + +- **What a call does there is the firmware's**, and the kernel bounds who, + when, where, which byte and how often: + `issues/a-firmware-call-does-what-its-handler-chooses-and-the-kernel-bounds-only-the-call.md`. +- **Eight calls in any second is no measurement** + (`toyos_userbound::firmware::CALLS`), and it bounds a count of calls, not + the time they hold the machine: the T14's enable held the boot processor + 2.0 to 2.1 ms on three boots and stopped every CPU, so eight a second is + about 16 ms of the whole machine in every second if a call costs what the + enable does, and no call's cost has been read. Owner: this stage. + **Exit**: the slice that evaluates the methods which call brings the T14's + count of them and the time each held the boot processor, from the + `counters` row's `firmware_calls` and `firmware_nanos` and the kernel's + line for the first call of each byte; it reads what eight a second does to + the audio and latency rows on the T14, a timing verdict coming only from + there; and the owner rules the number against them. +- **The `counters` row holds every CPU's SMI count to the commands the + kernel wrote to `SMI_CMD`** between its first read and its last, and to + nothing else, where it held the count flat; with no call made the two are + one judgement. It rests on one reading, that the enable moved every CPU's + count by one: a call that moves a CPU's count by none or by two reds the + row, and is a reading for the owner and no flake. That equality is the + exit of + `issues/the-t14s-firmware-interrupts-every-cpu-every-2-2-s-under-toyos.md`, + which names the two things it rests on. Before the row is read over a span + that holds a call, the slice that evaluates the methods which call takes + the row's two reads where no call is in flight, or bounds each CPU's + difference by the calls in flight; and reads every CPU's SMI count either + side of one real call, which only the enable has been. +- **No guest's call interrupts a firmware, and no guest writes the enable.** + q35's chipset keeps the byte, with `SMI_EN` reading 0, and its firmware + hands the machine over in ACPI mode. So `acpi_mediated_access` reads that + a call was written, on which CPU, how often and counted, and that the + kernel's own commands were not; the time a handler takes, and the SMI + count either side of one, are read on the T14 alone, and there only for + the enable and the disable. +- **`acpi_mediated_access` needs one of three calls to be asked off the boot + processor**, each from a thread that read itself there first. The tree + has no affinity (`issues/no-test-can-hold-a-thread-on-a-named-cpu.md`), so + a thread preempted between that read and the kernel's lock may be taken by + the boot processor, the window + `issues/no-t14-row-arranges-an-acpi-disable-asked-off-the-boot-processor.md` + describes. A boot where all three fall in it reds as `no firmware call was + asked from another CPU`, over three kernel lines reading `asked from cpu0` + beside the probe's own line naming three non-zero x2APIC ids: that is the + window and no defect of the write, and it is answered by the pin, never by + a second run. None has been read. +- **`acpi_mediated_access`'s storm is refused only if nine calls fit in one + second of the guest's clock.** The probe asks one after another and needs + the ninth refused `CommandRate`. A guest whose host gives it less than + nine calls' worth of time in a second of its own clock is refused none, + and after ten thousand calls reds as `firmware calls in a row were made, + and none refused`: a dependence on rate in a guest test, the host's load + and no defect of the bound, which `toyos-userbound`'s host test holds on a + clock of its own. None has been read. + The press issue's measurement of 2026-10-07 found the three presses it lost changing nothing its scout read, with the button's event enabled and no SMI taken, and its hypothesis is that the controller wants the firmware's diff --git a/kernel/pure/bootwrite.rs b/kernel/pure/bootwrite.rs new file mode 100644 index 00000000000..431d2cdf9ae --- /dev/null +++ b/kernel/pure/bootwrite.rs @@ -0,0 +1,107 @@ +//! What a CPU that asked the boot processor for a write to `SMI_CMD` decides +//! on one turn of its wait, from the clock and what the boot processor has +//! published. +//! +//! The wait has two subjects and each is held to the span on its own: +//! +//! - **The kick being taken.** Until the boot processor publishes that it has +//! the write, the span runs from the ask, and its end is a boot processor +//! that takes no interrupt ([`Turn::Deaf`]). +//! - **The firmware's handler.** The boot processor publishes that it has the +//! write before it makes it, with the time; from there the span runs from +//! that time, and its end is a firmware that has held the boot processor +//! the whole of it ([`Turn::Outlasted`]). The boot processor judges the +//! same span itself once the write has retired ([`outlasted`]), so a +//! handler that long ends the machine whichever CPU asked for it. +//! +//! Neither span has the other inside it: a handler's time is never the +//! boot processor's deafness. + +#![forbid(unsafe_code)] + +/// One turn of the asker's wait. +#[derive(Clone, Copy, PartialEq, Eq, Debug)] +pub enum Turn { + Wait, + /// The write has retired, and so has the firmware's handler. + Retired, + /// The boot processor has not taken the kick in the span. + Deaf, + /// The firmware has held the boot processor the span, from the time it + /// took the write. + Outlasted, +} + +/// Whether a write that has held the boot processor `held_ns` has outlasted +/// `span_ns`. +pub const fn outlasted(span_ns: u64, held_ns: u64) -> bool { + held_ns >= span_ns +} + +/// The turn at `now_ns` of a wait asked at `asked_ns`: `taken_ns` is when the +/// boot processor took the write, where it has, and `retired` whether the +/// write has retired. A write that has retired is never judged by the clock. +pub const fn turn(span_ns: u64, asked_ns: u64, taken_ns: Option, retired: bool, now_ns: u64) -> Turn { + if retired { + return Turn::Retired; + } + let (since, end) = match taken_ns { + None => (asked_ns, Turn::Deaf), + Some(taken_ns) => (taken_ns, Turn::Outlasted), + }; + if outlasted(span_ns, now_ns.saturating_sub(since)) { end } else { Turn::Wait } +} + +#[cfg(test)] +mod tests { + use super::*; + + const SPAN: u64 = 5_000_000_000; + const MS: u64 = 1_000_000; + + /// A kick taken late and a handler that then runs nearly the span: the + /// two together are far past it, and neither is. + #[test] + fn a_handlers_time_is_not_the_kicks_and_the_kicks_is_not_the_handlers() { + let (asked, taken) = (7 * MS, 7 * MS + SPAN - 1); + assert_eq!(turn(SPAN, asked, None, false, taken), Turn::Wait); + for now in [taken, taken + MS, asked + SPAN, asked + SPAN + MS, taken + SPAN - 1] { + assert_eq!(turn(SPAN, asked, Some(taken), false, now), Turn::Wait, "{now}"); + } + assert_eq!(turn(SPAN, asked, Some(taken), false, taken + SPAN), Turn::Outlasted); + } + + #[test] + fn a_kick_not_taken_in_the_span_is_a_deaf_boot_processor() { + let asked = 3 * MS; + assert_eq!(turn(SPAN, asked, None, false, asked), Turn::Wait); + assert_eq!(turn(SPAN, asked, None, false, asked + SPAN - 1), Turn::Wait); + assert_eq!(turn(SPAN, asked, None, false, asked + SPAN), Turn::Deaf); + assert_eq!(turn(SPAN, asked, None, false, u64::MAX), Turn::Deaf); + } + + #[test] + fn a_handler_that_holds_the_span_has_outlasted_it_on_either_cpu() { + let (asked, taken) = (3 * MS, 4 * MS); + assert_eq!(turn(SPAN, asked, Some(taken), false, taken + SPAN - 1), Turn::Wait); + assert_eq!(turn(SPAN, asked, Some(taken), false, taken + SPAN), Turn::Outlasted); + // The boot processor's own judgement, of the same span. + assert!(!outlasted(SPAN, SPAN - 1)); + assert!(outlasted(SPAN, SPAN)); + } + + /// The asker reads the clock before it reads whether the write retired, + /// so a write found retired is one, whatever the clock says. + #[test] + fn a_write_that_has_retired_is_never_judged_by_the_clock() { + for taken in [None, Some(4 * MS)] { + assert_eq!(turn(SPAN, 3 * MS, taken, true, u64::MAX), Turn::Retired); + } + } + + /// A clock read before the boot processor's own reads no time before it. + #[test] + fn a_take_the_clock_has_not_reached_is_waited_for() { + assert_eq!(turn(SPAN, 3 * MS, Some(5 * MS), false, 4 * MS), Turn::Wait); + } +} diff --git a/kernel/pure/lib.rs b/kernel/pure/lib.rs index f3a288bec08..e7004656b52 100644 --- a/kernel/pure/lib.rs +++ b/kernel/pure/lib.rs @@ -1,7 +1,9 @@ //! What the kernel decides without touching the machine: the scheduler core //! ([`sched`]), the process and thread lifecycle ([`proclife`]), which PCID an //! address space is handed ([`pcid`]), what type the range registers give a -//! range ([`mtrr`]) and what a pipe's ends are told of each other ([`pipe`]). +//! range ([`mtrr`]), what a pipe's ends are told of each other ([`pipe`]) and +//! what a CPU waiting on the boot processor's write to `SMI_CMD` decides +//! ([`bootwrite`]). //! The kernel binary links it; the host runs //! its tests, because none of it reads a register, a clock or a kernel lock. @@ -12,6 +14,7 @@ extern crate alloc; #[cfg(test)] extern crate std; +pub mod bootwrite; pub mod mtrr; pub mod pcid; pub mod pipe; diff --git a/kernel/src/arch/aarch64/counters.rs b/kernel/src/arch/aarch64/counters.rs index 38e9eeb8333..5a123549238 100644 --- a/kernel/src/arch/aarch64/counters.rs +++ b/kernel/src/arch/aarch64/counters.rs @@ -7,5 +7,5 @@ use crate::counters::Hardware; pub fn bring_up() {} pub fn read() -> Hardware { - Hardware { smi: None, aperf: None, mperf: None, envelope: None } + Hardware { smi: None, aperf: None, mperf: None, envelope: None, firmware: None } } diff --git a/kernel/src/arch/x86_64/acpi_mode.rs b/kernel/src/arch/x86_64/acpi_mode.rs index 3381ecff802..893c82976f2 100644 --- a/kernel/src/arch/x86_64/acpi_mode.rs +++ b/kernel/src/arch/x86_64/acpi_mode.rs @@ -41,6 +41,19 @@ //! whose CPUs holds registers that are on and not those (`mtrr::compare`): //! the read is made on whichever CPU the call runs on. //! +//! **A byte the holder's AML stores to `SMI_CMD` is a call into the firmware, +//! made here and by nobody else** ([`call`]): the policy answers which byte, +//! `smi_cmd::write` makes it on the boot processor and returns once the +//! firmware's handler has, and the holder's thread is the one that waits and +//! is charged. What the handler does with the byte, and with whatever the +//! holder wrote to firmware's memory before it, nothing here bounds: system +//! management mode outranks this kernel. What is bounded is who, a holder of +//! the claim; when, never once the stop has begun; where, the boot processor; +//! which byte, none the FADT gives a meaning; and how often, +//! `firmware::CALLS` in any second, because the AML that calls retries a +//! call its handler has not answered and each call stops every CPU. A call +//! past that is refused to the caller by name and written nowhere. +//! //! **The firmware's Global Lock is taken and given back here** (ACPI 6.5 //! §5.2.10.1), by compare-and-exchange on the FACS's lock word; a release the //! firmware asked for meanwhile is signalled by `GBL_RLS` in `PM1a_CNT` @@ -70,7 +83,9 @@ use core::sync::atomic::AtomicU32; use toyos_abi::acpi::{Access, AcpiInfo, Block, Refused, Space, Width, FIXED_POWER_BUTTON}; use toyos_abi::syscall::SyscallError; use toyos_acpi::{Ec, FixedHardware, LegacyMode, PowerButton}; -use toyos_userbound::firmware::{self, Ecam, Function as PciFunction, LockWordAt, Memory, MemoryAt, MemoryVerdict, PortAt}; +use toyos_userbound::firmware::{ + self, CallRate, Ecam, FirmwareCall, Function as PciFunction, LockWordAt, Memory, MemoryAt, MemoryVerdict, PortAt, PortVerdict, +}; use toyos_userbound::Ports; use super::pio::{self, Declared, TakenBack}; @@ -101,8 +116,6 @@ const HANDBACK: Duration = Duration::from_millis(100); struct Hardware { fixed: FixedHardware, control: Declared, - /// `SMI_CMD`, declared to `smi_cmd`, and what is written to it. - legacy: Option, /// Or why it is none a holder can be handed. ec: Result, rsdp: u64, @@ -146,6 +159,16 @@ struct Holder { locked: bool, /// The holder supplied the power-off's sleep type: it supplies no second. supplied: bool, + /// The calls into the firmware made for every holder there has been: a + /// holder that dies and is started again begins no new second. + calls: Calls, +} + +/// The firmware calls made for the claim's holders, and which have been said. +struct Calls { + rate: CallRate, + /// A bit a byte, set once a call of it has been said. + said: [u64; 4], } /// Held across everything this kernel does for the claim's holder, from the @@ -154,7 +177,11 @@ struct Holder { /// (`object::Held`), and `pcidev`'s machine record and then `paging`'s record /// of windows under it; nothing holding one of those two takes this or a /// claim's. -static HOLDER: Lock = Lock::new(Holder { locked: false, supplied: false }); +static HOLDER: Lock = Lock::new(Holder { + locked: false, + supplied: false, + calls: Calls { rate: CallRate::new(), said: [0; 4] }, +}); /// The right to act for the claim's holder, held across the act; none once /// the stop has begun. The claim is there for the whole of the act: its row @@ -215,10 +242,16 @@ pub fn init(rsdp_addr: u64) { let Some(control) = super::power::pm1a_control() else { return log!("acpi: no ACPI row — no PM1a control block declared"); }; - if let Some(Err(why)) = fixed.legacy.map(|legacy| smi_cmd::declare(legacy.smi_cmd)) { + let facs = toyos_acpi::facs(fadt.phys(), &fadt); + // A FACS this kernel cannot read leaves `S4BIOS_F` unread, and the byte kept. + let s4bios = match facs { + Ok(facs) => facs.s4bios, + Err(toyos_acpi::FacsRefused::Absent) => false, + Err(_) => true, + }; + if let Some(Err(why)) = fixed.smi_cmd.map(|named| smi_cmd::declare(named.port, named.named(s4bios))) { return log!("acpi: no ACPI row — SMI_CMD not declared: {why:?}"); } - let legacy = fixed.legacy; let ec = embedded_controller(rsdp_addr, fixed.gpe0); let Some(sci) = super::ioapic::sci(fixed.sci_int) else { return log!("acpi: no ACPI row — no I/O APIC carries the SCI"); @@ -252,8 +285,8 @@ pub fn init(rsdp_addr: u64) { if cpu::inw(control.port(0)) & SCI_EN != 0 { "ACPI" } else { "legacy" }, ); isa::fill(ROW, Function { name: "the ACPI fixed hardware", runs, irqs: vec![], wires: vec![sci] }); - let (ecam, lock) = (ecam(rsdp_addr), global_lock(&fadt)); - let hardware = Hardware { fixed, control, legacy, ec, rsdp: rsdp_addr, ecam, lock }; + let (ecam, lock) = (ecam(rsdp_addr), global_lock(facs)); + let hardware = Hardware { fixed, control, ec, rsdp: rsdp_addr, ecam, lock }; let was = HARDWARE.swap(Box::into_raw(Box::new(hardware)), Ordering::Release); assert!(was.is_null(), "acpi: init ran twice"); } @@ -275,12 +308,12 @@ fn ecam(rsdp_addr: u64) -> Option { /// The Global Lock of the FACS the FADT names, said by name where there is /// none or it is none this kernel takes: a FACS that does not decode, and one /// whose lock word is not in memory the firmware's map gives the firmware. -fn global_lock(fadt: &toyos_acpi::Table

) -> GlobalLock { +fn global_lock(facs: Result) -> GlobalLock { let refused = |why: core::fmt::Arguments| { log!("acpi: a Global Lock this kernel cannot take — {why}: every take is refused"); GlobalLock::Refused }; - let facs = match toyos_acpi::facs(fadt.phys(), fadt) { + let facs = match facs { Ok(facs) => facs, Err(toyos_acpi::FacsRefused::Absent) => { log!("acpi: no Global Lock — the FADT names no FACS: every take is answered taken"); @@ -343,6 +376,13 @@ fn info(hardware: &Hardware) -> AcpiInfo { } } +/// `SMI_CMD` and the way out of legacy mode and back, where the FADT names +/// all three. +fn legacy(hardware: &Hardware) -> Option<(u16, LegacyMode)> { + let named = hardware.fixed.smi_cmd?; + Some((named.port, named.legacy()?)) +} + fn sci_enabled(hardware: &Hardware) -> bool { cpu::inw(hardware.control.port(0)) & SCI_EN != 0 } @@ -360,7 +400,7 @@ fn enter(hardware: &Hardware) -> Result<(), ClaimError> { log!("acpi: this machine stays in legacy mode — {why}"); Err(ClaimError::Unusable) }; - let Some(legacy) = hardware.legacy else { + let Some((port, legacy)) = legacy(hardware) else { return refuse("the FADT names no SMI_CMD, ACPI_ENABLE and ACPI_DISABLE to leave it and come back with"); }; let enable = legacy.acpi_enable.get(); @@ -394,8 +434,7 @@ fn enter(hardware: &Hardware) -> Result<(), ClaimError> { } } log!( - "acpi: ACPI mode: ACPI_ENABLE {enable:#04x} written to SMI_CMD {:#x} {write}; SCI_EN set {} after", - legacy.smi_cmd, + "acpi: ACPI mode: ACPI_ENABLE {enable:#04x} written to SMI_CMD {port:#x} {write}; SCI_EN set {} after", crate::clock::now() - written, ); Ok(()) @@ -424,7 +463,7 @@ pub fn release() { /// the hardware's to reset (ACPI 6.5 §4.8.2.5, Table 4.13), so it is not /// cleared here before the write as Table 5.9's `ACPI_DISABLE` has it. fn leave(hardware: &Hardware) { - let legacy = hardware.legacy.expect("ACPI_ENABLE was written to it"); + let (port, legacy) = legacy(hardware).expect("ACPI_ENABLE was written to it"); let disable = legacy.acpi_disable.get(); let Some(write) = smi_cmd::write(disable) else { return log!("acpi: ACPI_DISABLE not written: the machine is stopping, and its power-off owns ACPI mode"); @@ -438,19 +477,17 @@ fn leave(hardware: &Hardware) { } if by.reached(crate::clock::now()) { return log!( - "acpi: still in ACPI mode: ACPI_DISABLE {disable:#04x} written to SMI_CMD {:#x} {write}; PM1a_CNT reads \ + "acpi: still in ACPI mode: ACPI_DISABLE {disable:#04x} written to SMI_CMD {port:#x} {write}; PM1a_CNT reads \ {control:#06x} {HANDBACK} after, SCI_EN still set: nothing serves this machine's buttons until a holder \ - claims them", - legacy.smi_cmd + claims them" ); } core::hint::spin_loop(); }; ENABLED.store(false, Ordering::Relaxed); log!( - "acpi: legacy mode again: ACPI_DISABLE {disable:#04x} written to SMI_CMD {:#x} {write}; PM1a_CNT reads \ + "acpi: legacy mode again: ACPI_DISABLE {disable:#04x} written to SMI_CMD {port:#x} {write}; PM1a_CNT reads \ {control:#06x} {} after, SCI_EN clear", - legacy.smi_cmd, crate::clock::now() - written, ); } @@ -727,24 +764,59 @@ fn memory(hardware: &Hardware, acting: &Holder, request: &mut Access, width: Wid } } +/// Why an access was not made: the policy's refusal, which goes back in the +/// request, or the stop, which begun after the claim's holder was let act. +enum Unmade { + Refused(Refused), + Stopping, +} + +impl From for Unmade { + fn from(refused: Refused) -> Self { + Self::Refused(refused) + } +} + /// One port access, decided and made. -fn port(_acting: &Holder, address: u64, width: Width, write: Option) -> Result { +fn port(hardware: &Hardware, acting: &mut Holder, address: u64, width: Width, write: Option) -> Result { let port = u16::try_from(address).map_err(|_| Refused::PortSpan)?; - let passed = firmware::port(|port| pio::standing(port, ROW), port, width, write.is_some())?; - match write { - None => Ok(read_port(&passed)), - Some(value) => { + match (firmware::port(|port| pio::standing(port, ROW), port, width, write), write) { + (PortVerdict::Through(passed), None) => Ok(read_port(&passed)), + (PortVerdict::Through(passed), Some(value)) => { write_port(&passed, value); Ok(0) } + (PortVerdict::FirmwareCall(asked), _) => call(hardware, &mut acting.calls, asked), + (PortVerdict::Refused(refused), _) => Err(refused.into()), + } +} + +/// Make the call into the firmware the policy passed, on the boot processor, +/// or refuse it past the rate; returned from once the firmware's handler has. +/// The first call of each byte is said with what the boot processor read +/// around it; every call is counted beside the `out` (`smi_cmd::counted`), +/// and a refusal is the caller's to say. +fn call(hardware: &Hardware, calls: &mut Calls, asked: FirmwareCall) -> Result { + if !calls.rate.admit(crate::clock::nanos_since_boot()) { + return Err(Refused::CommandRate.into()); + } + let value = asked.value(); + let written = smi_cmd::write(value).ok_or(Unmade::Stopping)?; + let (word, bit) = (&mut calls.said[usize::from(value / 64)], 1u64 << (value % 64)); + if *word & bit == 0 { + *word |= bit; + let port = hardware.fixed.smi_cmd.expect("the policy passed a call to the SMI_CMD the FADT names").port; + log!("acpi: firmware call {value:#04x} written to SMI_CMD {port:#x} {written}; the first of that byte"); } + Ok(0) } /// Make the access `request` names for the holder of the claim that lends /// `row`, or refuse it by name: a read's value, the refusal and the memory /// type are written back into it. `Err` is a request that names no space, /// width or direction, a value wider than its width, or a reserved byte that -/// is not zero; and `Gone` once the stop has begun. +/// is not zero; and `Gone` once the stop has begun, whether before the access +/// or under a call into the firmware it asked for. pub fn access(row: &isa::Row, request: &mut Access) -> Result<(), SyscallError> { let hardware = hardware().expect("a claimed row has its hardware"); let (Some(space), Some(width)) = (Space::from_raw(request.space), Width::from_raw(request.width)) else { @@ -758,18 +830,18 @@ pub fn access(row: &isa::Row, request: &mut Access) -> Result<(), SyscallError> if request.reserved != [0; 3] { return Err(SyscallError::InvalidArgument); } - let acting = acting(row)?; + let mut acting = acting(row)?; request.memory_type = toyos_abi::acpi::UNLISTED; let made = match space { - Space::SystemMemory => memory(hardware, &acting, request, width, write), - Space::SystemIo => port(&acting, request.address, width, write), + Space::SystemMemory => memory(hardware, &acting, request, width, write).map_err(Unmade::from), + Space::SystemIo => port(hardware, &mut acting, request.address, width, write), Space::PciConfig => { // `toyos_abi::acpi::pci_address`: nothing above the segment group. let at = request.address; let function = PciFunction { bus: (at >> 24) as u8, device: (at >> 19 & 0x1F) as u8, function: (at >> 16 & 7) as u8 }; match at >> 48 { - 0 => config(hardware, &acting, (at >> 32) as u16, function, at as u16, width, write.is_some()), - _ => Err(Refused::ConfigUnreachable), + 0 => config(hardware, &acting, (at >> 32) as u16, function, at as u16, width, write.is_some()).map_err(Unmade::from), + _ => Err(Refused::ConfigUnreachable.into()), } } }; @@ -780,7 +852,8 @@ pub fn access(row: &isa::Row, request: &mut Access) -> Result<(), SyscallError> request.value = value; } } - Err(refused) => request.refused = refused as u8, + Err(Unmade::Refused(refused)) => request.refused = refused as u8, + Err(Unmade::Stopping) => return Err(SyscallError::Gone), } Ok(()) } diff --git a/kernel/src/arch/x86_64/counters.rs b/kernel/src/arch/x86_64/counters.rs index 4c4b25dae3b..b5a26ee40fc 100644 --- a/kernel/src/arch/x86_64/counters.rs +++ b/kernel/src/arch/x86_64/counters.rs @@ -7,7 +7,7 @@ use core::sync::atomic::{AtomicU8, Ordering::Relaxed}; use toyos_cpuvuln::CounterFacts; use super::{cpu, percpu}; -use crate::counters::Hardware; +use crate::counters::{Firmware, Hardware}; use crate::scheduler::MAX_CPUS; const MSR_SMI_COUNT: u32 = 0x34; @@ -43,5 +43,6 @@ pub fn read() -> Hardware { aperf: msr(APERF_MPERF, IA32_APERF), mperf: msr(APERF_MPERF, IA32_MPERF), envelope: super::control_regs::envelope(), + firmware: super::smi_cmd::counted().map(|(calls, nanos)| Firmware { calls, nanos }), } } diff --git a/kernel/src/arch/x86_64/smi_cmd.rs b/kernel/src/arch/x86_64/smi_cmd.rs index 1143372f9ce..e0aea0018e2 100644 --- a/kernel/src/arch/x86_64/smi_cmd.rs +++ b/kernel/src/arch/x86_64/smi_cmd.rs @@ -14,12 +14,31 @@ //! whether it was idle or busy, or from a lock's spin where its interrupts are //! closed ([`serve_here`]), and spins until that round is answered. [`write`] //! therefore returns only once the `out` has retired, which is after the -//! firmware's handler has. A boot processor that answers neither way within -//! [`DEAF_CPU`] is a panic, as one that answers no TLB shootdown is. +//! firmware's handler has. +//! +//! **That wait has two subjects, and neither's time is inside the other's** +//! (`kernel::bootwrite`). The boot processor says it has the write before it +//! makes it: the `out` takes the [`Taken`] that saying so returns. Until then +//! the asker waits for its kick to be taken, and a boot processor that has +//! not taken it in [`DEAF_CPU`] is a panic, as one that answers no TLB +//! shootdown is. From then it waits for the firmware, whose handler no +//! instruction of this kernel's can end: one that holds the boot processor +//! [`DEAF_CPU`]'s span is a panic that says the firmware held it +//! ([`held_too_long`]), the asker's where the asker runs meanwhile and the +//! boot processor's own once the `out` retires, so whichever CPU asked. What +//! a handler that outlasts each of this kernel's bounds meets is +//! `issues/a-firmware-call-does-what-its-handler-chooses-and-the-kernel-bounds-only-the-call.md`'s. //! //! The port is this module's alone, so the `out` in [`answer`] is the only //! one the kernel can make to it, and [`answer`] reads which CPU it is on //! with interrupts closed beside that `out`: no caller's state decides it. +//! It is counted there too, with the time it held the boot processor +//! ([`counted`]), whoever asked for it. +//! +//! **The `acpi` claim's holder writes no byte here: it asks for one**, and the +//! declaration says which bytes are never written for it +//! (`toyos_userbound::Mediated::Command`): those the FADT gives a meaning, +//! which are this kernel's own commands. //! //! **None is made once the stop has begun**: the power-off waits out a write //! in flight ([`settle`]) and then owns the hardware, and an SMI it did not @@ -27,15 +46,16 @@ use core::fmt; use alloc::string::String; -use core::sync::atomic::{AtomicU32, AtomicU64, AtomicU8, Ordering::Relaxed}; +use core::sync::atomic::{AtomicU32, AtomicU64, AtomicU8, Ordering::{Acquire, Relaxed, Release}}; -use toyos_userbound::{Mediated, Ports, Undeclared}; +use toyos_userbound::{KeptCommands, Mediated, Ports, Undeclared}; use super::pio::{self, Slot, TakenBack}; use super::{apic, cpu, percpu, IrqGuard}; use crate::shootdown::Shootdown; use crate::sync::Lock; -use crate::time::{Deadline, Duration, DEAF_CPU}; +use crate::time::{Duration, DEAF_CPU}; +use kernel::bootwrite::{self, Turn}; /// The boot processor. const BOOT: u32 = 0; @@ -49,9 +69,18 @@ static WRITING: Lock<()> = Lock::new(()); static ROUND: Shootdown = Shootdown::new(); /// What the round in flight writes: stored before the round is issued, read by its answer. static ASKED: AtomicU8 = AtomicU8::new(0); +/// When the boot processor took the round in flight, in nanoseconds since +/// boot: stored by [`take`] before the `out`, [`UNTAKEN`] until then. +static TAKEN_NS: AtomicU64 = AtomicU64::new(UNTAKEN); +/// No time: the clock saturates before it reads this. +const UNTAKEN: u64 = u64::MAX; /// What the last answer read, published by the round's own answer. static ON: AtomicU32 = AtomicU32::new(0); static HELD_NS: AtomicU64 = AtomicU64::new(0); +/// Every write made and the nanoseconds they held the boot processor, which +/// alone writes and reads them. +static WRITES: AtomicU64 = AtomicU64::new(0); +static SPENT_NS: AtomicU64 = AtomicU64::new(0); static SMIS: [AtomicU64; 2] = [const { AtomicU64::new(UNREAD) }; 2]; /// No SMI count: the register holds 32 bits. @@ -81,14 +110,21 @@ impl fmt::Display for Written { } } -/// Declare the port the FADT names. Boot's. The `acpi` claim's holder reads it -/// and never writes it: a write is a command to the firmware, and [`write`] -/// makes every one. -pub fn declare(port: u16) -> Result<(), Undeclared> { - PORT.set(pio::declare("SMI_CMD", Ports::one(port), Mediated::ReadOnly)?); +/// Declare the port the FADT names, and the values it names for it. Boot's. +/// The `acpi` claim's holder reads it and never writes it: a write is a +/// command to the firmware, and [`write`] makes every one. +pub fn declare(port: u16, named: [Option; 5]) -> Result<(), Undeclared> { + PORT.set(pio::declare("SMI_CMD", Ports::one(port), Mediated::Command(KeptCommands(named)))?); Ok(()) } +/// How many writes this CPU has made and the nanoseconds they held it, on +/// the boot processor of a machine that names the port; `None` on every +/// other CPU, which makes none. +pub fn counted() -> Option<(u64, u64)> { + (percpu::cpu_id() == BOOT && PORT.get().is_some()).then(|| (WRITES.load(Relaxed), SPENT_NS.load(Relaxed))) +} + /// Write `value` to `SMI_CMD` on the boot processor and return once it is /// written; `None`, and nothing written, once the stop has begun. pub fn write(value: u8) -> Option { @@ -99,17 +135,25 @@ pub fn write(value: u8) -> Option { } let asked_from = percpu::cpu_id(); ASKED.store(value, Relaxed); + TAKEN_NS.store(UNTAKEN, Relaxed); let generation = ROUND.issue(); // The boot processor's own ask, unless a kick taken since the issue has answered it already. answer(); if !ROUND.served(BOOT as usize, generation) { apic::kick_cpu(BOOT); - let by = Deadline::at(crate::clock::now() + Duration::from_nanos(DEAF_CPU.nanos())); - while !ROUND.served(BOOT as usize, generation) { - assert!( - !by.reached(crate::clock::now()), - "smi_cmd: the boot processor has not written {value:#04x} for cpu{asked_from} in {DEAF_CPU}: it is not taking interrupts" - ); + let asked_ns = crate::clock::nanos_since_boot(); + loop { + // The clock before what it judges: a write found untaken or unretired was so at the time read. + let now_ns = crate::clock::nanos_since_boot(); + let taken_ns = Some(TAKEN_NS.load(Acquire)).filter(|&at| at != UNTAKEN); + match bootwrite::turn(DEAF_CPU.nanos(), asked_ns, taken_ns, ROUND.served(BOOT as usize, generation), now_ns) { + Turn::Retired => break, + Turn::Wait => {} + Turn::Deaf => panic!( + "smi_cmd: the boot processor has not taken cpu{asked_from}'s kick to write {value:#04x} in {DEAF_CPU}: it is not taking interrupts" + ), + Turn::Outlasted => held_too_long(value, None), + } core::hint::spin_loop(); // A caller with interrupts closed owes the boot processor's shootdown an answer while it waits. super::tlb::poll(); @@ -150,16 +194,58 @@ fn answer() { let value = ASKED.load(Relaxed); let smi = || super::counters::read().smi.unwrap_or(UNREAD); let before = smi(); - let from = crate::clock::now(); - // SAFETY: `SMI_CMD`, declared; the value is one the FADT names for it, by `write`'s caller. - unsafe { cpu::outb(port, value) }; - HELD_NS.store((crate::clock::now() - from).nanos(), Relaxed); + let held = out(take(), port, value); + if bootwrite::outlasted(DEAF_CPU.nanos(), held) { + held_too_long(value, Some(held)); + } + HELD_NS.store(held, Relaxed); + // One writer, this CPU with interrupts closed: no update is lost. + WRITES.store(WRITES.load(Relaxed) + 1, Relaxed); + SPENT_NS.store(SPENT_NS.load(Relaxed) + held, Relaxed); SMIS[0].store(before, Relaxed); SMIS[1].store(smi(), Relaxed); ON.store(percpu::cpu_id(), Relaxed); }); } +/// The boot processor has said it has the round in flight. The `out` is made +/// with one, so none is made before its asker can read that its kick was +/// taken. +struct Taken { + at_ns: u64, +} + +fn take() -> Taken { + let at_ns = crate::clock::nanos_since_boot(); + TAKEN_NS.store(at_ns, Release); + Taken { at_ns } +} + +/// The one `out` to the port, and the nanoseconds it held this CPU: to after +/// the firmware's handler, where the write raises an SMI. +fn out(taken: Taken, port: pio::Port, value: u8) -> u64 { + // SAFETY: `SMI_CMD`, declared; the value is one the FADT names for it + // or one the mediation's policy passed, by `write`'s caller. + unsafe { cpu::outb(port, value) }; + crate::clock::nanos_since_boot().saturating_sub(taken.at_ns) +} + +/// The firmware has held the boot processor [`DEAF_CPU`]'s span in the +/// handler of one write: for `Some` nanoseconds where the write has retired, +/// and still where it has not. +#[cold] +#[inline(never)] +fn held_too_long(value: u8, retired_after_ns: Option) -> ! { + match retired_after_ns { + Some(held_ns) => panic!( + "smi_cmd: the firmware held the boot processor {held_ns}ns in its handler for the {value:#04x} written to SMI_CMD: the bound is {DEAF_CPU}" + ), + None => panic!( + "smi_cmd: the firmware has held the boot processor in its handler for the {value:#04x} written to SMI_CMD and has not returned: the bound is {DEAF_CPU}" + ), + } +} + /// Wait out a write to `SMI_CMD` in flight: the stop has begun, so none follows. pub fn settle(_taken: &TakenBack) { drop(WRITING.lock()); diff --git a/kernel/src/counters.rs b/kernel/src/counters.rs index 7d7cd8c953b..7aef37d90ec 100644 --- a/kernel/src/counters.rs +++ b/kernel/src/counters.rs @@ -50,6 +50,14 @@ pub struct Hardware { pub aperf: Option, pub mperf: Option, pub envelope: Option, + pub firmware: Option, +} + +/// The commands this CPU wrote to the firmware, where it is the one that +/// writes them. +pub struct Firmware { + pub calls: u64, + pub nanos: u64, } /// The power envelope's registers, where the CPU's performance request is @@ -113,6 +121,8 @@ fn sample(me: usize) -> [u64; WORDS] { Counter::HwpRequest => hardware.envelope.as_ref().map(|e| e.hwp_request), Counter::HwpRequestPkg => hardware.envelope.as_ref().map(|e| e.hwp_request_pkg), Counter::EnergyPerfBias => hardware.envelope.as_ref().map(|e| e.energy_perf_bias), + Counter::FirmwareCalls => hardware.firmware.as_ref().map(|f| f.calls), + Counter::FirmwareNanos => hardware.firmware.as_ref().map(|f| f.nanos), }; if let Some(value) = value { words[1] |= 1 << counter as usize; diff --git a/tests/acpicase/system.toml b/tests/acpicase/system.toml index bba53a2a5d9..126332abcc2 100644 --- a/tests/acpicase/system.toml +++ b/tests/acpicase/system.toml @@ -12,15 +12,16 @@ syscap = ["logread"] # `device` because the job mints the `acpi` claim the server is handed, and # `dup` because test-runner hands a job its capability as a duplicate and -# hands none without it; `power` because the job list ends the machine, -# asking the supervisor. +# hands none without it; `counters` because `acpi_mediated` reads what the +# boot processor counted of the firmware calls it asked for; `power` because +# the job list ends the machine, asking the supervisor. # # `args` as it stands is the guest's: `acpi_mediated_access` and # `acpi_supply_outlives_holder` boot this file with their probe on ROOT. A # metal row's job list replaces it. [programs.test-runner] receives = ["power"] -syscap = ["device", "dup"] +syscap = ["device", "dup", "counters"] args = ["test_rs_acpi_mediated", "reboot"] [programs.toybox] diff --git a/tests/toyos-rust-tests/src/bin/acpi_mediated.rs b/tests/toyos-rust-tests/src/bin/acpi_mediated.rs index acbe19e3dba..0ed9c5bfa9a 100644 --- a/tests/toyos-rust-tests/src/bin/acpi_mediated.rs +++ b/tests/toyos-rust-tests/src/bin/acpi_mediated.rs @@ -11,6 +11,13 @@ //! machine whose firmware is not using it can take. Each address is found as //! a holder finds it, from the RSDP the claim's description names. //! +//! **A byte for `SMI_CMD` is a call into the firmware** ([`firmware`]), which +//! this guest's chipset model answers by keeping the byte, and by leaving +//! ACPI mode on the one the FADT names for that: the port read back is +//! QEMU's word for which bytes the kernel wrote, and `SCI_EN` its word that +//! the kernel's own command was not among them. The real machine's firmware +//! runs a handler on every byte, so none is written to it by a probe. +//! //! The power-off is the kernel's with the sleep type a holder supplies, once //! under its claim. On this boot nothing has, so first a shutdown is refused //! and the machine goes on. @@ -44,9 +51,13 @@ use toyos::endow::{Endowments, SYSCAP_LABEL}; use toyos::syscap::SysCap; use toyos::{AsHandle, Device}; use toyos_abi::acpi::{pci_address, Access, AcpiInfo, Refused, Space, Width, UNLISTED}; +use toyos_abi::counters::{Counter, RawRecord, Record}; use toyos_abi::syscall::{self, debug_action, DeviceType, SyscallError}; use toyos_abi::RawHandle; +#[path = "../arch/cpu.rs"] +mod cpu; + const SELF_PATH: &str = "/system/bin/test_rs_acpi_mediated"; const CLAIM_LABEL: &str = "acpi-claim"; @@ -179,6 +190,7 @@ fn probe() { memory(&holder, &info); ports(&holder, &info); + firmware(&holder, &info, &cap); configuration(&holder, &info); lock(&holder, &info); sleep_type(&holder); @@ -309,22 +321,163 @@ fn ports(holder: &Holder, info: &AcpiInfo) { // The POST port, which the kernel declared and opens. assert_eq!(holder.write(Space::SystemIo, 0x80, Width::Byte, 0x5A), Ok(()), "acpi: the POST port"); - // `PM1a_CNT` and `SMI_CMD`, as the FADT names them (Table 5.9, at 64 and 48): read, never written. + // `PM1a_CNT`, as the FADT names it (Table 5.9, at 64): read, never written. let fadt = holder.table(info.rsdp, b"FACP"); let control = holder.memory(fadt + 64, Width::DWord); let held = holder.read(Space::SystemIo, control, Width::Word).expect("acpi: PM1a_CNT reads"); assert_eq!(held & 1, 1, "acpi: PM1a_CNT reads {held:#06x}, SCI_EN clear, on a machine a claim put in ACPI mode"); assert_eq!(holder.write(Space::SystemIo, control, Width::Word, held), Err(Refused::ReadOnlyPort), "acpi: PM1a_CNT was written"); - let smi_cmd = holder.memory(fadt + 48, Width::DWord); - assert_ne!(smi_cmd, 0, "acpi: this firmware names no SMI_CMD"); - holder.read(Space::SystemIo, smi_cmd, Width::Byte).expect("acpi: SMI_CMD reads"); - assert_eq!(holder.write(Space::SystemIo, smi_cmd, Width::Byte, 0), Err(Refused::ReadOnlyPort), "acpi: SMI_CMD was written"); // The claim's own event block, which nothing declared, and a dword of it. let status = u64::from(info.pm1_event.port); holder.read(Space::SystemIo, status, Width::Word).expect("acpi: the claim's own PM1 status"); holder.read(Space::SystemIo, status, Width::DWord).expect("acpi: the claim's own PM1 event block as a dword"); - println!("acpi: COM1, the CMOS index, the 8259 and the configuration mechanism were refused KernelPort; the i8042's row ClaimedPort; PM1a_CNT and SMI_CMD read and refused their write ReadOnlyPort; the POST port was written"); + println!("acpi: COM1, the CMOS index, the 8259 and the configuration mechanism were refused KernelPort; the i8042's row ClaimedPort; PM1a_CNT read and refused its write ReadOnlyPort; the POST port was written"); +} + +/// Bytes this guest's FADT gives no meaning, which its chipset model keeps +/// and does nothing on: the calls asked from a thread off the boot processor +/// where there is one, and the two the storm alternates. +const CROSSED: [u64; 3] = [0x51, 0x52, 0x53]; +const STORMED: [u64; 2] = [0x54, 0x55]; + +/// Calls the storm asks at most: what firmware's AML asks of a handler that +/// never answers, one a millisecond for ten seconds. +const STORM: usize = 10_000; + +/// Threads started in turn before one must have found itself off the boot +/// processor, as `acpi_release`'s. +const STARTS: usize = 64; + +/// What the boot processor counts of the kernel's writes to `SMI_CMD`, read +/// in a round of the counters issued after the last one read here returned: a +/// read within the kernel's joining window of a round answers with that +/// round, which may be older than what the caller just did, so this reads +/// until the boot processor's stamp is one it has not seen. Nothing else on +/// this boot reads the counters, so a round not seen is one this asked for. +struct Counted<'a> { + cap: &'a SysCap, + seen: Option, +} + +impl Counted<'_> { + fn firmware_calls(&mut self) -> u64 { + loop { + let mut raw = vec![RawRecord::EMPTY; syscall::cpu_count() as usize]; + let n = self.cap.counters(&mut raw).expect("acpi: the estate's capability reads the counters"); + let records: Vec = raw[..n].iter().map(|r| Record::decode(r).expect("acpi: a record that decodes")).collect(); + let stamp = records[0].get(Counter::Stamp); + if records.iter().any(|r| r.stale) || stamp == self.seen { + std::thread::yield_now(); + continue; + } + self.seen = stamp; + // No other CPU holds the counter: none writes the port. + for r in &records[1..] { + assert_eq!(r.get(Counter::FirmwareCalls), None, "acpi: cpu{} counts firmware calls, and makes none", r.cpu); + } + return records[0].get(Counter::FirmwareCalls).expect("acpi: the boot processor counts its writes to SMI_CMD"); + } + } +} + +/// One byte for `SMI_CMD`, asked by a thread that read itself off the boot +/// processor as its first act where the machine has another CPU: the x2APIC +/// id it read, 0 on a machine of one. +fn call_off_the_boot_processor(holder: &Holder, smi_cmd: u64, value: u64) -> u32 { + let claim = holder.handle(); + let ask = || { + let mut access = Access::write(Space::SystemIo, smi_cmd, Width::Byte, value); + assert_eq!(syscall::acpi_access(claim, &mut access), Ok(Ok(value)), "acpi: firmware call {value:#04x}"); + }; + if syscall::cpu_count() == 1 { + ask(); + return 0; + } + for _ in 0..STARTS { + let on = std::thread::scope(|threads| { + let asker = threads.spawn(|| { + let on = cpu::x2apic_id(); + (on != 0).then(|| { + ask(); + on + }) + }); + asker.join().expect("acpi: the calling thread") + }); + if let Some(on) = on { + return on; + } + } + panic!("acpi: {STARTS} threads in a row started on the boot processor"); +} + +fn firmware(holder: &Holder, info: &AcpiInfo, cap: &SysCap) { + // Table 5.9: `SMI_CMD` at 48, `ACPI_ENABLE` at 52, `ACPI_DISABLE` at 53, `PM1a_CNT_BLK` at 64. + let fadt = holder.table(info.rsdp, b"FACP"); + let smi_cmd = holder.memory(fadt + 48, Width::DWord); + assert_ne!(smi_cmd, 0, "acpi: this firmware names no SMI_CMD"); + let named = [holder.memory(fadt + 52, Width::Byte), holder.memory(fadt + 53, Width::Byte)]; + let control = holder.memory(fadt + 64, Width::DWord); + let sci_en = || holder.read(Space::SystemIo, control, Width::Word).expect("acpi: PM1a_CNT reads") & 1; + let last_written = || holder.read(Space::SystemIo, smi_cmd, Width::Byte).expect("acpi: SMI_CMD reads"); + let write = |at: u64, width: Width, value: u64| holder.write(Space::SystemIo, at, width, value); + let mut counted = Counted { cap, seen: None }; + let hex = |bytes: &[u64]| bytes.iter().map(|byte| format!("{byte:#04x}")).collect::>().join(", "); + // Whether this chipset interrupts its firmware on a write to the port: + // the ICH9's `SMI_EN` at PMBASE + 30h, `APMC_EN` its bit 5, PMBASE being + // where the FADT puts the PM1a event block. A reading. + let smi_en = holder.read(Space::SystemIo, u64::from(info.pm1_event.port) + 0x30, Width::DWord).expect("acpi: SMI_EN reads"); + + // The kernel's own commands: refused, and the chipset saw neither, which + // leaves ACPI mode on the second. + let before = (counted.firmware_calls(), last_written()); + assert!(named.iter().all(|&byte| byte != 0 && !CROSSED.contains(&byte) && !STORMED.contains(&byte)), "acpi: this FADT names {}", hex(&named)); + for kept in named { + assert_eq!(write(smi_cmd, Width::Byte, kept), Err(Refused::KernelCommand), "acpi: {kept:#04x}, which the FADT names, was written to SMI_CMD"); + } + assert_eq!(sci_en(), 1, "acpi: SCI_EN reads clear after a refused ACPI_DISABLE"); + // A command is one byte to the one port. + for (at, width) in [(smi_cmd, Width::Word), (smi_cmd, Width::DWord), (smi_cmd - 1, Width::Word), (smi_cmd - 3, Width::DWord)] { + assert_eq!(write(at, width, CROSSED[0]), Err(Refused::CommandSpan), "acpi: a {width:?} at {at:#x}"); + } + assert_eq!((counted.firmware_calls(), last_written()), before, "acpi: a refused command was written or counted"); + + // A call: the chipset has the byte, and the boot processor counted one write. + let mut asked_from = vec![call_off_the_boot_processor(holder, smi_cmd, CROSSED[0])]; + assert_eq!(last_written(), CROSSED[0], "acpi: SMI_CMD does not hold the byte the kernel was asked to write"); + assert_eq!(counted.firmware_calls(), before.0 + 1, "acpi: one firmware call"); + for value in &CROSSED[1..] { + asked_from.push(call_off_the_boot_processor(holder, smi_cmd, *value)); + assert_eq!(last_written(), *value); + } + assert_eq!(sci_en(), 1, "acpi: SCI_EN reads clear after calls of {}", hex(&CROSSED)); + + // The storm: the same call asked over and over is made a few times and + // then refused, and the refused byte never reached the port. + let mut made = CROSSED.len(); + let stormed = (0..STORM).find(|i| { + let value = STORMED[i % 2]; + match write(smi_cmd, Width::Byte, value) { + Ok(()) => { + made += 1; + assert_eq!(last_written(), value); + false + } + Err(Refused::CommandRate) => true, + Err(other) => panic!("acpi: firmware call {value:#04x} was refused {other:?}"), + } + }); + let refused_at = stormed.unwrap_or_else(|| panic!("acpi: {STORM} firmware calls in a row were made, and none refused")); + assert_ne!(last_written(), STORMED[refused_at % 2], "acpi: a call refused CommandRate reached SMI_CMD"); + assert_eq!(counted.firmware_calls(), before.0 + made as u64, "acpi: the boot processor counted another number of writes than were made"); + println!( + "acpi: firmware calls of {} were asked from the CPUs of x2APIC id {asked_from:?}; this chipset's SMI_EN reads {smi_en:#010x}, APMC_EN {}", + hex(&CROSSED), + if smi_en & 1 << 5 == 0 { "clear, so no call interrupts its firmware" } else { "set, so each call interrupts its firmware" }, + ); + println!("acpi: {made} firmware calls were made before call {} of the storm was refused", refused_at + 1); + println!("acpi: ACPI_ENABLE and ACPI_DISABLE were refused SMI_CMD as KernelCommand and SCI_EN stayed set, a write wider than a byte was refused CommandSpan, every firmware call made was read back from the port and counted on the boot processor, and a storm of them was refused CommandRate"); } fn configuration(holder: &Holder, info: &AcpiInfo) { diff --git a/tests/toyos.rs b/tests/toyos.rs index 95cb23ad433..da4ecd56f21 100644 --- a/tests/toyos.rs +++ b/tests/toyos.rs @@ -1677,13 +1677,14 @@ fn virt_job(profile: qemu::Profile, job: &str, said: &str) -> Result<(), String> /// What `acpi_mediated` says, an arm a line, once the kernel answered each as /// its policy says. -const ACPI_MEDIATED_SAID: [&str; 9] = [ +const ACPI_MEDIATED_SAID: [&str; 10] = [ "acpi: an unbound claim was refused its access, the lock and the power-off's sleep type", "acpi: RAM was refused both ways as UsableMemory", "acpi: the RSDP read through as type 9 and its write was refused TableWrite", "acpi: an unlisted register was read and refused its write MemoryType, an unlisted address below 1 MiB was refused UnlistedCached, and the interrupt controllers, the HPET and a function's BAR DeviceMemory", "acpi: the FACS read through as type 10, its write was refused FacsWrite, and the memory after it was written and put back", - "acpi: COM1, the CMOS index, the 8259 and the configuration mechanism were refused KernelPort; the i8042's row ClaimedPort; PM1a_CNT and SMI_CMD read and refused their write ReadOnlyPort; the POST port was written", + "acpi: COM1, the CMOS index, the 8259 and the configuration mechanism were refused KernelPort; the i8042's row ClaimedPort; PM1a_CNT read and refused its write ReadOnlyPort; the POST port was written", + "acpi: ACPI_ENABLE and ACPI_DISABLE were refused SMI_CMD as KernelCommand and SCI_EN stayed set, a write wider than a byte was refused CommandSpan, every firmware call made was read back from the port and counted on the boot processor, and a storm of them was refused CommandRate", "by its address and through ECAM, and every write to configuration space was refused ConfigWrite", "acpi: the lock a dead holder left taken read free; it was taken and given back, given back with GBL_RLS where the firmware had asked, and found pending while the firmware owned it", "acpi: a sleep type wider than three bits was refused InvalidArgument, the next holder's replaced a dead one's, and a second under one claim was refused AlreadyExists", @@ -1701,6 +1702,13 @@ const ACPI_MEDIATED_SAID: [&str; 9] = [ /// from that, so its verdict is its last line, the kernel's and QEMU's; a /// probe that ends instead is one whose arm failed. On `cpus` CPUs: the /// second one's range registers are read where there is one. +/// +/// And the probe's calls into the firmware, the first write to `SMI_CMD` +/// this kernel makes on a guest, whose firmware hands it over in ACPI mode: +/// the kernel says the first call of each byte, five of them, each written +/// on the boot processor by that CPU's own reading beside the `out`, and +/// where there is a second CPU one at least asked from it, the probe having +/// asked three from a thread that read itself there. fn acpi_mediated_access(cpus: u32) -> Result<(), String> { const JOB: &str = "test_rs_acpi_mediated"; const HELD_INTO_THE_STOP: &str = "acpi: holding the Global Lock, and asking for the power-off with it"; @@ -1743,6 +1751,22 @@ fn acpi_mediated_access(cpus: u32) -> Result<(), String> { for line in ACPI_MEDIATED_SAID { eprintln!(" [acpi] {}", said.must_say(line)?.trim()); } + for reading in ["acpi: firmware calls of ", " firmware calls were made before call "] { + eprintln!(" [acpi] {}", said.must_say(reading)?.trim()); + } + let calls: Vec<&str> = said.text().lines().filter(|l| l.contains("acpi: firmware call 0x") && l.contains(" written to SMI_CMD ")).collect(); + if calls.len() != 5 { + return Err(format!("the kernel said {} first firmware calls where the probe made calls of five bytes: {calls:#?}", calls.len())); + } + let mut crossed = 0; + for line in &calls { + smi_cmd_writer(line)?; + crossed += usize::from(number_between(line, ", asked from cpu", "; the write held cpu")? != 0); + eprintln!(" [acpi] {}", line.trim()); + } + if cpus > 1 && crossed == 0 { + return Err(format!("no firmware call was asked from another CPU than the boot processor, so none crossed to it: {calls:#?}")); + } // The shutdown refused by the kernel's own name for it, before anything // was stopped: the probe ran on and said every line above. eprintln!(" [acpi] {}", said.must_say(NO_S5)?.trim()); @@ -3778,8 +3802,11 @@ const SMI_SPAN_NS: u64 = 4_444_000_000; /// edge's included because a line stamped in it may follow the read: the /// second is the idle machine's. The boot ran in ACPI mode, which /// `/system/bin/acpiserver`'s claim put it in: `idle0` reads after the -/// kernel's one write to `SMI_CMD`, and from there to `spin`, at least -/// [`SMI_SPAN_NS`] apart, no CPU's SMI count moves. Across the spin every +/// kernel's write of the enable to `SMI_CMD`, which the boot processor +/// counted with the time it held it, and no other CPU counts one; and from +/// there to `spin`, at least [`SMI_SPAN_NS`] apart, every CPU's SMI count +/// moves by the commands the kernel wrote to `SMI_CMD` in that span and by +/// nothing else: every firmware interrupt is one ToyOS asked for. Across the spin every /// CPU's MPERF ran nine tenths of its stamp or more [e], a CPU in C0 the whole /// span: MPERF counts at the TSC's rate there (SDM Vol. 3B, "Hardware /// Coordination Feedback"); and every CPU's busy frequency reached the lowest @@ -3885,23 +3912,39 @@ fn counters_on_metal(back: &metal::Readback) -> Result<(), String> { return Err(format!("idle0 and spin are {} ns apart, short of {SMI_SPAN_NS}: lengthen the spin", at2 - at0)); } let delta = |a: &Read, b: &Read, cpu: usize, name: &str| b[&cpu][name] - a[&cpu][name]; - // The boot's one write to `SMI_CMD`, and what the CPU that made it read after it. + // The enable's write to `SMI_CMD`, and what the CPU that made it read after it. let enabled = kernel.must_say("acpi: ACPI mode: ACPI_ENABLE ")?; let writer = smi_cmd_writer(enabled)?; let after = number_between(enabled, " before the write and ", " after")?; if idle0[&writer]["smi"] < after { - return Err(format!("idle0 read cpu{writer}'s SMI count below what it read after the ACPI enable ({after}): it read before the boot's last write to SMI_CMD")); + return Err(format!("idle0 read cpu{writer}'s SMI count below what it read after the ACPI enable ({after}): it read before that write to SMI_CMD")); } if let Ok(left) = kernel.must_say("acpi: legacy mode again") { return Err(format!("the machine left ACPI mode inside the boot: {left}")); } + // The boot processor writes every command and alone counts them: the + // enable is among those idle0 read, and where it is the only one, the + // time they held that CPU is the time the kernel's line gives it. + if let Some(cpu) = (0..cpus).find(|&cpu| idle0[&cpu].contains_key("firmware_calls") != (cpu == writer)) { + return Err(format!("cpu{cpu} counts firmware calls or the boot processor does not: {idle0:?}")); + } + let (calls, nanos) = (idle0[&writer]["firmware_calls"], idle0[&writer]["firmware_nanos"]); + let held = number_between(enabled, "; the write held cpu0 ", "ns, its SMI count ")?; + if calls == 0 || nanos < held || (calls == 1 && nanos != held) { + return Err(format!("idle0 read {calls} firmware calls that held cpu{writer} {nanos} ns, after an enable that held it {held} ns: {enabled}")); + } + let asked = delta(idle0, spin, writer, "firmware_calls"); let smis: Vec = (0..cpus).map(|cpu| delta(idle0, spin, cpu, "smi")).collect(); - if smis.iter().any(|&n| n != 0) { - return Err(format!("in ACPI mode the SMI count moved by {smis:?} over {} ns", at2 - at0)); + if smis.iter().any(|&n| n != asked) { + return Err(format!( + "in ACPI mode the SMI count moved by {smis:?} over {} ns, in which the kernel wrote {asked} commands to SMI_CMD", + at2 - at0 + )); } let firsts: Vec = (0..cpus).map(|cpu| idle0[&cpu]["smi"]).collect(); eprintln!(" [counters] {}", enabled.trim()); eprintln!(" [counters] idle0's SMI count per cpu {firsts:?}, the writer's after the enable {after} (a reading)"); + eprintln!(" [counters] idle0 read {calls} firmware call(s) that held cpu{writer} {nanos} ns, the enable's {held} ns among them"); let tsc_mhz = delta(idle0, spin, 0, "stamp") as f64 * 1e3 / (at2 - at0) as f64; let ratio = |a: &Read, b: &Read, cpu: usize, top: &str, bottom: &str| { delta(a, b, cpu, top) as f64 / delta(a, b, cpu, bottom) as f64 @@ -3936,7 +3979,7 @@ fn counters_on_metal(back: &metal::Readback) -> Result<(), String> { } let spinning: Vec = (0..cpus).map(|cpu| tsc_mhz * ratio(idle1, spin, cpu, "aperf", "mperf")).collect(); eprintln!( - " [counters] {cpus} cpus, SMI flat on each over {} ms; TSC {tsc_mhz:.0} MHz", + " [counters] {cpus} cpus, the SMI count on each moved by {asked} over {} ms, the commands the kernel wrote to SMI_CMD in it; TSC {tsc_mhz:.0} MHz", (at2 - at0) / 1_000_000 ); for (cpu, busy) in busy.iter().enumerate() { diff --git a/toyos-abi/src/acpi.rs b/toyos-abi/src/acpi.rs index b5cee977e3d..d4ce49f49a7 100644 --- a/toyos-abi/src/acpi.rs +++ b/toyos-abi/src/acpi.rs @@ -19,6 +19,12 @@ //! is mapped and no port opened. The firmware's Global Lock is taken and given //! back the same way ([`op::LOCK_TAKE`]), and goes back with the claim. //! +//! **A byte written to the FADT's `SMI_CMD` is a call into the firmware, which +//! the kernel makes on the boot processor** (ACPI 6.5 Table 5.9) and returns +//! from once the firmware's handler has. It makes none of a value the FADT +//! itself gives a meaning ([`Refused::KernelCommand`]), and no more than a +//! fixed few in a second ([`Refused::CommandRate`]). +//! //! **The power-off is the kernel's, with the sleep type the holder supplies** //! ([`op::S5`]): what `\_S5` evaluates to is in the firmware's AML, and //! until a holder has supplied it [`crate::syscall::SYS_SHUTDOWN`] is refused @@ -208,6 +214,15 @@ pub enum Refused { /// what types the address a register is not what every CPU reads it /// under. RangeRegistersDiffer = 16, + /// A byte for `SMI_CMD` that the FADT gives a meaning (ACPI 6.5 Table + /// 5.9: `ACPI_ENABLE`, `ACPI_DISABLE`, `S4BIOS_REQ`, `PSTATE_CNT`, + /// `CST_CNT`): the kernel's to write, or nobody's. + KernelCommand = 17, + /// A write that reaches `SMI_CMD` and is not one byte to it. + CommandSpan = 18, + /// A byte for `SMI_CMD` past the most the kernel writes for the holder in + /// a second. + CommandRate = 19, } impl Refused { @@ -229,6 +244,9 @@ impl Refused { 14 => Self::ConfigWrite, 15 => Self::UnlistedCached, 16 => Self::RangeRegistersDiffer, + 17 => Self::KernelCommand, + 18 => Self::CommandSpan, + 19 => Self::CommandRate, _ => return None, }) } diff --git a/toyos-abi/src/counters.rs b/toyos-abi/src/counters.rs index 69643bd7ff6..d9526c30a3e 100644 --- a/toyos-abi/src/counters.rs +++ b/toyos-abi/src/counters.rs @@ -10,8 +10,9 @@ //! not be read whole, and says nothing about it but its index. //! //! Every counter counts up from boot, so the difference of two reads is exact; -//! the power envelope's registers ([`Counter::HwpRequest`] and those after it) -//! count nothing and are what the CPU held when it read them. +//! the power envelope's registers ([`Counter::HwpRequest`], +//! [`Counter::HwpRequestPkg`] and [`Counter::EnergyPerfBias`]) count nothing +//! and are what the CPU held when it read them. //! //! [`SYS_COUNTERS`]: crate::syscall::SYS_COUNTERS @@ -40,10 +41,19 @@ pub enum Counter { HwpRequestPkg, /// Its energy/performance bias (`IA32_ENERGY_PERF_BIAS`), beside it. EnergyPerfBias, + /// Commands the kernel wrote to the firmware on this CPU (the FADT's + /// `SMI_CMD`), whoever asked: its own to enter and leave ACPI mode, and + /// each call it made for the `acpi` claim's holder. Held by the CPU that + /// writes them, the boot processor, on a machine that names the port. + FirmwareCalls, + /// Nanoseconds those writes held that CPU, each from before the write to + /// its return: the firmware's handler, where a write raises an interrupt + /// to it. + FirmwareNanos, } impl Counter { - pub const COUNT: usize = 8; + pub const COUNT: usize = 10; pub const ALL: [Counter; Self::COUNT] = [ Self::Stamp, Self::Smi, @@ -53,6 +63,8 @@ impl Counter { Self::HwpRequest, Self::HwpRequestPkg, Self::EnergyPerfBias, + Self::FirmwareCalls, + Self::FirmwareNanos, ]; pub const fn name(self) -> &'static str { @@ -65,13 +77,15 @@ impl Counter { Self::HwpRequest => "hwp_request", Self::HwpRequestPkg => "hwp_request_pkg", Self::EnergyPerfBias => "energy_perf_bias", + Self::FirmwareCalls => "firmware_calls", + Self::FirmwareNanos => "firmware_nanos", } } /// The rights a `SysCap` carries for this counter to be answered. pub const fn needs(self) -> Rights { match self { - Self::Stamp | Self::Smi => Rights::COUNTERS, + Self::Stamp | Self::Smi | Self::FirmwareCalls | Self::FirmwareNanos => Rights::COUNTERS, Self::Aperf | Self::Mperf | Self::Kicks @@ -179,8 +193,8 @@ mod tests { #[test] fn every_shape_of_record_reads_back_as_written() { for r in [ - record(false, [Some(1), Some(u64::from(u32::MAX)), Some(3), Some(u64::MAX), Some(0), Some(0x8000_2a04), Some(6), Some(0)]), - record(true, [Some(9), None, None, None, Some(4), None, Some(0x8000_ff01), None]), + record(false, [Some(1), Some(u64::from(u32::MAX)), Some(3), Some(u64::MAX), Some(0), Some(0x8000_2a04), Some(6), Some(0), Some(2), Some(31_936)]), + record(true, [Some(9), None, None, None, Some(4), None, Some(0x8000_ff01), None, Some(0), None]), record(true, [None; Counter::COUNT]), ] { assert_eq!(Record::decode(&r.encode()), Ok(r)); @@ -190,7 +204,7 @@ mod tests { /// Absent is not zero: a zero that is there reads back there. #[test] fn a_present_zero_is_not_an_absent_counter() { - let r = record(false, [Some(5), Some(0), None, None, None, None, None, None]); + let r = record(false, [Some(5), Some(0), None, None, None, None, None, None, None, None]); assert_eq!(Record::decode(&r.encode()).unwrap().get(Counter::Smi), Some(0)); assert_eq!(Record::decode(&r.encode()).unwrap().get(Counter::Aperf), None); } @@ -207,20 +221,21 @@ mod tests { #[test] fn a_word_under_an_absent_counter_is_refused() { - let mut raw = record(false, [Some(1), None, None, None, None, None, None, None]).encode(); + let mut raw = record(false, [Some(1), None, None, None, None, None, None, None, None, None]).encode(); raw.0[16 + 8 * Counter::Mperf as usize] = 1; assert_eq!(Record::decode(&raw), Err(Undecodable::Absent(Counter::Mperf))); } /// The counters that time programs and the power envelope are the trace - /// right's, and the two that are neither are readable on `COUNTERS` alone. + /// right's, and the four that are neither, the CPU's stamp and what its + /// firmware took of it, are readable on `COUNTERS` alone. #[test] fn what_times_a_program_or_is_power_needs_the_trace_right() { for counter in Counter::ALL { assert!(counter.needs().contains(Rights::COUNTERS), "{counter:?}"); assert_eq!( counter.needs().contains(Rights::TRACE), - !matches!(counter, Counter::Stamp | Counter::Smi), + !matches!(counter, Counter::Stamp | Counter::Smi | Counter::FirmwareCalls | Counter::FirmwareNanos), "{counter:?}" ); } diff --git a/toyos-abi/src/handle.rs b/toyos-abi/src/handle.rs index d6cac4d37d6..ec93f51273a 100644 --- a/toyos-abi/src/handle.rs +++ b/toyos-abi/src/handle.rs @@ -140,8 +140,10 @@ impl Rights { pub const INVENTORY: Rights = Rights(1 << 12); /// On a `SysCap`: read the machine's counters. /// - /// [`SYS_COUNTERS`] answers, per CPU, its own clock reading and how many - /// times firmware took it over, `MSR_SMI_COUNT` where the CPU counts that: + /// [`SYS_COUNTERS`] answers, per CPU, its own clock reading, how many + /// times firmware took it over, `MSR_SMI_COUNT` where the CPU counts that, + /// and the commands the kernel wrote to the firmware with the time they + /// held the CPU: /// what the machine is doing, not what a program on it does. The counters /// that time programs need [`TRACE`](Self::TRACE) beside this. /// `inspect` holds it, and `test-runner`. diff --git a/toyos-acpi/src/facs.rs b/toyos-acpi/src/facs.rs index 23f2ccf1f8b..4e85de87f8e 100644 --- a/toyos-acpi/src/facs.rs +++ b/toyos-acpi/src/facs.rs @@ -9,10 +9,13 @@ use crate::fadt::{FADT_FIRMWARE_CTRL, FADT_X_FIRMWARE_CTRL}; use crate::{Phys, Table, MAX_TABLE_LEN, SDT_REVISION}; -/// Table 5.13: `Length` at 4, the Global Lock at 16, in a structure of 64 -/// bytes or more; §5.2.10 aligns it on a 64-byte boundary. +/// Table 5.13: `Length` at 4, the Global Lock at 16 and `Flags` at 20, in a +/// structure of 64 bytes or more; §5.2.10 aligns it on a 64-byte boundary. const FACS_LENGTH: u64 = 4; pub const FACS_GLOBAL_LOCK: u64 = 16; +const FACS_FLAGS: u64 = 20; +/// Table 5.14, bit 0: "Indicates whether the platform supports S4BIOS_REQ." +const S4BIOS_F: u32 = 1 << 0; const FACS_MIN_LEN: u32 = 64; const FACS_ALIGN: u64 = 64; @@ -20,11 +23,13 @@ const FACS_ALIGN: u64 = 64; pub const PENDING: u32 = 1 << 0; pub const OWNED: u32 = 1 << 1; -/// Where the FACS is, and the length it declares. +/// Where the FACS is, the length it declares, and its `S4BIOS_F`. #[derive(Clone, Copy, PartialEq, Eq, Debug)] pub struct Facs { pub base: u64, pub len: u32, + /// The firmware gives the FADT's `S4BIOS_REQ` its meaning. + pub s4bios: bool, } /// Why a machine's FACS is none this decoder hands out. @@ -65,7 +70,7 @@ pub fn facs(phys: P, fadt: &Table

) -> Result { if len < FACS_MIN_LEN || len as usize > MAX_TABLE_LEN { return Err(FacsRefused::Length(len)); } - Ok(Facs { base, len }) + Ok(Facs { base, len, s4bios: crate::u32le(phys, base + FACS_FLAGS) & S4BIOS_F != 0 }) } /// §5.2.10.1's `AcquireGlobalLock`: the word to exchange `word` for, and diff --git a/toyos-acpi/src/fadt.rs b/toyos-acpi/src/fadt.rs index 62f8b81560b..f6cef2936c5 100644 --- a/toyos-acpi/src/fadt.rs +++ b/toyos-acpi/src/fadt.rs @@ -178,6 +178,8 @@ const FADT_SCI_INT: usize = 46; const FADT_SMI_CMD: usize = 48; const FADT_ACPI_ENABLE: usize = 52; const FADT_ACPI_DISABLE: usize = 53; +const FADT_S4BIOS_REQ: usize = 54; +const FADT_PSTATE_CNT: usize = 55; const FADT_PM1A_EVT_BLK: usize = 56; const FADT_PM1B_EVT_BLK: usize = 60; const FADT_PM1B_CNT_BLK: usize = 68; @@ -187,6 +189,7 @@ const FADT_PM1_EVT_LEN: usize = 88; const FADT_PM1_CNT_LEN: usize = 89; const FADT_GPE0_BLK_LEN: usize = 92; const FADT_GPE1_BLK_LEN: usize = 93; +const FADT_CST_CNT: usize = 95; const FADT_X_PM1A_EVT_BLK: usize = 148; const FADT_X_PM1B_EVT_BLK: usize = 160; const FADT_X_PM1A_CNT_BLK: usize = 172; @@ -214,23 +217,72 @@ pub enum PowerButton { ControlMethod, } -/// The way out of legacy mode and back into it: the port, and the value -/// written to it for each. +/// The way out of legacy mode and back into it: the value written to +/// `SMI_CMD` for each. #[derive(Clone, Copy, PartialEq, Eq, Debug)] pub struct LegacyMode { - pub smi_cmd: u16, pub acpi_enable: NonZeroU8, pub acpi_disable: NonZeroU8, } +/// `SMI_CMD`, and each value Table 5.9 gives a meaning written to it, as the +/// FADT holds them. The table writes `SMI_CMD` in six of its rows, the +/// port's own and these five, and names no other value for it. +#[derive(Clone, Copy, PartialEq, Eq, Debug)] +pub struct SmiCmd { + pub port: u16, + /// "The value to write to SMI_CMD to disable SMI ownership of the ACPI + /// hardware registers. [...] This field is reserved and must be zero on + /// systems that do not support Legacy Mode." + pub acpi_enable: u8, + /// "The value to write to SMI_CMD to re-enable SMI ownership of the ACPI + /// hardware registers. [...] This field is reserved and must be zero on + /// systems that do not support Legacy Mode." + pub acpi_disable: u8, + /// "The value to write to SMI_CMD to enter the S4BIOS state. [...] A + /// value of zero in S4BIOS_F indicates S4BIOS_REQ is not supported." The + /// FACS's flag says whether this names a value, and a zero here does not. + pub s4bios_req: u8, + /// "If non-zero, this field contains the value OSPM writes to the + /// SMI_CMD register to assume processor performance state control + /// responsibility." + pub pstate_cnt: u8, + /// "If non-zero, this field contains the value OSPM writes to the + /// SMI_CMD register to indicate OS support for the _CST object and C + /// States Changed notification." + pub cst_cnt: u8, +} + +impl SmiCmd { + /// `None` where `ACPI_ENABLE` or `ACPI_DISABLE` is zero: Table 5.9 + /// reserves each as zero on a machine without legacy mode, and one that + /// names a way in and no way back is not taken in. + pub fn legacy(&self) -> Option { + Some(LegacyMode { acpi_enable: NonZeroU8::new(self.acpi_enable)?, acpi_disable: NonZeroU8::new(self.acpi_disable)? }) + } + + /// The five values, each `None` where the tables name none: `S4BIOS_REQ` + /// where `s4bios`, the FACS's `S4BIOS_F`, is clear, whatever the field + /// holds, and each other where its field is zero. + pub fn named(&self, s4bios: bool) -> [Option; 5] { + let nonzero = |value: u8| (value != 0).then_some(value); + [ + nonzero(self.acpi_enable), + nonzero(self.acpi_disable), + s4bios.then_some(self.s4bios_req), + nonzero(self.pstate_cnt), + nonzero(self.cst_cnt), + ] + } +} + /// The fixed hardware an OS serves the SCI through, as the FADT names it. #[derive(Clone, Copy, PartialEq, Eq, Debug)] pub struct FixedHardware { pub sci_int: u16, - /// `None` where the FADT leaves `SMI_CMD`, `ACPI_ENABLE` or `ACPI_DISABLE` - /// zero: Table 5.9 reserves each as zero on a machine without legacy mode, - /// and one that names a way in and no way back is not taken in. - pub legacy: Option, + /// `None` where the FADT leaves `SMI_CMD` zero, as Table 5.9 has a + /// machine without System Management Mode leave it. + pub smi_cmd: Option, pub pm1a_event: toyos_abi::acpi::Block, /// [`toyos_abi::acpi::Block::NONE`] where the machine has no GPE0 block. pub gpe0: toyos_abi::acpi::Block, @@ -363,14 +415,20 @@ pub fn fixed_hardware(fadt: &Table

) -> Result Some(LegacyMode { smi_cmd, acpi_enable, acpi_disable }), - _ => None, + let smi_cmd = match u16::try_from(smi_cmd).map_err(|_| FixedRefused::SmiCmd(smi_cmd))? { + 0 => None, + port => Some(SmiCmd { + port, + acpi_enable: byte(FADT_ACPI_ENABLE)?, + acpi_disable: byte(FADT_ACPI_DISABLE)?, + s4bios_req: byte(FADT_S4BIOS_REQ)?, + pstate_cnt: byte(FADT_PSTATE_CNT)?, + cst_cnt: byte(FADT_CST_CNT)?, + }), }; Ok(FixedHardware { sci_int: fadt.u16_at(FADT_SCI_INT).ok_or(short)?, - legacy, + smi_cmd, pm1a_event: block(Field::Pm1aEvent, pm1_event, pm1_event_len)?, gpe0, power_button: if flags & PWR_BUTTON == 0 { PowerButton::Fixed } else { PowerButton::ControlMethod }, diff --git a/toyos-acpi/src/lib.rs b/toyos-acpi/src/lib.rs index 18ea34589f2..120e58501bb 100644 --- a/toyos-acpi/src/lib.rs +++ b/toyos-acpi/src/lib.rs @@ -29,7 +29,7 @@ pub use facs::{acquire, facs, release, Facs, FacsRefused, FACS_GLOBAL_LOCK, OWNE pub use fadt::{ century_of, dsdt_address, fixed_hardware, iapc_boot_arch, pm1a_control, psci, reset_register, rtc_century, - Century, Field, FixedHardware, FixedRefused, LegacyMode, PowerButton, Psci, Reset, CMOS_RAM, + Century, Field, FixedHardware, FixedRefused, LegacyMode, PowerButton, Psci, Reset, SmiCmd, CMOS_RAM, FADT_FOR_FIXED_HARDWARE, FADT_FOR_RESET, FADT_PM1A_CNT_BLK, }; use fadt::FADT_DSDT; diff --git a/toyos-acpi/tests/corpus.rs b/toyos-acpi/tests/corpus.rs index e8a98c78455..1c17895b268 100644 --- a/toyos-acpi/tests/corpus.rs +++ b/toyos-acpi/tests/corpus.rs @@ -14,7 +14,7 @@ use toyos_abi::acpi::Block; use toyos_acpi::{ definition_blocks, dsdt_address, ecam_base, ecdt, find_table, fixed_hardware, hpet_base, iapc_boot_arch, isa_line, madt_entries, memory_windows, pm1a_control, psci, reset_register, rtc_century, sci_line, Century, - EcRefused, Field, FixedRefused, LegacyMode, Line, MadtEntry, MadtHalt, Phys, Polarity, PowerButton, Psci, + EcRefused, Field, FixedRefused, LegacyMode, Line, MadtEntry, MadtHalt, Phys, Polarity, PowerButton, Psci, SmiCmd, Register, Reset, SourceOverride, Table, TableError, Trigger, ECDT_NEEDED, FADT_FOR_FIXED_HARDWARE, MADT_ENTRIES, MAX_TABLE_LEN, }; @@ -832,16 +832,60 @@ fn fixed_hardware_reads_whichever_form_names_a_block() { .expect("no GPE0 block"); assert_eq!(no_gpe.gpe0, Block::NONE); let command = |b: u8| core::num::NonZeroU8::new(b).expect("a command"); - let legacy = LegacyMode { smi_cmd: 0xb2, acpi_enable: command(2), acpi_disable: command(3) }; - assert_eq!(fixed(|_| {}).map(|f| f.legacy), Ok(Some(legacy))); - // No port, no way in, or a way in and no way back. - for zeroed in [48..52, 52..53, 53..54] { - assert_eq!(fixed(|t| t[zeroed.clone()].fill(0)).map(|f| f.legacy), Ok(None), "{zeroed:?}"); + let named = SmiCmd { port: 0xb2, acpi_enable: 2, acpi_disable: 3, s4bios_req: 0, pstate_cnt: 0, cst_cnt: 0 }; + assert_eq!(fixed(|_| {}).map(|f| f.smi_cmd), Ok(Some(named))); + assert_eq!(named.legacy(), Some(LegacyMode { acpi_enable: command(2), acpi_disable: command(3) })); + // No port is no `SMI_CMD`; a port with no way in, or a way in and no way + // back, is one with no legacy mode to leave. + assert_eq!(fixed(|t| t[48..52].fill(0)).map(|f| f.smi_cmd), Ok(None)); + for zeroed in [52, 53] { + let smi_cmd = fixed(|t| t[zeroed] = 0).expect("a FADT").smi_cmd.expect("a port"); + assert_eq!((smi_cmd.port, smi_cmd.legacy()), (0xb2, None), "{zeroed}"); } let method = fixed(|t| t[112] = 1 << 4).expect("a control-method button"); assert_eq!(method.power_button, PowerButton::ControlMethod); } +/// Every value Table 5.9 gives a meaning written to `SMI_CMD`, each from its +/// own byte, and the bytes either side of them read as none: `ACPI_ENABLE` +/// at 52, `ACPI_DISABLE` at 53, `S4BIOS_REQ` at 54, `PSTATE_CNT` at 55 and +/// `CST_CNT` at 95. A zero names no value in four of them, as the table says +/// of each; `S4BIOS_REQ` names its byte, zero too, exactly where the FACS's +/// `S4BIOS_F` is set. +#[test] +fn the_values_the_fadt_names_for_smi_cmd_are_each_read_from_their_own_byte() { + let all = fixed(|t| { + t[51..57].copy_from_slice(&[0, 0xa0, 0xa1, 0xa2, 0xa3, 0]); + t[94..97].copy_from_slice(&[0x77, 0xa4, 0x77]); + }) + .expect("a FADT naming all five") + .smi_cmd + .expect("a port"); + assert_eq!(all, SmiCmd { port: 0xb2, acpi_enable: 0xa0, acpi_disable: 0xa1, s4bios_req: 0xa2, pstate_cnt: 0xa3, cst_cnt: 0xa4 }); + assert_eq!(all.named(true), [Some(0xa0), Some(0xa1), Some(0xa2), Some(0xa3), Some(0xa4)]); + assert_eq!(all.named(false), [Some(0xa0), Some(0xa1), None, Some(0xa3), Some(0xa4)]); + for (at, named) in [(52, 0), (53, 1), (54, 2), (55, 3), (95, 4)] { + let one = fixed(|t| { + t[52..56].fill(0); + t[95] = 0; + t[at] = 0x5a; + }) + .expect("a FADT naming one") + .smi_cmd + .expect("a port"); + let mut want = [None; 5]; + want[named] = Some(0x5a); + // With the flag set `S4BIOS_REQ` names the byte it holds, a zero where it holds one. + let mut flagged = want; + flagged[2] = Some(if named == 2 { 0x5a } else { 0 }); + assert_eq!(one.named(true), flagged, "the byte at {at}, S4BIOS_F set"); + if named == 2 { + want[2] = None; + } + assert_eq!(one.named(false), want, "the byte at {at}, S4BIOS_F clear"); + } +} + /// A FADT before revision 2 has no `X_` fields, so the bytes past 116 are not /// read even where the table runs on. #[test] diff --git a/toyos-acpi/tests/facs.rs b/toyos-acpi/tests/facs.rs index 5b59db36e31..c495ac8cc9b 100644 --- a/toyos-acpi/tests/facs.rs +++ b/toyos-acpi/tests/facs.rs @@ -38,8 +38,8 @@ fn decoded(fadt: &[u8], at: u64, structure: &[u8]) -> Result fn the_wide_address_wins_where_it_is_not_zero_and_the_narrow_one_serves_where_it_is() { assert!(QEMU_FADT[8] >= 2 && QEMU_FADT.len() >= X_FIRMWARE_CTRL + 8, "the fixture is a revision with X_FIRMWARE_CTRL"); let structure = a_facs(64); - assert_eq!(decoded(&fadt_naming(0x1000, 0x2_0000_0040), 0x2_0000_0040, &structure), Ok(Facs { base: 0x2_0000_0040, len: 64 })); - assert_eq!(decoded(&fadt_naming(0x1000, 0), 0x1000, &structure), Ok(Facs { base: 0x1000, len: 64 })); + assert_eq!(decoded(&fadt_naming(0x1000, 0x2_0000_0040), 0x2_0000_0040, &structure), Ok(Facs { base: 0x2_0000_0040, len: 64, s4bios: false })); + assert_eq!(decoded(&fadt_naming(0x1000, 0), 0x1000, &structure), Ok(Facs { base: 0x1000, len: 64, s4bios: false })); // The narrow address is not read where the wide one names another place. assert_eq!(decoded(&fadt_naming(0x1000, 0x2_0000_0040), 0x1000, &structure), Err(FacsRefused::Unmapped(0x2_0000_0040))); assert_eq!(decoded(&fadt_naming(0, 0), 0x1000, &structure), Err(FacsRefused::Absent)); @@ -56,6 +56,24 @@ fn a_structure_that_is_no_facs_is_refused_by_name() { assert_eq!(decoded(&fadt, 0x1000, &a_facs(64)[..63]), Err(FacsRefused::Unmapped(0x1000))); } +/// Table 5.14: `S4BIOS_F` is bit 0 of the dword at 20, and no other bit of +/// the structure is read as it. +#[test] +fn s4bios_f_is_bit_zero_of_the_flags() { + let fadt = fadt_naming(0x1000, 0); + let flagged = |at: usize, byte: u8| { + let mut structure = a_facs(64); + structure[at] = byte; + decoded(&fadt, 0x1000, &structure).expect("a FACS").s4bios + }; + assert!(flagged(20, 1)); + assert!(flagged(20, 0xFF)); + assert!(!flagged(20, 0xFE), "the flags' other bits"); + for at in [16, 19, 21, 23, 24, 36] { + assert!(!flagged(at, 0xFF), "the byte at {at}"); + } +} + /// §5.2.10 aligns the FACS on a 64-byte boundary: every base that is not on /// one is refused, by either address field, and the boundary itself is not. #[test] @@ -67,7 +85,7 @@ fn a_facs_off_its_sixty_four_byte_boundary_is_refused() { assert_eq!(decoded(&fadt_naming(0x1000, wide), wide, &a_facs(64)), Err(FacsRefused::Misaligned(wide)), "wide, {off} off"); } for at in [0x40u32, 0x1000, 0x1040, 0xFFFF_FFC0] { - assert_eq!(decoded(&fadt_naming(at, 0), u64::from(at), &a_facs(64)), Ok(Facs { base: u64::from(at), len: 64 }), "{at:#x}"); + assert_eq!(decoded(&fadt_naming(at, 0), u64::from(at), &a_facs(64)), Ok(Facs { base: u64::from(at), len: 64, s4bios: false }), "{at:#x}"); } } diff --git a/toyos-acpi/tests/fixtures.rs b/toyos-acpi/tests/fixtures.rs index 1bac0168534..ac97851a0e0 100644 --- a/toyos-acpi/tests/fixtures.rs +++ b/toyos-acpi/tests/fixtures.rs @@ -3,15 +3,13 @@ mod common; -use core::num::NonZeroU8; - use common::{t14_root_bridge, Machine, OVMF_ROOT_BRIDGE}; use toyos_abi::boot::RootBridgeWindow; use toyos_abi::acpi::Block; use toyos_acpi::{ century_of, definition_blocks, dsdt_address, ecam_base, find_table, fixed_hardware, hpet_base, iapc_boot_arch, isa_line, madt_entries, memory_windows, pm1a_control, psci, reset_register, rtc_century, sci_line, - Century, FixedHardware, IoApicEntry, LegacyMode, Line, MadtEntry, Polarity, PowerButton, Psci, Reset, + Century, FixedHardware, IoApicEntry, Line, MadtEntry, Polarity, PowerButton, Psci, Reset, SmiCmd, SourceOverride, TableError, Trigger, FADT_FOR_FIXED_HARDWARE, FADT_PM1A_CNT_BLK, MADT_ENTRIES, }; @@ -220,11 +218,7 @@ fn the_q35_fadt_names_the_fixed_hardware_its_sci_is_served_through() { fixed_hardware(&fadt), Ok(FixedHardware { sci_int: 9, - legacy: Some(LegacyMode { - smi_cmd: 0xb2, - acpi_enable: NonZeroU8::new(0x02).expect("a command"), - acpi_disable: NonZeroU8::new(0x03).expect("a command"), - }), + smi_cmd: Some(SmiCmd { port: 0xb2, acpi_enable: 0x02, acpi_disable: 0x03, s4bios_req: 0, pstate_cnt: 0, cst_cnt: 0 }), pm1a_event: Block { port: 0x600, len: 4 }, gpe0: Block { port: 0x620, len: 16 }, power_button: PowerButton::Fixed, diff --git a/toyos-acpi/tests/thinkpad_t14.rs b/toyos-acpi/tests/thinkpad_t14.rs index 603bf434be5..7f05a10bdb7 100644 --- a/toyos-acpi/tests/thinkpad_t14.rs +++ b/toyos-acpi/tests/thinkpad_t14.rs @@ -9,13 +9,11 @@ mod common; -use core::num::NonZeroU8; - use common::{entry, madt, sdt, Machine}; use toyos_abi::acpi::Block; use toyos_acpi::{ - ecdt, fixed_hardware, madt_entries, pm1a_control, sci_line, Ec, FixedHardware, LegacyMode, Line, MadtEntry, - Polarity, PowerButton, SourceOverride, Table, Trigger, ECDT_NEEDED, FADT_FOR_FIXED_HARDWARE, + ecdt, fixed_hardware, madt_entries, pm1a_control, sci_line, Ec, FixedHardware, Line, MadtEntry, + Polarity, PowerButton, SmiCmd, SourceOverride, Table, Trigger, ECDT_NEEDED, FADT_FOR_FIXED_HARDWARE, MADT_ENTRIES, }; @@ -90,11 +88,7 @@ fn the_t14s_fadt_names_the_blocks_linux_served_its_sci_through() { fixed_hardware(&fadt), Ok(FixedHardware { sci_int: 9, - legacy: Some(LegacyMode { - smi_cmd: 0xb2, - acpi_enable: NonZeroU8::new(0xf0).expect("a command"), - acpi_disable: NonZeroU8::new(0xf1).expect("a command"), - }), + smi_cmd: Some(SmiCmd { port: 0xb2, acpi_enable: 0xf0, acpi_disable: 0xf1, s4bios_req: 0, pstate_cnt: 0, cst_cnt: 0 }), pm1a_event: Block { port: 0x1800, len: 4 }, gpe0: Block { port: 0x1860, len: 32 }, power_button: PowerButton::Fixed, diff --git a/toyos-inspect/src/kernel.rs b/toyos-inspect/src/kernel.rs index 47b8de70d8b..646fecaffbe 100644 --- a/toyos-inspect/src/kernel.rs +++ b/toyos-inspect/src/kernel.rs @@ -63,28 +63,32 @@ mod tests { Record { cpu, hardware_id: cpu * 2, stale, values } } - /// The T14's shape: every counter on every CPU. + /// The T14's shape: every counter on every CPU, and the firmware's + /// calls on the boot processor, which makes them. fn whole(cpu: u32) -> Record { - record(cpu, false, [Some(10), Some(4817), Some(100), Some(200), Some(3), Some(0x8000_2a04), Some(0x8000_ff01), Some(6)]) + let (calls, nanos) = if cpu == 0 { (Some(1), Some(15_968)) } else { (None, None) }; + record(cpu, false, [Some(10), Some(4817), Some(100), Some(200), Some(3), Some(0x8000_2a04), Some(0x8000_ff01), Some(6), calls, nanos]) } #[test] fn every_counter_a_record_carries_is_a_path_the_grammar_accepts() { let got = render(&(0..8).map(whole).collect::>()).unwrap(); - assert_eq!(got.len(), 8 * (2 + Counter::COUNT)); + assert_eq!(got.len(), 8 * Counter::COUNT + 2); for path in got.keys() { check_path(path).unwrap_or_else(|e| panic!("{path}: {e:?}")); } assert_eq!(got["kernel.cpu.7.smi"], Value::U64(4817)); assert_eq!(got["kernel.cpu.7.hardware_id"], Value::U64(14)); assert_eq!(got["kernel.cpu.7.stale"], Value::Bool(false)); + assert_eq!(got["kernel.cpu.0.firmware_calls"], Value::U64(1)); + assert!(!got.contains_key("kernel.cpu.7.firmware_calls")); } /// A machine whose CPU counts no SMIs, read on a capability without /// `TRACE`: only what the record carries, and no zero in place of the rest. #[test] fn an_absent_counter_has_no_path() { - let got = render(&[record(0, false, [Some(10), None, None, None, None, None, None, None])]).unwrap(); + let got = render(&[record(0, false, [Some(10), None, None, None, None, None, None, None, None, None])]).unwrap(); let paths: Vec<&str> = got.keys().map(String::as_str).collect(); assert_eq!(paths, vec!["kernel.cpu.0.hardware_id", "kernel.cpu.0.stale", "kernel.cpu.0.stamp"]); } diff --git a/toyos-userbound/src/firmware.rs b/toyos-userbound/src/firmware.rs index 830379db5fb..870ae111c53 100644 --- a/toyos-userbound/src/firmware.rs +++ b/toyos-userbound/src/firmware.rs @@ -52,6 +52,15 @@ //! ([`crate::port::Mediated`]), another claim's where a row names it, and //! passes otherwise. //! +//! **A byte written to the port that commands the firmware is no port write: +//! it is a call into the firmware, the kernel's to make** ([`FirmwareCall`]). +//! What it does there nothing here can bound; this bounds which byte and how +//! often. A byte the machine's tables give a meaning is the kernel's own +//! command and is refused, and of every other the kernel makes [`CALLS`] in +//! any [`CALL_PERIOD_NS`] ([`CallRate`]): firmware's AML retries a call its +//! handler has not answered, and each call stops every CPU of the machine +//! for as long as that handler takes. +//! //! **Configuration space is read and never written.** //! //! **The sleep type of the power-off is the holder's to supply and the @@ -63,7 +72,7 @@ use toyos_abi::acpi::{Refused, Width, UNLISTED}; use toyos_abi::boot::MemoryMapEntry; -use crate::port::{Mediated, IO_PORTS}; +use crate::port::{KeptCommands, Mediated, IO_PORTS}; use crate::span::PAGE_4K; /// `EfiReservedMemoryType`, `EfiRuntimeServicesData`, `EfiACPIReclaimMemory` @@ -334,22 +343,103 @@ impl PortAt { } } -/// Decide an access of `width` at `port`: every port it spans is asked of -/// `standing`. -pub fn port(standing: impl Fn(u16) -> Standing, port: u16, width: Width, write: bool) -> Result { +/// A command to the firmware the policy passed. +/// +/// ```compile_fail,E0451 +/// let _ = toyos_userbound::firmware::FirmwareCall { value: 0x10 }; +/// ``` +#[derive(Clone, Copy, PartialEq, Eq, Debug)] +pub struct FirmwareCall { + value: u8, +} + +impl FirmwareCall { + pub const fn value(&self) -> u8 { + self.value + } +} + +/// What a port access is. +#[derive(Clone, Copy, PartialEq, Eq, Debug)] +pub enum PortVerdict { + Through(PortAt), + /// A byte for the port that commands the firmware: the kernel's to + /// write, where and as often as it writes one. + FirmwareCall(FirmwareCall), + Refused(Refused), +} + +/// Decide an access of `width` at `port`, a write of `write`: every port it +/// spans is asked of `standing`. +pub fn port(standing: impl Fn(u16) -> Standing, port: u16, width: Width, write: Option) -> PortVerdict { + use PortVerdict::Refused as No; if width == Width::QWord || port as usize + width.bytes() as usize > IO_PORTS { - return Err(Refused::PortSpan); + return No(Refused::PortSpan); } + let mut commanded: Option = None; for port in port..=port + (width.bytes() as u16 - 1) { match standing(port) { Standing::Free | Standing::Declared(Mediated::Open) => {} - Standing::Declared(Mediated::ReadOnly) if !write => {} - Standing::Declared(Mediated::ReadOnly) => return Err(Refused::ReadOnlyPort), - Standing::Declared(Mediated::Kept) => return Err(Refused::KernelPort), - Standing::Row => return Err(Refused::ClaimedPort), + Standing::Declared(Mediated::ReadOnly | Mediated::Command(_)) if write.is_none() => {} + Standing::Declared(Mediated::ReadOnly) => return No(Refused::ReadOnlyPort), + Standing::Declared(Mediated::Command(kept)) => commanded = Some(kept), + Standing::Declared(Mediated::Kept) => return No(Refused::KernelPort), + Standing::Row => return No(Refused::ClaimedPort), + } + } + match (commanded, write) { + (Some(kept), Some(value)) => match u8::try_from(value) { + Ok(value) if width == Width::Byte => { + if kept.holds(value) { + No(Refused::KernelCommand) + } else { + PortVerdict::FirmwareCall(FirmwareCall { value }) + } + } + _ => No(Refused::CommandSpan), + }, + _ => PortVerdict::Through(PortAt { port, width }), + } +} + +/// The most commands to the firmware the kernel writes for the holder in any +/// [`CALL_PERIOD_NS`]. The kernel's own number, held against no measurement +/// of a machine's calls: the most one evaluation of the one machine's AML +/// that was read made is two, and its retry of an unanswered call is one a +/// millisecond. +pub const CALLS: usize = 8; +pub const CALL_PERIOD_NS: u64 = 1_000_000_000; + +/// When the last [`CALLS`] commands were admitted. +#[derive(Clone, Copy, PartialEq, Eq, Debug)] +pub struct CallRate { + admitted: [Option; CALLS], + /// The oldest of them, which the next admission replaces. + next: usize, +} + +impl Default for CallRate { + fn default() -> Self { + Self::new() + } +} + +impl CallRate { + pub const fn new() -> Self { + Self { admitted: [None; CALLS], next: 0 } + } + + /// Admit a command at `now`, nanoseconds on a clock that does not go + /// back; or refuse it, and keep nothing of it, where [`CALLS`] were + /// admitted less than [`CALL_PERIOD_NS`] before it. + pub fn admit(&mut self, now: u64) -> bool { + if self.admitted[self.next].is_some_and(|oldest| now.saturating_sub(oldest) < CALL_PERIOD_NS) { + return false; } + self.admitted[self.next] = Some(now); + self.next = (self.next + 1) % CALLS; + true } - Ok(PortAt { port, width }) } /// A configuration read the policy passed. diff --git a/toyos-userbound/src/lib.rs b/toyos-userbound/src/lib.rs index 76bb2aae490..f869c30e3be 100644 --- a/toyos-userbound/src/lib.rs +++ b/toyos-userbound/src/lib.rs @@ -38,7 +38,7 @@ pub mod span; pub use fault::Ring; pub use place::{PageSpan, Window}; -pub use port::{port_access, IoBitmap, Mediated, PortAccess, Ports, Reserved, Undeclared, IO_PORTS}; +pub use port::{port_access, IoBitmap, KeptCommands, Mediated, PortAccess, Ports, Reserved, Undeclared, IO_PORTS}; pub use segment::{pieces, segments, Pinned, Pins, Segment}; pub use span::{ align_2m_checked, in_user_half, is_user_addr, is_user_object, rebase_base, Access, Entry, diff --git a/toyos-userbound/src/port.rs b/toyos-userbound/src/port.rs index cd59b17f466..d66740e3736 100644 --- a/toyos-userbound/src/port.rs +++ b/toyos-userbound/src/port.rs @@ -119,6 +119,20 @@ pub enum Mediated { ReadOnly, /// Read and written for the holder. Open, + /// Read for the holder; a byte written is a command to the firmware, + /// which the kernel makes or refuses ([`crate::firmware::FirmwareCall`]). + Command(KeptCommands), +} + +/// The bytes a port that commands the firmware takes from the kernel alone: +/// those the machine's tables give a meaning, `None` where they name none. +#[derive(Clone, Copy, PartialEq, Eq, Debug)] +pub struct KeptCommands(pub [Option; 5]); + +impl KeptCommands { + pub fn holds(self, value: u8) -> bool { + self.0.contains(&Some(value)) + } } /// The ports no grant reaches, each run named by what holds it: the one diff --git a/toyos-userbound/tests/firmware.rs b/toyos-userbound/tests/firmware.rs index 8c2c7c35acc..aec6232f63a 100644 --- a/toyos-userbound/tests/firmware.rs +++ b/toyos-userbound/tests/firmware.rs @@ -4,8 +4,11 @@ use toyos_abi::acpi::{Refused, Width}; use toyos_abi::boot::MemoryMapEntry; -use toyos_userbound::firmware::{config, lock_word, port, sleep_type, type_word, Ecam, Function, Memory, MemoryVerdict, NoLockWord, Standing, FIXED_RANGE_END}; -use toyos_userbound::Mediated; +use toyos_userbound::firmware::{ + config, lock_word, port, sleep_type, type_word, CallRate, Ecam, Function, Memory, MemoryVerdict, NoLockWord, PortVerdict, Standing, CALLS, + CALL_PERIOD_NS, FIXED_RANGE_END, +}; +use toyos_userbound::{KeptCommands, Mediated}; const fn e(uefi_type: u32, start: u64, end: u64) -> MemoryMapEntry { MemoryMapEntry { uefi_type, start, end } @@ -489,50 +492,155 @@ fn the_lock_word_is_exchanged_only_where_all_four_bytes_are_the_firmwares_own() assert_eq!(lock_word(&[e(10, u64::MAX - 0xFFF, u64::MAX)], 4 * GIB, u64::MAX - 3), Err(NoLockWord::Type(None)), "the address space's last dword"); } -/// The kernel's declarations as q35 boots with them, and the i8042's row. +/// What a crafted FADT names for `SMI_CMD`: `ACPI_ENABLE`, `ACPI_DISABLE`, +/// `S4BIOS_REQ` and `CST_CNT`, and no `PSTATE_CNT`. +const KEPT: KeptCommands = KeptCommands([Some(0xF0), Some(0xF1), Some(0xF2), None, Some(0x85)]); + +/// The kernel's declarations as q35 boots with them, with [`KEPT`] for +/// `SMI_CMD`'s, and the i8042's row. fn standing(port: u16) -> Standing { match port { 0x3F8..=0x3FF | 0x20..=0x21 | 0xA0..=0xA1 | 0x70..=0x71 | 0xCF8 | 0xCFC..=0xCFF | 0xCF9 => Standing::Declared(Mediated::Kept), 0x80 => Standing::Declared(Mediated::Open), - 0xB2 | 0x604..=0x605 | 0x660..=0x67F => Standing::Declared(Mediated::ReadOnly), + 0xB2 => Standing::Declared(Mediated::Command(KEPT)), + 0x604..=0x605 | 0x660..=0x67F => Standing::Declared(Mediated::ReadOnly), 0x60 | 0x64 => Standing::Row, _ => Standing::Free, } } +const NO: PortVerdict = PortVerdict::Refused(Refused::KernelPort); + +fn refused_port(refused: Refused) -> PortVerdict { + PortVerdict::Refused(refused) +} + +fn through(verdict: PortVerdict) -> Option<(u16, Width)> { + match verdict { + PortVerdict::Through(witness) => Some((witness.port(), witness.width())), + _ => None, + } +} + #[test] fn a_port_answers_as_its_declaration_says() { - for write in [false, true] { + for write in [None, Some(0)] { for (at, width) in [(0x3F8, Width::Byte), (0x3FF, Width::Byte), (0x20, Width::Word), (0x70, Width::Byte), (0xCF8, Width::DWord), (0xCF9, Width::Byte)] { - assert_eq!(port(standing, at, width, write), Err(Refused::KernelPort), "{at:#x}"); + assert_eq!(port(standing, at, width, write), NO, "{at:#x}"); } - assert_eq!(port(standing, 0x60, Width::Byte, write), Err(Refused::ClaimedPort)); - assert_eq!(port(standing, 0x64, Width::Byte, write), Err(Refused::ClaimedPort)); + assert_eq!(port(standing, 0x60, Width::Byte, write), refused_port(Refused::ClaimedPort)); + assert_eq!(port(standing, 0x64, Width::Byte, write), refused_port(Refused::ClaimedPort)); // The POST port, and ports nothing declared. for (at, width) in [(0x80, Width::Byte), (0x72, Width::Word), (0x1800, Width::DWord), (0xFFFC, Width::DWord), (0xFFFF, Width::Byte)] { - let witness = port(standing, at, width, write).expect("a free port"); - assert_eq!((witness.port(), witness.width()), (at, width)); + assert_eq!(through(port(standing, at, width, write)), Some((at, width)), "a free port"); + } + } + // The PM1a control block and the TCO block: read, never written. + for (at, width) in [(0x604, Width::Word), (0x605, Width::Byte), (0x660, Width::DWord)] { + assert!(through(port(standing, at, width, None)).is_some(), "{at:#x} reads"); + assert_eq!(port(standing, at, width, Some(0)), refused_port(Refused::ReadOnlyPort), "{at:#x}"); + } +} + +/// `SMI_CMD` is read as a port is. A byte written to it is a call into the +/// firmware carrying that byte, for every byte but those the FADT gives a +/// meaning, zero among the callable where the tables name none with it, and +/// kept where they do, as a FACS whose `S4BIOS_F` is set over an +/// `S4BIOS_REQ` of zero does. +#[test] +fn a_byte_for_smi_cmd_is_a_firmware_call_unless_the_fadt_names_it() { + assert_eq!(through(port(standing, 0xB2, Width::Byte, None)), Some((0xB2, Width::Byte))); + for value in 0..=0xFFu64 { + let verdict = port(standing, 0xB2, Width::Byte, Some(value)); + if [0xF0, 0xF1, 0xF2, 0x85].contains(&value) { + assert_eq!(verdict, refused_port(Refused::KernelCommand), "{value:#04x}"); + } else { + let PortVerdict::FirmwareCall(call) = verdict else { panic!("{value:#04x} answered {verdict:?}") }; + assert_eq!(u64::from(call.value()), value); + } + } + // A FADT that names none keeps none. + let unnamed = |_| Standing::Declared(Mediated::Command(KeptCommands([None; 5]))); + for value in [0u64, 0xF0, 0xFF] { + assert!(matches!(port(unnamed, 0xB2, Width::Byte, Some(value)), PortVerdict::FirmwareCall(call) if u64::from(call.value()) == value)); + } + // And one that names zero keeps zero. + let zero = |_| Standing::Declared(Mediated::Command(KeptCommands([None, None, Some(0), None, None]))); + assert_eq!(port(zero, 0xB2, Width::Byte, Some(0)), refused_port(Refused::KernelCommand)); + assert!(matches!(port(zero, 0xB2, Width::Byte, Some(1)), PortVerdict::FirmwareCall(_))); +} + +/// A command is one byte to the one port: a wider write that reaches it, +/// from below or from it, is refused, whatever byte would land there, and so +/// is a value no byte holds. +#[test] +fn a_write_that_reaches_smi_cmd_and_is_no_byte_to_it_is_refused() { + for (at, width) in [(0xB2, Width::Word), (0xB1, Width::Word), (0xB2, Width::DWord), (0xAF, Width::DWord)] { + for value in [0u64, 0x10, 0xF1, 0xF100, 0x10_0000] { + assert_eq!(port(standing, at, width, Some(value)), refused_port(Refused::CommandSpan), "{width:?} at {at:#x}"); } + assert_eq!(through(port(standing, at, width, None)), Some((at, width)), "{width:?} at {at:#x} reads"); } - // SMI_CMD, the PM1a control block and the TCO block: read, never written. - for (at, width) in [(0xB2, Width::Byte), (0x604, Width::Word), (0x605, Width::Byte), (0x660, Width::DWord)] { - assert!(port(standing, at, width, false).is_ok(), "{at:#x} reads"); - assert_eq!(port(standing, at, width, true), Err(Refused::ReadOnlyPort), "{at:#x}"); + assert_eq!(port(standing, 0xB2, Width::Byte, Some(0x100)), refused_port(Refused::CommandSpan)); + // Beside it, a port nothing declared. + assert_eq!(through(port(standing, 0xB3, Width::Byte, Some(0xF1))), Some((0xB3, Width::Byte))); + assert_eq!(through(port(standing, 0xAE, Width::DWord, Some(0xF1))), Some((0xAE, Width::DWord))); +} + +/// [`CALLS`] are admitted however close together, and the next only once the +/// oldest of them is a whole period old: in no period, wherever it begins, +/// are there more. +#[test] +fn the_kernel_admits_a_fixed_few_firmware_calls_in_any_period() { + let mut rate = CallRate::new(); + // A storm: one a millisecond, as firmware's AML retries an unanswered call. + let admitted: Vec = (0..10_000u64).map(|ms| ms * 1_000_000).filter(|&now| rate.admit(now)).collect(); + assert_eq!(admitted.len(), 10 * CALLS, "ten seconds of a storm"); + for (i, &at) in admitted.iter().enumerate() { + let within = admitted[i..].iter().take_while(|&&later| later - at < CALL_PERIOD_NS).count(); + assert!(within <= CALLS, "{within} calls in the period from {at}"); } + + // The edge: all at one instant, then one nanosecond short of the period, then at it. + let mut rate = CallRate::new(); + let from = 5_000_000_000; + assert!((0..CALLS).all(|_| rate.admit(from))); + assert!(!rate.admit(from)); + assert!(!rate.admit(from + CALL_PERIOD_NS - 1)); + assert!(rate.admit(from + CALL_PERIOD_NS)); + // That one replaced the first of the eight, so the next waits on the second, which is as old. + assert!((1..CALLS).all(|_| rate.admit(from + CALL_PERIOD_NS))); + assert!(!rate.admit(from + CALL_PERIOD_NS)); + assert!(!rate.admit(from + 2 * CALL_PERIOD_NS - 1)); + + // A refused call is not kept: refusals do not push the next admission out. + let mut rate = CallRate::new(); + assert!((0..CALLS).all(|_| rate.admit(0))); + assert!((1..CALL_PERIOD_NS / 1_000_000).all(|ms| !rate.admit(ms * 1_000_000))); + assert!(rate.admit(CALL_PERIOD_NS)); + + // A clock that reads earlier than an admission admits nothing early. + let mut rate = CallRate::new(); + assert!((0..CALLS).all(|_| rate.admit(CALL_PERIOD_NS))); + assert!(!rate.admit(0)); + + // Spaced a period's eighth apart, every call is admitted. + let mut rate = CallRate::new(); + assert!((0..1000u64).all(|i| rate.admit(i * (CALL_PERIOD_NS / CALLS as u64)))); } #[test] fn a_wide_access_is_held_to_every_port_it_spans() { // A word that begins on a free port and ends on a kept one, and the same // into a row, a read-only run and out of the port space. - assert_eq!(port(standing, 0x3F7, Width::Word, false), Err(Refused::KernelPort)); - assert_eq!(port(standing, 0x1D, Width::DWord, false), Err(Refused::KernelPort)); - assert_eq!(port(standing, 0x5F, Width::Word, true), Err(Refused::ClaimedPort)); - assert_eq!(port(standing, 0xB1, Width::Word, true), Err(Refused::ReadOnlyPort)); - assert!(port(standing, 0xB1, Width::Word, false).is_ok()); - assert_eq!(port(standing, 0xFFFF, Width::Word, false), Err(Refused::PortSpan)); - assert_eq!(port(standing, 0xFFFD, Width::DWord, true), Err(Refused::PortSpan)); - assert_eq!(port(standing, 0x1800, Width::QWord, false), Err(Refused::PortSpan), "no port access is a qword"); + assert_eq!(port(standing, 0x3F7, Width::Word, None), NO); + assert_eq!(port(standing, 0x1D, Width::DWord, None), NO); + assert_eq!(port(standing, 0x5F, Width::Word, Some(0)), refused_port(Refused::ClaimedPort)); + assert_eq!(port(standing, 0x603, Width::Word, Some(0)), refused_port(Refused::ReadOnlyPort)); + assert!(through(port(standing, 0x603, Width::Word, None)).is_some()); + assert_eq!(port(standing, 0xFFFF, Width::Word, None), refused_port(Refused::PortSpan)); + assert_eq!(port(standing, 0xFFFD, Width::DWord, Some(0)), refused_port(Refused::PortSpan)); + assert_eq!(port(standing, 0x1800, Width::QWord, None), refused_port(Refused::PortSpan), "no port access is a qword"); } const HOST_BRIDGE: Function = Function { bus: 0, device: 0, function: 0 };