From 689e4e792b70b4fc75d70b2a45b17f7a7f09e69b Mon Sep 17 00:00:00 2001 From: japabu Date: Mon, 12 Oct 2026 20:53:11 +0200 Subject: [PATCH 1/3] issues: what the shipped OS takes from QEMU, and the track to run in any open-source hypervisor An audit of kernel/, loader/, system/, lib/ and sdk/ for every place the shipped OS depends on QEMU rather than on UEFI, ACPI, PCIe and virtio 1.2, with each other target hypervisor's devices read from its own documentation and source (Cloud Hypervisor, crosvm, VirtualBox, bhyve), not booted. Seven dependencies filed, one issue each, and the track that lists them with the five filed before and the per-hypervisor picture: - diskserver's rows name QEMU's NVMe by its PCI id; - the virtio drivers match only the non-transitional id, against virtio 1.2 section 4.1.2.2; - a virtio function that offers no ACCESS_PLATFORM is handed to a process as though the unit translated it, against virtio 1.2 section 6.1; - the x86 clock panics without an HPET table; - a hardware-reduced x86 FADT leaves no power-off and no power button; - virtio-iommu is not driven, so no claim stands under Cloud Hypervisor or crosvm, with the x86 interrupt-remapping half the owner's to rule; - the AP trampoline is written at 0x8000 without the memory map's word. No code changes. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_015xhaydXCVRNscoccHYFUWy --- ...ne-has-no-power-off-and-no-power-button.md | 42 ++++++ ...s-platform-is-handed-over-as-translated.md | 46 +++++++ ...kserver-claims-qemus-nvme-by-its-pci-id.md | 50 +++++++ ...ets-an-address-space-under-virtio-iommu.md | 49 +++++++ ...0-whatever-the-memory-map-says-is-there.md | 35 +++++ ...match-only-a-non-transitional-device-id.md | 57 ++++++++ ...ates-against-an-hpet-or-the-boot-panics.md | 43 ++++++ ...oyos-runs-in-any-open-source-hypervisor.md | 124 ++++++++++++++++++ 8 files changed, 446 insertions(+) create mode 100644 issues/a-hardware-reduced-x86-machine-has-no-power-off-and-no-power-button.md create mode 100644 issues/a-virtio-function-that-offers-no-access-platform-is-handed-over-as-translated.md create mode 100644 issues/diskserver-claims-qemus-nvme-by-its-pci-id.md create mode 100644 issues/no-claim-gets-an-address-space-under-virtio-iommu.md create mode 100644 issues/the-ap-trampoline-is-written-at-0x8000-whatever-the-memory-map-says-is-there.md create mode 100644 issues/the-virtio-drivers-match-only-a-non-transitional-device-id.md create mode 100644 issues/the-x86-clock-calibrates-against-an-hpet-or-the-boot-panics.md create mode 100644 issues/toyos-runs-in-any-open-source-hypervisor.md diff --git a/issues/a-hardware-reduced-x86-machine-has-no-power-off-and-no-power-button.md b/issues/a-hardware-reduced-x86-machine-has-no-power-off-and-no-power-button.md new file mode 100644 index 0000000000..ca2a6762ea --- /dev/null +++ b/issues/a-hardware-reduced-x86-machine-has-no-power-off-and-no-power-button.md @@ -0,0 +1,42 @@ +--- +status: open +kind: defect +opened: 2026-10-12 +--- + +# A hardware-reduced x86 machine has no power-off and no power button + +On x86-64 the kernel powers off only through the FADT's PM1a control block +(`kernel/src/arch/x86_64/power.rs`), with the `\_S5` sleep type the holder of +the `acpi` claim supplies, and that claim exists only over the fixed hardware: +`toyos_acpi::fixed_hardware` refuses a FADT with `HW_REDUCED_ACPI` set +(`lib/acpi/src/fadt.rs`), and `acpi_mode::init` +(`kernel/src/arch/x86_64/acpi_mode.rs`) then mints no row. So on such a machine +acpiserver gets no claim, nothing supplies S5, a shutdown is refused with +`no ACPI server supplied S5`, and a press of the power button reaches nobody. + +A hardware-reduced platform has neither PM1 block nor SMI_CMD. ACPI 6.5 gives +it the FADT's `SLEEP_CONTROL_REG` and `SLEEP_STATUS_REG` for sleep and +soft-off (§4.8.3.7, Sleep Control and Status Registers), and the Generic Event +Device, `ACPI0013`, for its events, the power button's among them: an +interrupt whose `_EVT` method the AML runs. Every AArch64 machine is one, which +this tree meets with PSCI (`kernel/src/arch/aarch64/power.rs`). + +Cloud Hypervisor's x86-64 FADT sets `HW_REDUCED_ACPI` and names the two sleep +registers and a reset register (`create_facp_table` in its `vmm/src/acpi.rs`); it +writes PM1a blocks only under its `legacy_acpi_pm1a` option, at ports it says +"conforming guests ignore". Its power button, which its API's +`/vm.power-button` presses, is a GED whose `_EVT` notifies `\_SB_.PWRB` +(its `devices/src/acpi.rs`). QEMU's `microvm` is hardware-reduced too. q35, +crosvm, VirtualBox and bhyve publish the fixed hardware. + +**Owner**: whoever holds +`issues/toyos-runs-the-machine-in-acpi-mode-and-interprets-its-aml.md`, whose +server holds the machine's power button and S5. + +**Exit**: a hardware-reduced FADT and DSDT, recorded from a Cloud Hypervisor +guest with their provenance as `lib/acpi/fixtures/` records QEMU's, decode in +a host test to the sleep control register and the GED's power-button method, +and acpiserver's host test runs that method to a press; and a guest boot on a +hardware-reduced machine, once the harness boots one, powers off on +`shutdown`. diff --git a/issues/a-virtio-function-that-offers-no-access-platform-is-handed-over-as-translated.md b/issues/a-virtio-function-that-offers-no-access-platform-is-handed-over-as-translated.md new file mode 100644 index 0000000000..1daa21dd7d --- /dev/null +++ b/issues/a-virtio-function-that-offers-no-access-platform-is-handed-over-as-translated.md @@ -0,0 +1,46 @@ +--- +status: open +kind: defect +opened: 2026-10-12 +--- + +# A virtio function that offers no `ACCESS_PLATFORM` is handed over as translated + +Virtio 1.2 §6, `VIRTIO_F_ACCESS_PLATFORM`: "If this feature bit is set to 0, +then the device has same access to memory addresses supplied to it as the +driver has. In particular, the device will always use physical addresses +matching addresses used by the driver ... When clear, this overrides any +platform-specific description of whether device access is limited or +translated in any way, e.g. whether an IOMMU may be present." And §6.1: "If +VIRTIO_F_ACCESS_PLATFORM is not offered, then a driver MUST pass only physical +addresses to the device." + +`pcidev` hands a process a function once the unit the firmware's tables put +it behind has given it a domain (`kernel/src/pcidev/mod.rs`), and never asks +whether a virtio function offers the bit; the holder's driver accepts it where +offered and goes on without it (`Offer::accept`, `lib/virtio/src/pci.rs`). A +function that does not offer it is not behind the unit at all: QEMU keeps it +on the machine's own address space whatever the `DMAR` says +(`hw/virtio/virtio-bus.c`, which `tests/common/iommu.rs` cites). So an honest +holder's descriptors, which carry domain addresses above all memory +(`kernel/src/iommu/window.rs`), reach nothing, and a hostile holder's, which +carry physical addresses, reach all of memory: the claim's isolation is void +for exactly the functions it is promised for. + +The launcher and the harness hide it. Each passes `iommu_platform=on` to every +virtio function wherever a unit exists (`build/qemu.rs`), and +`iommu_virtio_platform` reds a profile that does not; QEMU's own default is +off. A transitional virtio function cannot offer the bit in QEMU at all, and +VirtualBox's virtio core offers it nowhere +(`issues/the-virtio-drivers-match-only-a-non-transitional-device-id.md`). + +The check cannot be the holder's: the holder is who the domain protects the +machine from. + +**Owner**: whoever holds `issues/toyos-runs-in-any-open-source-hypervisor.md`. + +**Exit**: a claim on a virtio function that does not offer +`VIRTIO_F_ACCESS_PLATFORM` behind a unit is refused by a `Refusal` of its own; +a guest boot under the harness's VT-d with netstack's virtio-net created +without `iommu_platform=on` reads that refusal on its `pcidev: … NOT HANDED +OVER —` line, and is handed over with the check deleted. diff --git a/issues/diskserver-claims-qemus-nvme-by-its-pci-id.md b/issues/diskserver-claims-qemus-nvme-by-its-pci-id.md new file mode 100644 index 0000000000..41bb4961ff --- /dev/null +++ b/issues/diskserver-claims-qemus-nvme-by-its-pci-id.md @@ -0,0 +1,50 @@ +--- +status: open +kind: defect +opened: 2026-10-12 +--- + +# diskserver claims QEMU's NVMe by its PCI id and no other controller + +Both shipped boot configs give diskserver one device, `pci:1b36:0010` +(`images/system.toml`, `images/console.toml`): the vendor and device id of +QEMU's own `nvme` model (`hw/nvme/ctrl.c`, `PCI_DEVICE_ID_REDHAT_NVME` under +Red Hat's vendor id). A claim matches the id exactly (`pcidev::claim`, +`kernel/src/pcidev/mod.rs`), so on any other NVMe controller the supervisor +endows diskserver nothing, it says `no NVMe controller this row names is on +this machine; serving no partition`, and DATA, LOG and BOOT are absent unless +the image is on a USB stick the kernel drives. The driver is not QEMU's: +`Controller::open` (`system/diskserver/src/nvme.rs`) takes any function whose +class code is `01:08:02`. + +Read from each hypervisor's source, not booted: + +| | NVMe id | MSI-X table | +|---|---|---| +| QEMU `nvme` | `1b36:0010`, or `8086:5845` with `use-intel-id=on` | BAR 0, unless `msix-exclusive-bar=on` | +| VirtualBox, `src/VBox/Devices/Storage/DevNVMe.cpp` | `80ee:4e56` | BAR 5 | +| bhyve, `usr.sbin/bhyve/pci_nvme.c` | `fb5d:0a0a` | BAR 4 | + +Cloud Hypervisor and crosvm offer no NVMe. Every controller on metal carries +its maker's id. + +The identity every NVMe controller shares is its class code: base class 01h, +subclass 08h, programming interface 02h, the "NVM Express I/O controller" of +the PCI-SIG's *PCI Code and ID Assignment Specification*, which QEMU, VirtualBox +and bhyve each set. `issues/the-lan-is-not-yet-production-grade.md` plans the +same for NICs ("init binds drivers to hardware from a declared match table by +vendor, device and class"). + +Two constraints a class row meets that an id row does not: a machine with two +NVMe controllers matches twice, which `pcidev::claim` refuses as `Ambiguous` +today; and a machine whose NVMe holds another operating system, as the T14's +holds Ubuntu, is handed to diskserver, which then serves only the partitions +its table and grants name (`system/diskserver/src/main.rs`). + +**Owner**: whoever holds `issues/toyos-runs-in-any-open-source-hypervisor.md`. + +**Exit**: a `devices` row names a PCI function by class code, which +`toyos_abi::syscall::DeviceRequest::parse` takes and refuses malformed; both +shipped configs name NVMe that way; and a guest boot whose controller is +QEMU's `nvme` with `use-intel-id=on` serves DATA through diskserver, which a +row naming `1b36:0010` instead reds. diff --git a/issues/no-claim-gets-an-address-space-under-virtio-iommu.md b/issues/no-claim-gets-an-address-space-under-virtio-iommu.md new file mode 100644 index 0000000000..337828bb2c --- /dev/null +++ b/issues/no-claim-gets-an-address-space-under-virtio-iommu.md @@ -0,0 +1,49 @@ +--- +status: open +kind: defect +opened: 2026-10-12 +--- + +# No claim gets an address space under virtio-iommu + +A process is handed a PCI function only once the function has an address +space of its own at a unit and, on x86-64, interrupt remapping +(`kernel/src/pcidev/mod.rs`'s header; `Refusal::NotRemapped`, +`Refusal::Untranslated`). The units this kernel programs are VT-d (`DMAR`), +AMD-Vi (`IVRS`) and SMMUv3 (`IORT`) (`kernel/src/iommu/`). QEMU emulates all +three, and VirtualBox VT-d or AMD-Vi under `--iommu` on its ICH9 chipset, off +by default (`VBoxManage modifyvm`). Cloud Hypervisor and crosvm put their +devices behind virtio-iommu instead (virtio 1.2 §5.13, device id 23, PCI +`1af4:1057`): Cloud Hypervisor's documentation says it implements it "in order +to provide a virtual IOMMU to its users" and "will not try to implement a full +emulation of a physical IOMMU" (its `docs/iommu.md`), describes it to the +guest in an ACPI `VIOT` table (its `vmm/src/acpi.rs`), and offers an SMMUv3 +only for a passthrough device behind a physical one (`docs/iommu.md`, +"Emulated ARM SMMUv3"). So under either, every claim is refused and no +userland driver runs. + +The two architectures differ in what virtio-iommu leaves undone: + +- **AArch64.** A claimed function's message is its slot's LPI through the + ITS (`kernel/src/arch/aarch64/irqchip/its.rs`), whose translation by + DeviceID is the per-function boundary; virtio-iommu need only translate DMA. + Cloud Hypervisor's AArch64 machine has an ITS (its `IORT`'s ITS group). +- **x86-64.** Virtio 1.2 defines no interrupt remapping: an MSI doorbell is a + reserved region the driver may not map (§5.13.6.8.1, + `VIRTIO_IOMMU_RESV_MEM_T_MSI`), whose writes pass untranslated. Nothing mints + a `Remapping` there, so a claim still ends at `NotRemapped`. Whether a claim + may stand under virtio-iommu with its messages unremapped is the owner's to + rule: `issues/every-driver-is-still-in-the-kernel.md` records his ruling that + "the IOMMU lands first and completely, interrupt remapping included". + +bhyve offers its guest no IOMMU of any kind; +`issues/a-machine-without-an-iommu-refuses-every-claim.md` holds that case and +the owner's ruling on it. + +**Owner**: whoever holds `issues/toyos-runs-in-any-open-source-hypervisor.md`. + +**Exit**: on AArch64, a claim under virtio-iommu is handed over in a domain +of its own and a DMA outside its grants is a `DMA FAULT` record, under QEMU's +`virt` with `virtio-iommu-pci` and its emulated ITS; on x86-64, the owner's +ruling recorded here, and the claim refused by a `Refusal` that names +virtio-iommu's missing interrupt remapping or handed over as he rules. diff --git a/issues/the-ap-trampoline-is-written-at-0x8000-whatever-the-memory-map-says-is-there.md b/issues/the-ap-trampoline-is-written-at-0x8000-whatever-the-memory-map-says-is-there.md new file mode 100644 index 0000000000..7da2448122 --- /dev/null +++ b/issues/the-ap-trampoline-is-written-at-0x8000-whatever-the-memory-map-says-is-there.md @@ -0,0 +1,35 @@ +--- +status: open +kind: defect +opened: 2026-10-12 +--- + +# The AP trampoline is written at 0x8000 whatever the memory map says is there + +`kernel/src/arch/x86_64/smp.rs` copies the application processors' start-up +code to physical page `0x8000` (`TRAMPOLINE_PAGE`) and starts every AP there. +`arch::boot::reserved` withholds the page from the allocator, and +`kernel_main` (`kernel/src/main.rs`) checks every region it withholds against +the firmware's map except this one: "The architecture's own page is not a +loader allocation, so it is named here". Nothing asks whether the page is +memory at all, nor whose. + +It holds on QEMU's q35, whose edk2 hands over `0x0..0x87000` as conventional +memory (the map recorded in `lib/bootmap/tests/direct_map.rs`), and the T14 +starts its CPUs through it; how the T14's map types the page was not read. It +is a fixed address chosen for one layout: a firmware that keeps anything of +its own at `0x8000` — runtime code, ACPI NVS, a reserved range — has it +overwritten when the first AP starts, though the map the kernel was handed +would have said so. No other hypervisor's map was read for it. + +What the architecture needs is a page below 1 MiB, its number the `STARTUP` +IPI's vector (Intel SDM Vol. 3A, "MP Initialization Protocol Algorithm"), and +UEFI lets the loader ask for exactly that before `ExitBootServices`: +`AllocatePages` with `AllocateMaxAddress` (UEFI 2.10 §7.2), the page then +`LoaderData` like every other region `kernel_main` checks. + +**Owner**: whoever holds `issues/toyos-runs-in-any-open-source-hypervisor.md`. + +**Exit**: the trampoline's page is one the loader allocated below 1 MiB and +hands over in `KernelArgs`, held by the check `kernel_main` makes of every +other withheld region; deleting the allocation reds that check at boot. diff --git a/issues/the-virtio-drivers-match-only-a-non-transitional-device-id.md b/issues/the-virtio-drivers-match-only-a-non-transitional-device-id.md new file mode 100644 index 0000000000..438816d5dc --- /dev/null +++ b/issues/the-virtio-drivers-match-only-a-non-transitional-device-id.md @@ -0,0 +1,57 @@ +--- +status: open +kind: defect +opened: 2026-10-12 +--- + +# The virtio drivers match only a non-transitional device id + +Virtio 1.2 §4.1.2.2: "Drivers for device types listed in section 4.1.2 MUST +match devices with the PCI Vendor ID 0x1AF4 and the Transitional PCI Device ID +indicated in section 4.1.2" — `0x1000` for a network card, `0x1003` for a +console, among others — beside the `0x1040`-based id. A transitional device +also publishes the virtio PCI capabilities, and §4.1.4 has a non-transitional +driver drive it through them, and "fail gracefully" where they are absent. That +is not the legacy interface, which this tree refuses. + +The tree matches the `0x1040`-based id alone: + +- netstack's `CARDS` (`system/netstack/src/main.rs`) and its row in + `images/system.toml`, `pci:1af4:1041`; +- the kernel's console, `VIRTIO_CONSOLE_DEVICE` `0x1043` + (`kernel/src/drivers/virtio_console.rs`). + +The launcher and the harness hide it: each asks QEMU for +`virtio-net-pci-non-transitional` and `virtio-serial-pci-non-transitional` +(`build/qemu.rs`, `tests/common/qemu.rs`). QEMU's own default is transitional +on q35's root bus — `disable-legacy=auto` resolves to off for a device not +behind a PCIe port (`virtio_pci_realize`, `hw/virtio/virtio-pci.c`) — and +VirtualBox's virtio-net is `1af4:1000` (`PCI_DEVICE_ID_VIRTIONET_HOST`, +`src/VBox/Devices/Network/DevVirtioNet.cpp`). On either, netstack has no card +and the kernel no virtio console. Cloud Hypervisor and crosvm use the +`0x1040`-based ids (Cloud Hypervisor's +`virtio-devices/src/transport/pci_device.rs`, crosvm's +`devices/src/virtio/virtio_pci_device.rs`). bhyve's virtio-net and +virtio-blk are legacy-only (its `usr.sbin/bhyve/virtio.c` allocates an I/O BAR +and no virtio capability), so a conforming driver matches them and refuses +them by name. +Virtio-gpu and virtio-sound have no transitional id, so their matches are +already whole. + +A transitional device offers no `VIRTIO_F_ACCESS_PLATFORM` on either: QEMU +refuses the pair ("VIRTIO_F_IOMMU_PLATFORM was supported by neither legacy nor +transitional device", `hw/virtio/virtio-pci.c`), and VirtualBox's virtio core +implements none (`VIRTIO_F_ACCESS_PLATFORM`, "VirtIO 1.1 NYI", +`src/VBox/Devices/VirtIO/VirtioCore.h`). So behind a unit such a NIC is one no +claim may take +(`issues/a-virtio-function-that-offers-no-access-platform-is-handed-over-as-translated.md`); +the matches reach a card only on the path +`issues/a-machine-without-an-iommu-refuses-every-claim.md` rules for a machine +with no unit, and the kernel's console on every machine. + +**Owner**: whoever holds `issues/toyos-runs-in-any-open-source-hypervisor.md`. + +**Exit**: netstack's row, its `CARDS` and the kernel's console each name both +ids of their type; the kernel's console carries a guest boot whose console is +QEMU's transitional `virtio-serial-pci`, and refuses the same device under +`disable-modern=on` by name. diff --git a/issues/the-x86-clock-calibrates-against-an-hpet-or-the-boot-panics.md b/issues/the-x86-clock-calibrates-against-an-hpet-or-the-boot-panics.md new file mode 100644 index 0000000000..0b5e80e4f4 --- /dev/null +++ b/issues/the-x86-clock-calibrates-against-an-hpet-or-the-boot-panics.md @@ -0,0 +1,43 @@ +--- +status: open +kind: defect +opened: 2026-10-12 +--- + +# The x86 clock calibrates against an HPET, or the boot panics + +`arch::boot::clock` (`kernel/src/arch/x86_64/boot.rs`) takes the HPET's base +from its ACPI table with `.expect("ACPI: HPET not found")`, and every rate the +kernel keeps descends from that one measurement: the TSC's +(`kernel/src/arch/x86_64/hpet.rs`) and the local APIC timer's, calibrated +against the TSC (`apic::init_timer`). CPUID 15H/16H is read after it, as a +cross-check only. A machine whose firmware publishes no `HPET` table stops +there, before a timer ticks, and before PCI is enumerated, so a console that +is a virtio function never carries the panic. + +QEMU's q35 has one. Of the other targets, read from their sources and not +booted: + +| | HPET table | ACPI PM timer | TSC rate the platform states | +|---|---|---|---| +| Cloud Hypervisor | none (`vmm/src/acpi.rs` builds no `HPET`) | `X_PM_TMR_BLK` | CPUID `0x40000010` EAX, in kHz, where the host's TSC is invariant (`arch/src/x86_64/mod.rs`) | +| crosvm | none (no HPET device in `devices/src`) | none ("not supported", `x86_64/src/acpi.rs`) | CPUID 15H, where a calibrated leaf is configured (`x86_64/src/cpuid.rs`) | +| VirtualBox | only with `--x86-hpet on`, default off (`HpetEnabled`, `DevACPI.cpp`) | yes | not read | +| bhyve | yes (`usr.sbin/bhyve/acpi.c`) | yes | not read | + +So Cloud Hypervisor, crosvm and VirtualBox without `--x86-hpet on` all end at +this `expect`. + +What a machine states instead, by its own documents: the ACPI PM timer, fixed +hardware at 3.579545 MHz (ACPI 6.5 §4.8.3.3), which q35, Cloud Hypervisor, +VirtualBox and bhyve publish; CPUID 15H's crystal ratio (Intel SDM Vol. 2A), +which the kernel already reads through `toyos_tsc::stated_hz` and which no AMD +part states (`issues/the-loaders-lines-carry-no-time-on-an-x86-cpu-that-states-no-tsc-rate.md`); +and a hypervisor's timing leaf `0x40000010`. crosvm without a configured leaf +states none of them. + +**Owner**: whoever holds `issues/toyos-runs-in-any-open-source-hypervisor.md`. + +**Exit**: a q35 boot with `-machine hpet=off` reaches `Boot: complete`, its +`TSC:` line naming the source its rate came from; and a machine that states no +source is refused by a line that names each one it looked for. diff --git a/issues/toyos-runs-in-any-open-source-hypervisor.md b/issues/toyos-runs-in-any-open-source-hypervisor.md new file mode 100644 index 0000000000..c5519a1a02 --- /dev/null +++ b/issues/toyos-runs-in-any-open-source-hypervisor.md @@ -0,0 +1,124 @@ +--- +status: open +kind: track +opened: 2026-10-12 +--- + +# ToyOS runs in any open-source hypervisor + +The owner's aim, as the orchestrator briefed this audit: ToyOS runs in any +popular open-source virtual machine — QEMU under KVM, HVF or TCG, Cloud +Hypervisor, VirtualBox, bhyve and crosvm — with QEMU still the launcher and the +test runner; under virtualization it rests on UEFI, ACPI, PCIe and virtio 1.2 +and the real devices it already drives (NVMe, xHCI and USB HID, the Intel NICs, +the i8042), and nothing it ships depends on a QEMU-only device or behaviour. + +Nothing in this tree records a boot of this image under any hypervisor but +QEMU. What this file says of the others is read from their documentation and +source at their default branches, not measured. Cloud Hypervisor and crosvm run +on Linux hosts and bhyve on FreeBSD, none of them on the macOS development +host. + +## What the shipped OS takes from QEMU + +The kernel, the loader and every program drive no fw_cfg, ramfb, pvpanic, +isa-debug-exit, debugcon or Bochs display, and test for no hypervisor +signature. What they do depend on, each filed: + +- `issues/diskserver-claims-qemus-nvme-by-its-pci-id.md` +- `issues/the-virtio-drivers-match-only-a-non-transitional-device-id.md` +- `issues/a-virtio-function-that-offers-no-access-platform-is-handed-over-as-translated.md` +- `issues/the-x86-clock-calibrates-against-an-hpet-or-the-boot-panics.md` +- `issues/a-hardware-reduced-x86-machine-has-no-power-off-and-no-power-button.md` +- `issues/no-claim-gets-an-address-space-under-virtio-iommu.md` +- `issues/the-ap-trampoline-is-written-at-0x8000-whatever-the-memory-map-says-is-there.md` + +and, filed before this track: + +- `issues/a-machine-without-an-iommu-refuses-every-claim.md` — bhyve, and + VirtualBox and QEMU by default. +- `issues/a-controller-whose-msix-table-is-in-bar-0-cannot-be-driven-from-userland.md` + — QEMU's NVMe by default; and every virtio function of Cloud Hypervisor and + crosvm, which keep the MSI-X table and PBA in the BAR their virtio + structures are in (Cloud Hypervisor's + `virtio-devices/src/transport/pci_device.rs`, crosvm's + `devices/src/virtio/virtio_pci_device.rs`). +- `issues/a-hid-interface-outside-the-boot-subclass-is-read-as-qemus-tablet.md` + — VirtualBox's USB tablet, a subclass-0 interface (its + `src/VBox/Devices/Input/UsbMouse.cpp`); bhyve's declares the boot mouse + subclass and protocol (its `usr.sbin/bhyve/usb_mouse.c`). +- `issues/getsockname-answers-an-address-nobody-asked-netstack-for.md` — + QEMU's user-mode network address in `sdk/libc`. +- `issues/pci-is-enumerated-on-segment-group-0-alone.md` — Cloud + Hypervisor's `num_pci_segments`. + +Three of the launcher's QEMU options keep three of these off every guest +test: the `-non-transitional` virtio devices the transitional id, +`iommu_platform=on` the missing `ACCESS_PLATFORM`, and `msix-exclusive-bar=on` +the MSI-X table (`build/qemu.rs`, `tests/common/qemu.rs`). The rest of what is +QEMU's there — the xHCI models, the boot stick, `-vga std`, `ramfb` under +AArch64's edk2, `caching-mode=on`, `kernel-irqchip`, `iommu-testdev`, QMP — is +the harness's alone. + +Adaptations that degrade nothing: the TCO table's q35 row (`lib/tco`, armed +only by the `watchdog` parameter and refused by name elsewhere), netstack's +82574L row for QEMU's `e1000e`, the hard-lockup bound's path for a machine +with no PMU, `toyos-cpuvuln`'s reading of CPUID's hypervisor bit as Linux +reads it, and the loader's read of PI's DXE services (`loader/src/gcd.rs`), +which hands over the root bridges' windows alone where firmware has none. + +## Each hypervisor, x86-64 + +| | Firmware | Disk | Display | Network | Input | Platform | IOMMU | +|---|---|---|---|---|---|---|---| +| QEMU q35 | edk2 | NVMe `1b36:0010`, virtio-blk, AHCI, USB | std-vga GOP, virtio-gpu | virtio-net (transitional by default), e1000e | i8042, USB HID, virtio-input | HPET, full ACPI | VT-d, AMD-Vi, virtio-iommu | +| Cloud Hypervisor | `CLOUDHV.fd` (edk2) | virtio-blk | none | virtio-net | none | no HPET; hardware-reduced ACPI; I/O APIC only with the serial port | virtio-iommu | +| crosvm | a `--bios` image; which UEFI boots under it was not read | virtio-blk, virtio-scsi | virtio-gpu | virtio-net | virtio-input | no HPET, no PM timer | virtio-iommu | +| VirtualBox 7 | EFI with `--firmware efi`; PCIe and MSI only with `--chipset ich9` | NVMe `80ee:4e56`, AHCI, USB, virtio-scsi | VMSVGA GOP | 8254x e1000s, virtio-net `1af4:1000` | i8042, USB HID | HPET off by default | VT-d or AMD-Vi with `--iommu`, off by default | +| bhyve | `BHYVE_UEFI.fd` (edk2) | NVMe `fb5d:0a0a`, AHCI, legacy virtio-blk | `fbuf` GOP | 82545EM, legacy virtio-net | i8042, xHCI tablet | HPET, full ACPI; HDA on INTx only | none | + +First stop, on paper. VirtualBox's defaults, BIOS and PIIX3, boot nothing of +this image and enumerate no PCIe; with `--firmware efi` and `--chipset ich9` +it stops, like Cloud Hypervisor and crosvm, at the HPET `expect`, which on +Cloud Hypervisor no console carries unless its serial port is on. VirtualBox +with `--x86-hpet on` too, and bhyve: a desktop on a GOP panel with i8042 or +USB input, no disk, no network, and on bhyve an HDA the kernel arms no +interrupt on. + +Once the dependencies close, each still lacks a driver this tree does not +have: virtio-blk on Cloud Hypervisor and crosvm, whose only disk it is; +virtio-input on crosvm, whose only input it is; and on bhyve, whose virtio is +legacy, a driver for its 82545EM, its only other NIC, and an interrupt for its +HDA, which raises INTx alone where the kernel's driver arms MSI-X or MSI +(`kernel/src/drivers/hda.rs`, `arm_interrupt`). VirtualBox lacks none: its +NVMe, GOP, i8042, xHCI and HDA are driven here, and its virtio-net once its +transitional id is. + +## AArch64 + +Outside `issues/toyos-runs-on-arm64.md`'s ruled target, QEMU `virt` on the +development Mac, and read only: + +- **Cloud Hypervisor**: `CLOUDHV_EFI.fd`, GICv3 with an ITS, the PL011 in the + SPCR, virtio-iommu, no display. +- **VirtualBox 7**: UEFI always, virtio-scsi its only disk, its GIC's ITS off + by default (`--arm-gic-its`). +- **bhyve**: U-Boot as firmware (`bhyve(8)`); whether it publishes an ACPI + RSDP, without which the loader stops, was not read. +- **crosvm**: not read. + +## Exit + +The seven dependencies this file lists, the five filed before them and the +drivers each hypervisor still lacks closed; and the same image, built by +`cargo run -- --build-only`, booted by hand once under each of Cloud +Hypervisor, crosvm, VirtualBox and bhyve on x86-64 to `Boot: complete` with +DATA served and a DHCP lease, each boot's log on the evidence repository under +the hypervisor's version and command line. + +**Blocked on**: the owner's ruling on interrupt remapping under virtio-iommu +(`issues/no-claim-gets-an-address-space-under-virtio-iommu.md`) and the +signed-driver design `issues/a-machine-without-an-iommu-refuses-every-claim.md` +owes, and a host for each hypervisor. + +**Owner**: the orchestrator; work starts on the owner's go. From 6ef728cfb55550e3a52fd41cdb2b03ba09a1982c Mon Sep 17 00:00:00 2001 From: japabu Date: Tue, 13 Oct 2026 02:12:31 +0200 Subject: [PATCH 2/3] issues: the review round 1 fixes, in progress Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_015xhaydXCVRNscoccHYFUWy --- ...s-platform-is-handed-over-as-translated.md | 22 ++++-- ...ets-an-address-space-under-virtio-iommu.md | 67 +++++++++++++------ ...match-only-a-non-transitional-device-id.md | 6 +- ...ates-against-an-hpet-or-the-boot-panics.md | 16 +++-- 4 files changed, 75 insertions(+), 36 deletions(-) diff --git a/issues/a-virtio-function-that-offers-no-access-platform-is-handed-over-as-translated.md b/issues/a-virtio-function-that-offers-no-access-platform-is-handed-over-as-translated.md index 1daa21dd7d..91c3997b99 100644 --- a/issues/a-virtio-function-that-offers-no-access-platform-is-handed-over-as-translated.md +++ b/issues/a-virtio-function-that-offers-no-access-platform-is-handed-over-as-translated.md @@ -34,13 +34,21 @@ off. A transitional virtio function cannot offer the bit in QEMU at all, and VirtualBox's virtio core offers it nowhere (`issues/the-virtio-drivers-match-only-a-non-transitional-device-id.md`). -The check cannot be the holder's: the holder is who the domain protects the -machine from. +The check is not the holder's, for the holder is who the domain protects the +machine from, nor the kernel's, which the owner's target keeps free of virtio +(`issues/every-driver-is-still-in-the-kernel.md`; `kernel/src/pcidev` and +`kernel/src/iommu` name none today). By the orchestrator's ruling (#896, +issuecomment-6131213771, ruling 1) it is the supervisor's, which mints the +claim and is trusted: it reads the function's offered features before it moves +the claim in, and on a machine whose functions are translated refuses a virtio +function that does not offer the bit. How it reads the word is the stage's +design. **Owner**: whoever holds `issues/toyos-runs-in-any-open-source-hypervisor.md`. -**Exit**: a claim on a virtio function that does not offer -`VIRTIO_F_ACCESS_PLATFORM` behind a unit is refused by a `Refusal` of its own; -a guest boot under the harness's VT-d with netstack's virtio-net created -without `iommu_platform=on` reads that refusal on its `pcidev: … NOT HANDED -OVER —` line, and is handed over with the check deleted. +**Exit**: a guest boot under the harness's VT-d with netstack's virtio-net +created without `iommu_platform=on` reads the supervisor's line refusing +netstack that function for its missing `VIRTIO_F_ACCESS_PLATFORM`, and +`netstack: no NIC on this machine`; the same boot with the check deleted +endows netstack the function; and `git grep -i virtio kernel/src/pcidev +kernel/src/iommu` still finds nothing. diff --git a/issues/no-claim-gets-an-address-space-under-virtio-iommu.md b/issues/no-claim-gets-an-address-space-under-virtio-iommu.md index 337828bb2c..c1c95b126d 100644 --- a/issues/no-claim-gets-an-address-space-under-virtio-iommu.md +++ b/issues/no-claim-gets-an-address-space-under-virtio-iommu.md @@ -9,20 +9,41 @@ opened: 2026-10-12 A process is handed a PCI function only once the function has an address space of its own at a unit and, on x86-64, interrupt remapping (`kernel/src/pcidev/mod.rs`'s header; `Refusal::NotRemapped`, -`Refusal::Untranslated`). The units this kernel programs are VT-d (`DMAR`), -AMD-Vi (`IVRS`) and SMMUv3 (`IORT`) (`kernel/src/iommu/`). QEMU emulates all -three, and VirtualBox VT-d or AMD-Vi under `--iommu` on its ICH9 chipset, off -by default (`VBoxManage modifyvm`). Cloud Hypervisor and crosvm put their -devices behind virtio-iommu instead (virtio 1.2 §5.13, device id 23, PCI -`1af4:1057`): Cloud Hypervisor's documentation says it implements it "in order -to provide a virtual IOMMU to its users" and "will not try to implement a full -emulation of a physical IOMMU" (its `docs/iommu.md`), describes it to the -guest in an ACPI `VIOT` table (its `vmm/src/acpi.rs`), and offers an SMMUv3 -only for a passthrough device behind a physical one (`docs/iommu.md`, -"Emulated ARM SMMUv3"). So under either, every claim is refused and no -userland driver runs. - -The two architectures differ in what virtio-iommu leaves undone: +`Refusal::Untranslated`). The units this kernel programs are VT-d (`DMAR`, +`kernel/src/arch/x86_64/vtd/`) and SMMUv3 (`IORT`, +`kernel/src/arch/aarch64/smmu/`). It finds AMD-Vi units (`IVRS`) and programs +none (`kernel/src/arch/x86_64/iommu_unit.rs`), so a machine whose unit is AMD-Vi, +VirtualBox's `--iommu amd` and QEMU's `amd-iommu` among them, refuses every +claim until `issues/an-amd-vi-machine-isolates-no-device.md` lands. + +QEMU emulates VT-d, AMD-Vi and SMMUv3, and VirtualBox VT-d or AMD-Vi under +`--iommu` on its ICH9 chipset, off by default (`VBoxManage modifyvm`). Cloud +Hypervisor offers +virtio-iommu instead (virtio 1.2 §5.13, device id 23, PCI `1af4:1057`): its +documentation says it implements it "in order to provide a virtual IOMMU to its +users" and "will not try to implement a full emulation of a physical IOMMU", +and puts a device behind it only where the device is tagged `iommu=virtio`, +whose "default value will always be `off`" (its `docs/iommu.md`); it describes +the unit to the guest in a `VIOT` table (its `vmm/src/acpi.rs`), and offers an +SMMUv3 only for a passthrough device behind a physical one (`docs/iommu.md`, +"Emulated ARM SMMUv3"). crosvm puts only a host device passed through with +`--vfio …,iommu=viommu` behind its virtio-iommu (`src/crosvm/cmdline.rs`), never +a device it emulates: `PciDevice::supports_iommu` answers false +(`devices/src/pci/pci_device.rs`) and its `VirtioPciDevice` does not override +it (`devices/src/virtio/virtio_pci_device.rs`). So Cloud Hypervisor by default, +and crosvm for every device it emulates, are the no-unit case of +`issues/a-machine-without-an-iommu-refuses-every-claim.md`, and a device +Cloud Hypervisor is told to put behind its virtio-iommu is refused here too. + +**Programming a virtio-iommu puts virtio in the kernel**, on either +architecture. Its requests travel on a virtqueue, its `requestq` (§5.13.2), +over the virtio PCI transport, and the unit a claim's domain lives at is the +kernel's (`pcidev`'s header). That conflicts with the owner's target that +"`virtio` does not appear anywhere in the kernel" +(`issues/every-driver-is-still-in-the-kernel.md`), and where a virtio-iommu +driver lives is the owner's to rule, for both architectures. + +The two architectures differ in what virtio-iommu leaves undone besides: - **AArch64.** A claimed function's message is its slot's LPI through the ITS (`kernel/src/arch/aarch64/irqchip/its.rs`), whose translation by @@ -33,17 +54,19 @@ The two architectures differ in what virtio-iommu leaves undone: `VIRTIO_IOMMU_RESV_MEM_T_MSI`), whose writes pass untranslated. Nothing mints a `Remapping` there, so a claim still ends at `NotRemapped`. Whether a claim may stand under virtio-iommu with its messages unremapped is the owner's to - rule: `issues/every-driver-is-still-in-the-kernel.md` records his ruling that - "the IOMMU lands first and completely, interrupt remapping included". + rule too: `issues/every-driver-is-still-in-the-kernel.md` records his ruling + that "the IOMMU lands first and completely, interrupt remapping included". bhyve offers its guest no IOMMU of any kind; `issues/a-machine-without-an-iommu-refuses-every-claim.md` holds that case and the owner's ruling on it. -**Owner**: whoever holds `issues/toyos-runs-in-any-open-source-hypervisor.md`. +**Owner**: whoever holds `issues/toyos-runs-in-any-open-source-hypervisor.md`, +blocked on the owner's rulings above. -**Exit**: on AArch64, a claim under virtio-iommu is handed over in a domain -of its own and a DMA outside its grants is a `DMA FAULT` record, under QEMU's -`virt` with `virtio-iommu-pci` and its emulated ITS; on x86-64, the owner's -ruling recorded here, and the claim refused by a `Refusal` that names -virtio-iommu's missing interrupt remapping or handed over as he rules. +**Exit**: the owner's ruling on where a virtio-iommu driver lives, and on +x86-64 his ruling on unremapped messages, recorded here; and on each +architecture a claim under QEMU's `virtio-iommu-pci` (on AArch64 under `virt` +with its emulated ITS) handed over in a domain of its own, a DMA outside its +grants a `DMA FAULT` record, or refused by a `Refusal` that names +virtio-iommu, as he rules. diff --git a/issues/the-virtio-drivers-match-only-a-non-transitional-device-id.md b/issues/the-virtio-drivers-match-only-a-non-transitional-device-id.md index 438816d5dc..de635bbf9a 100644 --- a/issues/the-virtio-drivers-match-only-a-non-transitional-device-id.md +++ b/issues/the-virtio-drivers-match-only-a-non-transitional-device-id.md @@ -10,9 +10,9 @@ Virtio 1.2 §4.1.2.2: "Drivers for device types listed in section 4.1.2 MUST match devices with the PCI Vendor ID 0x1AF4 and the Transitional PCI Device ID indicated in section 4.1.2" — `0x1000` for a network card, `0x1003` for a console, among others — beside the `0x1040`-based id. A transitional device -also publishes the virtio PCI capabilities, and §4.1.4 has a non-transitional -driver drive it through them, and "fail gracefully" where they are absent. That -is not the legacy interface, which this tree refuses. +also publishes the virtio PCI capabilities (§4.1.4), and §4.1.5.1.1.1 has a +non-transitional driver look for them and "fail gracefully" where they are +absent. That is not the legacy interface, which this tree refuses. The tree matches the `0x1040`-based id alone: diff --git a/issues/the-x86-clock-calibrates-against-an-hpet-or-the-boot-panics.md b/issues/the-x86-clock-calibrates-against-an-hpet-or-the-boot-panics.md index 0b5e80e4f4..7a48c06381 100644 --- a/issues/the-x86-clock-calibrates-against-an-hpet-or-the-boot-panics.md +++ b/issues/the-x86-clock-calibrates-against-an-hpet-or-the-boot-panics.md @@ -20,7 +20,7 @@ booted: | | HPET table | ACPI PM timer | TSC rate the platform states | |---|---|---|---| -| Cloud Hypervisor | none (`vmm/src/acpi.rs` builds no `HPET`) | `X_PM_TMR_BLK` | CPUID `0x40000010` EAX, in kHz, where the host's TSC is invariant (`arch/src/x86_64/mod.rs`) | +| Cloud Hypervisor | none (`vmm/src/acpi.rs` builds no `HPET`) | none: an `X_PM_TMR_BLK` in a FADT that sets `HW_REDUCED_ACPI`, below | CPUID `0x40000010` EAX, in kHz, where the host's TSC is invariant (`arch/src/x86_64/mod.rs`) | | crosvm | none (no HPET device in `devices/src`) | none ("not supported", `x86_64/src/acpi.rs`) | CPUID 15H, where a calibrated leaf is configured (`x86_64/src/cpuid.rs`) | | VirtualBox | only with `--x86-hpet on`, default off (`HpetEnabled`, `DevACPI.cpp`) | yes | not read | | bhyve | yes (`usr.sbin/bhyve/acpi.c`) | yes | not read | @@ -29,12 +29,20 @@ So Cloud Hypervisor, crosvm and VirtualBox without `--x86-hpet on` all end at this `expect`. What a machine states instead, by its own documents: the ACPI PM timer, fixed -hardware at 3.579545 MHz (ACPI 6.5 §4.8.3.3), which q35, Cloud Hypervisor, -VirtualBox and bhyve publish; CPUID 15H's crystal ratio (Intel SDM Vol. 2A), +hardware at 3.579545 MHz (ACPI 6.5 §4.8.3.3), which q35, VirtualBox and bhyve +publish; CPUID 15H's crystal ratio (Intel SDM Vol. 2A), which the kernel already reads through `toyos_tsc::stated_hz` and which no AMD part states (`issues/the-loaders-lines-carry-no-time-on-an-x86-cpu-that-states-no-tsc-rate.md`); and a hypervisor's timing leaf `0x40000010`. crosvm without a configured leaf -states none of them. +states none of them. Cloud Hypervisor states the leaf alone: its FADT writes +`X_PM_TMR_BLK` (offset 208) but sets `HW_REDUCED_ACPI`, under which OSPM +"will ignore fields related to the ACPI HW register interface: Fields at +offsets 46 through 108 and 148 through 232" (ACPI 6.5 §5.2.9), the fixed +hardware being "not implemented" there (§4.1); `toyos_acpi::fixed_hardware` +refuses such a FADT whole (`lib/acpi/src/fadt.rs`). + +The ACPI sections cited here are read in a third-party copy of ACPI 6.6 and +numbered as 6.6 numbers them; 6.5 itself is unread. **Owner**: whoever holds `issues/toyos-runs-in-any-open-source-hypervisor.md`. From d0b5ae1149a48d65cccd06366c6e5cfe13ed6815 Mon Sep 17 00:00:00 2001 From: japabu Date: Tue, 13 Oct 2026 02:17:31 +0200 Subject: [PATCH 3/3] issues: review round 1 answered through the orchestrator's rulings MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ACCESS_PLATFORM is the supervisor's check, the kernel staying without virtio; virtio-iommu needs virtio in the kernel on both architectures and is the owner's to rule; Cloud Hypervisor's virtio-iommu is per device and off by default, and crosvm's takes no emulated device; AMD-Vi programs no unit; Cloud Hypervisor's PM timer is ignored under HW_REDUCED_ACPI; bhyve's INTx-only HDA is unsupported; the T14's NVMe row stays out of the class exit; the hardware-reduced exit names Cloud Hypervisor; the cpuvuln clause cites Intel's CPUID.1:ECX[31]; virtio's "fail gracefully" is §4.1.5.1.1.1. Hypervisor sources are read clean-room: their files and commits move to the evidence repository's sources.md, and the ACPI sections stay marked as read in a third-party copy of 6.6. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_015xhaydXCVRNscoccHYFUWy --- ...ne-has-no-power-off-and-no-power-button.md | 28 +++++---- ...s-platform-is-handed-over-as-translated.md | 2 +- ...kserver-claims-qemus-nvme-by-its-pci-id.md | 14 +++-- ...ets-an-address-space-under-virtio-iommu.md | 14 ++--- ...match-only-a-non-transitional-device-id.md | 21 +++---- ...ates-against-an-hpet-or-the-boot-panics.md | 19 +++--- ...oyos-runs-in-any-open-source-hypervisor.md | 59 +++++++++++-------- 7 files changed, 80 insertions(+), 77 deletions(-) diff --git a/issues/a-hardware-reduced-x86-machine-has-no-power-off-and-no-power-button.md b/issues/a-hardware-reduced-x86-machine-has-no-power-off-and-no-power-button.md index ca2a6762ea..6d508a51fe 100644 --- a/issues/a-hardware-reduced-x86-machine-has-no-power-off-and-no-power-button.md +++ b/issues/a-hardware-reduced-x86-machine-has-no-power-off-and-no-power-button.md @@ -19,24 +19,26 @@ A hardware-reduced platform has neither PM1 block nor SMI_CMD. ACPI 6.5 gives it the FADT's `SLEEP_CONTROL_REG` and `SLEEP_STATUS_REG` for sleep and soft-off (§4.8.3.7, Sleep Control and Status Registers), and the Generic Event Device, `ACPI0013`, for its events, the power button's among them: an -interrupt whose `_EVT` method the AML runs. Every AArch64 machine is one, which -this tree meets with PSCI (`kernel/src/arch/aarch64/power.rs`). +interrupt whose `_EVT` method the AML runs (§4.1.1.1). Every AArch64 machine +is one, which this tree meets with PSCI (`kernel/src/arch/aarch64/power.rs`). +These ACPI sections are as a third-party copy of ACPI 6.6 numbers them; 6.5 +itself is unread. Cloud Hypervisor's x86-64 FADT sets `HW_REDUCED_ACPI` and names the two sleep -registers and a reset register (`create_facp_table` in its `vmm/src/acpi.rs`); it -writes PM1a blocks only under its `legacy_acpi_pm1a` option, at ports it says -"conforming guests ignore". Its power button, which its API's -`/vm.power-button` presses, is a GED whose `_EVT` notifies `\_SB_.PWRB` -(its `devices/src/acpi.rs`). QEMU's `microvm` is hardware-reduced too. q35, +registers and a reset register, and PM1a blocks only under an option of its +own, at otherwise unused ports. Its power button, which its API's +`/vm.power-button` presses (its `docs/api.md`), is a GED whose `_EVT` notifies +`\_SB_.PWRB`. QEMU's `microvm` is hardware-reduced too. q35, crosvm, VirtualBox and bhyve publish the fixed hardware. **Owner**: whoever holds `issues/toyos-runs-the-machine-in-acpi-mode-and-interprets-its-aml.md`, whose server holds the machine's power button and S5. -**Exit**: a hardware-reduced FADT and DSDT, recorded from a Cloud Hypervisor -guest with their provenance as `lib/acpi/fixtures/` records QEMU's, decode in -a host test to the sleep control register and the GED's power-button method, -and acpiserver's host test runs that method to a press; and a guest boot on a -hardware-reduced machine, once the harness boots one, powers off on -`shutdown`. +**Exit**: a hardware-reduced FADT and DSDT, recorded by ToyOS from a Cloud +Hypervisor guest with their provenance as `lib/acpi/fixtures/` records QEMU's, +decode in a host test to the sleep control register and the GED's +power-button method, and acpiserver's host test runs that method to a press; +and the image booted by hand under Cloud Hypervisor on x86-64, its log on the +evidence repository under the hypervisor's version and command line, powers +off on `shutdown` and on a press through `/vm.power-button`. diff --git a/issues/a-virtio-function-that-offers-no-access-platform-is-handed-over-as-translated.md b/issues/a-virtio-function-that-offers-no-access-platform-is-handed-over-as-translated.md index 91c3997b99..b2713094f0 100644 --- a/issues/a-virtio-function-that-offers-no-access-platform-is-handed-over-as-translated.md +++ b/issues/a-virtio-function-that-offers-no-access-platform-is-handed-over-as-translated.md @@ -21,7 +21,7 @@ whether a virtio function offers the bit; the holder's driver accepts it where offered and goes on without it (`Offer::accept`, `lib/virtio/src/pci.rs`). A function that does not offer it is not behind the unit at all: QEMU keeps it on the machine's own address space whatever the `DMAR` says -(`hw/virtio/virtio-bus.c`, which `tests/common/iommu.rs` cites). So an honest +(`tests/common/iommu.rs`). So an honest holder's descriptors, which carry domain addresses above all memory (`kernel/src/iommu/window.rs`), reach nothing, and a hostile holder's, which carry physical addresses, reach all of memory: the claim's isolation is void diff --git a/issues/diskserver-claims-qemus-nvme-by-its-pci-id.md b/issues/diskserver-claims-qemus-nvme-by-its-pci-id.md index 41bb4961ff..47bfa96fd0 100644 --- a/issues/diskserver-claims-qemus-nvme-by-its-pci-id.md +++ b/issues/diskserver-claims-qemus-nvme-by-its-pci-id.md @@ -8,8 +8,7 @@ opened: 2026-10-12 Both shipped boot configs give diskserver one device, `pci:1b36:0010` (`images/system.toml`, `images/console.toml`): the vendor and device id of -QEMU's own `nvme` model (`hw/nvme/ctrl.c`, `PCI_DEVICE_ID_REDHAT_NVME` under -Red Hat's vendor id). A claim matches the id exactly (`pcidev::claim`, +QEMU's own `nvme` model, under Red Hat's vendor id. A claim matches the id exactly (`pcidev::claim`, `kernel/src/pcidev/mod.rs`), so on any other NVMe controller the supervisor endows diskserver nothing, it says `no NVMe controller this row names is on this machine; serving no partition`, and DATA, LOG and BOOT are absent unless @@ -17,13 +16,13 @@ the image is on a USB stick the kernel drives. The driver is not QEMU's: `Controller::open` (`system/diskserver/src/nvme.rs`) takes any function whose class code is `01:08:02`. -Read from each hypervisor's source, not booted: +Read, not booted: | | NVMe id | MSI-X table | |---|---|---| | QEMU `nvme` | `1b36:0010`, or `8086:5845` with `use-intel-id=on` | BAR 0, unless `msix-exclusive-bar=on` | -| VirtualBox, `src/VBox/Devices/Storage/DevNVMe.cpp` | `80ee:4e56` | BAR 5 | -| bhyve, `usr.sbin/bhyve/pci_nvme.c` | `fb5d:0a0a` | BAR 4 | +| VirtualBox | `80ee:4e56` | BAR 5 | +| bhyve | `fb5d:0a0a` | BAR 4 | Cloud Hypervisor and crosvm offer no NVMe. Every controller on metal carries its maker's id. @@ -47,4 +46,7 @@ its table and grants name (`system/diskserver/src/main.rs`). `toyos_abi::syscall::DeviceRequest::parse` takes and refuses malformed; both shipped configs name NVMe that way; and a guest boot whose controller is QEMU's `nvme` with `use-intel-id=on` serves DATA through diskserver, which a -row naming `1b36:0010` instead reds. +row naming `1b36:0010` instead reds. The configs the T14 boots keep their id +row, outside this exit, until the T14's NVMe holds no Ubuntu: a class row there +would hand diskserver the disk the metal workflow reboots through (the +orchestrator's ruling, #896, issuecomment-6131213771, ruling 4). diff --git a/issues/no-claim-gets-an-address-space-under-virtio-iommu.md b/issues/no-claim-gets-an-address-space-under-virtio-iommu.md index c1c95b126d..7253645039 100644 --- a/issues/no-claim-gets-an-address-space-under-virtio-iommu.md +++ b/issues/no-claim-gets-an-address-space-under-virtio-iommu.md @@ -24,14 +24,12 @@ documentation says it implements it "in order to provide a virtual IOMMU to its users" and "will not try to implement a full emulation of a physical IOMMU", and puts a device behind it only where the device is tagged `iommu=virtio`, whose "default value will always be `off`" (its `docs/iommu.md`); it describes -the unit to the guest in a `VIOT` table (its `vmm/src/acpi.rs`), and offers an -SMMUv3 only for a passthrough device behind a physical one (`docs/iommu.md`, -"Emulated ARM SMMUv3"). crosvm puts only a host device passed through with -`--vfio …,iommu=viommu` behind its virtio-iommu (`src/crosvm/cmdline.rs`), never -a device it emulates: `PciDevice::supports_iommu` answers false -(`devices/src/pci/pci_device.rs`) and its `VirtioPciDevice` does not override -it (`devices/src/virtio/virtio_pci_device.rs`). So Cloud Hypervisor by default, -and crosvm for every device it emulates, are the no-unit case of +the unit to the guest in a `VIOT` table, and offers an SMMUv3 only for a +passthrough device behind a physical one (`docs/iommu.md`, "Emulated ARM +SMMUv3"). crosvm puts only a host device passed through with +`--vfio …,iommu=viommu` behind its virtio-iommu, never a device it emulates. +So Cloud Hypervisor by default, and crosvm for every device it emulates, are +the no-unit case of `issues/a-machine-without-an-iommu-refuses-every-claim.md`, and a device Cloud Hypervisor is told to put behind its virtio-iommu is refused here too. diff --git a/issues/the-virtio-drivers-match-only-a-non-transitional-device-id.md b/issues/the-virtio-drivers-match-only-a-non-transitional-device-id.md index de635bbf9a..8358f95c15 100644 --- a/issues/the-virtio-drivers-match-only-a-non-transitional-device-id.md +++ b/issues/the-virtio-drivers-match-only-a-non-transitional-device-id.md @@ -25,24 +25,17 @@ The launcher and the harness hide it: each asks QEMU for `virtio-net-pci-non-transitional` and `virtio-serial-pci-non-transitional` (`build/qemu.rs`, `tests/common/qemu.rs`). QEMU's own default is transitional on q35's root bus — `disable-legacy=auto` resolves to off for a device not -behind a PCIe port (`virtio_pci_realize`, `hw/virtio/virtio-pci.c`) — and -VirtualBox's virtio-net is `1af4:1000` (`PCI_DEVICE_ID_VIRTIONET_HOST`, -`src/VBox/Devices/Network/DevVirtioNet.cpp`). On either, netstack has no card -and the kernel no virtio console. Cloud Hypervisor and crosvm use the -`0x1040`-based ids (Cloud Hypervisor's -`virtio-devices/src/transport/pci_device.rs`, crosvm's -`devices/src/virtio/virtio_pci_device.rs`). bhyve's virtio-net and -virtio-blk are legacy-only (its `usr.sbin/bhyve/virtio.c` allocates an I/O BAR -and no virtio capability), so a conforming driver matches them and refuses -them by name. +behind a PCIe port — and VirtualBox's virtio-net is `1af4:1000`. On either, +netstack has no card and the kernel no virtio console. Cloud Hypervisor and +crosvm use the `0x1040`-based ids. bhyve's virtio-net and virtio-blk are +legacy-only, an I/O BAR and no virtio capability, so a conforming driver +matches them and refuses them by name. Virtio-gpu and virtio-sound have no transitional id, so their matches are already whole. A transitional device offers no `VIRTIO_F_ACCESS_PLATFORM` on either: QEMU -refuses the pair ("VIRTIO_F_IOMMU_PLATFORM was supported by neither legacy nor -transitional device", `hw/virtio/virtio-pci.c`), and VirtualBox's virtio core -implements none (`VIRTIO_F_ACCESS_PLATFORM`, "VirtIO 1.1 NYI", -`src/VBox/Devices/VirtIO/VirtioCore.h`). So behind a unit such a NIC is one no +refuses `iommu_platform=on` on a transitional device, and VirtualBox's virtio +devices offer the bit nowhere. So behind a unit such a NIC is one no claim may take (`issues/a-virtio-function-that-offers-no-access-platform-is-handed-over-as-translated.md`); the matches reach a card only on the path diff --git a/issues/the-x86-clock-calibrates-against-an-hpet-or-the-boot-panics.md b/issues/the-x86-clock-calibrates-against-an-hpet-or-the-boot-panics.md index 7a48c06381..85da700133 100644 --- a/issues/the-x86-clock-calibrates-against-an-hpet-or-the-boot-panics.md +++ b/issues/the-x86-clock-calibrates-against-an-hpet-or-the-boot-panics.md @@ -15,22 +15,21 @@ cross-check only. A machine whose firmware publishes no `HPET` table stops there, before a timer ticks, and before PCI is enumerated, so a console that is a virtio function never carries the panic. -QEMU's q35 has one. Of the other targets, read from their sources and not -booted: +QEMU's q35 has one. Of the other targets, read and not booted: | | HPET table | ACPI PM timer | TSC rate the platform states | |---|---|---|---| -| Cloud Hypervisor | none (`vmm/src/acpi.rs` builds no `HPET`) | none: an `X_PM_TMR_BLK` in a FADT that sets `HW_REDUCED_ACPI`, below | CPUID `0x40000010` EAX, in kHz, where the host's TSC is invariant (`arch/src/x86_64/mod.rs`) | -| crosvm | none (no HPET device in `devices/src`) | none ("not supported", `x86_64/src/acpi.rs`) | CPUID 15H, where a calibrated leaf is configured (`x86_64/src/cpuid.rs`) | -| VirtualBox | only with `--x86-hpet on`, default off (`HpetEnabled`, `DevACPI.cpp`) | yes | not read | -| bhyve | yes (`usr.sbin/bhyve/acpi.c`) | yes | not read | +| Cloud Hypervisor | none | none: an `X_PM_TMR_BLK` in a FADT that sets `HW_REDUCED_ACPI`, below | CPUID `0x40000010` EAX, in kHz, where the host's TSC is invariant | +| crosvm | none | none | CPUID 15H, where a calibrated leaf is configured | +| VirtualBox | only with `--x86-hpet on`, default off (`VBoxManage modifyvm`) | yes | not read | +| bhyve | yes | yes | not read | So Cloud Hypervisor, crosvm and VirtualBox without `--x86-hpet on` all end at this `expect`. What a machine states instead, by its own documents: the ACPI PM timer, fixed hardware at 3.579545 MHz (ACPI 6.5 §4.8.3.3), which q35, VirtualBox and bhyve -publish; CPUID 15H's crystal ratio (Intel SDM Vol. 2A), +publish; CPUID 15H's crystal ratio (Intel SDM Vol. 1, 253665-093US, Table 21-53), which the kernel already reads through `toyos_tsc::stated_hz` and which no AMD part states (`issues/the-loaders-lines-carry-no-time-on-an-x86-cpu-that-states-no-tsc-rate.md`); and a hypervisor's timing leaf `0x40000010`. crosvm without a configured leaf @@ -39,10 +38,8 @@ states none of them. Cloud Hypervisor states the leaf alone: its FADT writes "will ignore fields related to the ACPI HW register interface: Fields at offsets 46 through 108 and 148 through 232" (ACPI 6.5 §5.2.9), the fixed hardware being "not implemented" there (§4.1); `toyos_acpi::fixed_hardware` -refuses such a FADT whole (`lib/acpi/src/fadt.rs`). - -The ACPI sections cited here are read in a third-party copy of ACPI 6.6 and -numbered as 6.6 numbers them; 6.5 itself is unread. +refuses such a FADT whole (`lib/acpi/src/fadt.rs`). These ACPI sections are as +a third-party copy of ACPI 6.6 numbers them; 6.5 itself is unread. **Owner**: whoever holds `issues/toyos-runs-in-any-open-source-hypervisor.md`. diff --git a/issues/toyos-runs-in-any-open-source-hypervisor.md b/issues/toyos-runs-in-any-open-source-hypervisor.md index c5519a1a02..16f2c0b4d5 100644 --- a/issues/toyos-runs-in-any-open-source-hypervisor.md +++ b/issues/toyos-runs-in-any-open-source-hypervisor.md @@ -14,10 +14,13 @@ and the real devices it already drives (NVMe, xHCI and USB HID, the Intel NICs, the i8042), and nothing it ships depends on a QEMU-only device or behaviour. Nothing in this tree records a boot of this image under any hypervisor but -QEMU. What this file says of the others is read from their documentation and -source at their default branches, not measured. Cloud Hypervisor and crosvm run -on Linux hosts and bhyve on FreeBSD, none of them on the macOS development -host. +QEMU. What this file and the issues it lists say of the others is read from +their documentation and source at their default branches, not measured; the +source is read clean-room, so the files and commits it was read from are +provenance in +https://github.com/ToyOSOrg/evidence/blob/0b27193281e0baee2d7a135f6c17934b5d6b456f/sources.md, +and nothing here cites them. Cloud Hypervisor and crosvm run on Linux hosts +and bhyve on FreeBSD, none of them on the macOS development host. ## What the shipped OS takes from QEMU @@ -35,23 +38,25 @@ signature. What they do depend on, each filed: and, filed before this track: -- `issues/a-machine-without-an-iommu-refuses-every-claim.md` — bhyve, and - VirtualBox and QEMU by default. +- `issues/a-machine-without-an-iommu-refuses-every-claim.md` — bhyve; crosvm + for every device it emulates; and VirtualBox, QEMU and Cloud Hypervisor by + default (`issues/no-claim-gets-an-address-space-under-virtio-iommu.md`). - `issues/a-controller-whose-msix-table-is-in-bar-0-cannot-be-driven-from-userland.md` — QEMU's NVMe by default; and every virtio function of Cloud Hypervisor and crosvm, which keep the MSI-X table and PBA in the BAR their virtio - structures are in (Cloud Hypervisor's - `virtio-devices/src/transport/pci_device.rs`, crosvm's - `devices/src/virtio/virtio_pci_device.rs`). + structures are in. - `issues/a-hid-interface-outside-the-boot-subclass-is-read-as-qemus-tablet.md` - — VirtualBox's USB tablet, a subclass-0 interface (its - `src/VBox/Devices/Input/UsbMouse.cpp`); bhyve's declares the boot mouse - subclass and protocol (its `usr.sbin/bhyve/usb_mouse.c`). + — VirtualBox's USB tablet, a subclass-0 interface; bhyve's declares the + boot mouse subclass and protocol. - `issues/getsockname-answers-an-address-nobody-asked-netstack-for.md` — QEMU's user-mode network address in `sdk/libc`. - `issues/pci-is-enumerated-on-segment-group-0-alone.md` — Cloud Hypervisor's `num_pci_segments`. +`issues/an-amd-vi-machine-isolates-no-device.md` is met only where a target is +asked for AMD-Vi, VirtualBox's `--iommu amd` and QEMU's `amd-iommu`, and no +target's default asks. + Three of the launcher's QEMU options keep three of these off every guest test: the `-non-transitional` virtio devices the transitional id, `iommu_platform=on` the missing `ACCESS_PLATFORM`, and `msix-exclusive-bar=on` @@ -63,8 +68,10 @@ the harness's alone. Adaptations that degrade nothing: the TCO table's q35 row (`lib/tco`, armed only by the `watchdog` parameter and refused by name elsewhere), netstack's 82574L row for QEMU's `e1000e`, the hard-lockup bound's path for a machine -with no PMU, `toyos-cpuvuln`'s reading of CPUID's hypervisor bit as Linux -reads it, and the loader's read of PI's DXE services (`loader/src/gcd.rs`), +with no PMU, `toyos-cpuvuln`'s reading of `CPUID.1:ECX[31]`, which Intel +defines as "Not Used: Intel processors always return 0. Allocated for use by +software emulation" (Intel SDM Vol. 1, 253665-093US, Table 21-9; AMD's APM +unread), and the loader's read of PI's DXE services (`loader/src/gcd.rs`), which hands over the root bridges' windows alone where firmware has none. ## Each hypervisor, x86-64 @@ -72,8 +79,8 @@ which hands over the root bridges' windows alone where firmware has none. | | Firmware | Disk | Display | Network | Input | Platform | IOMMU | |---|---|---|---|---|---|---|---| | QEMU q35 | edk2 | NVMe `1b36:0010`, virtio-blk, AHCI, USB | std-vga GOP, virtio-gpu | virtio-net (transitional by default), e1000e | i8042, USB HID, virtio-input | HPET, full ACPI | VT-d, AMD-Vi, virtio-iommu | -| Cloud Hypervisor | `CLOUDHV.fd` (edk2) | virtio-blk | none | virtio-net | none | no HPET; hardware-reduced ACPI; I/O APIC only with the serial port | virtio-iommu | -| crosvm | a `--bios` image; which UEFI boots under it was not read | virtio-blk, virtio-scsi | virtio-gpu | virtio-net | virtio-input | no HPET, no PM timer | virtio-iommu | +| Cloud Hypervisor | `CLOUDHV.fd` (edk2) | virtio-blk | none | virtio-net | none | no HPET; hardware-reduced ACPI; an I/O APIC its MADT lists always, which its `docs/device_model.md` disables without the serial port | virtio-iommu, per device and off by default | +| crosvm | a `--bios` image; which UEFI boots under it was not read | virtio-blk, virtio-scsi | virtio-gpu | virtio-net | virtio-input | no HPET, no PM timer | virtio-iommu for a passed-through host device alone | | VirtualBox 7 | EFI with `--firmware efi`; PCIe and MSI only with `--chipset ich9` | NVMe `80ee:4e56`, AHCI, USB, virtio-scsi | VMSVGA GOP | 8254x e1000s, virtio-net `1af4:1000` | i8042, USB HID | HPET off by default | VT-d or AMD-Vi with `--iommu`, off by default | | bhyve | `BHYVE_UEFI.fd` (edk2) | NVMe `fb5d:0a0a`, AHCI, legacy virtio-blk | `fbuf` GOP | 82545EM, legacy virtio-net | i8042, xHCI tablet | HPET, full ACPI; HDA on INTx only | none | @@ -82,18 +89,20 @@ this image and enumerate no PCIe; with `--firmware efi` and `--chipset ich9` it stops, like Cloud Hypervisor and crosvm, at the HPET `expect`, which on Cloud Hypervisor no console carries unless its serial port is on. VirtualBox with `--x86-hpet on` too, and bhyve: a desktop on a GOP panel with i8042 or -USB input, no disk, no network, and on bhyve an HDA the kernel arms no -interrupt on. +USB input, no disk, no network, and on bhyve no sound (below). Once the dependencies close, each still lacks a driver this tree does not have: virtio-blk on Cloud Hypervisor and crosvm, whose only disk it is; virtio-input on crosvm, whose only input it is; and on bhyve, whose virtio is -legacy, a driver for its 82545EM, its only other NIC, and an interrupt for its -HDA, which raises INTx alone where the kernel's driver arms MSI-X or MSI -(`kernel/src/drivers/hda.rs`, `arm_interrupt`). VirtualBox lacks none: its -NVMe, GOP, i8042, xHCI and HDA are driven here, and its virtio-net once its +legacy, a driver for its 82545EM, its only other NIC. VirtualBox lacks none: +its NVMe, GOP, i8042, xHCI and HDA are driven here, and its virtio-net once its transitional id is. +Unsupported, and not owed: bhyve's HDA, which raises INTx alone. ToyOS arms +MSI-X or MSI only (`kernel/src/drivers/hda.rs`'s `arm_interrupt`, `pcidev`) +and takes up no line-based interrupt (the orchestrator's ruling, #896, +issuecomment-6131213771, ruling 3), so bhyve has no sound. + ## AArch64 Outside `issues/toyos-runs-on-arm64.md`'s ruled target, QEMU `virt` on the @@ -116,8 +125,10 @@ Hypervisor, crosvm, VirtualBox and bhyve on x86-64 to `Boot: complete` with DATA served and a DHCP lease, each boot's log on the evidence repository under the hypervisor's version and command line. -**Blocked on**: the owner's ruling on interrupt remapping under virtio-iommu -(`issues/no-claim-gets-an-address-space-under-virtio-iommu.md`) and the +**Blocked on**: the owner's rulings on where a virtio-iommu driver lives, on +both architectures, against his target of a kernel without virtio, and on +interrupt remapping under virtio-iommu on x86-64 +(`issues/no-claim-gets-an-address-space-under-virtio-iommu.md`); the signed-driver design `issues/a-machine-without-an-iommu-refuses-every-claim.md` owes, and a host for each hypervisor.