VT-d: a moved context entry is invalidated under the domain it held, in one 16-byte write, and GCMD follows §11.4.4.1

issues/kernel/a-function-refused-under-translation-faults-read-permission-after-the-hand-over.md

@@ -0,0 +1,69 @@
+---
+status: open
+kind: defect
+opened: 2026-10-04
+---
+
+# A function refused under translation faults read permission after the hand-over
+
+On the T14, when the integrated GPU (`00:02.0`, unit0) has its scanout reads
+refused for about 2 ms (0.153 s to 0.155 s) by a unit left translating, and that unit is then
+handed over and programmed, the GPU's next reads fault `0x06` (read
+permission) through this kernel's identity domain, which grants them, and the
+kernel halts on `DMA FAULT owner=kernel`. `vtd::enable` and `Unit::hand_over`
+(`kernel/src/arch/x86_64/vtd/mod.rs`) do what VT-d Rev. 4.1 asks before `TE`
+goes back on: `SRTP` to a fresh root, then a global context-cache and a global
+IOTLB invalidation with both drains, acknowledged (§6.6). A unit reporting
+`CAP.CM=0`, as unit0 does, caches no faulted translation (§6.2.4.1), so a
+refusal before the hand-over leaves nothing in the unit the spec lets fault
+after it. What carries it across, the GPU or the unit, is unknown.
+
+Firmware does not hand the T14 over this way: its four units read
+`gsts=0x40000000`. The state is the one the `iommu-firmware-left` actuator left
+at `9be0a8ac8` of pull request #700, which pointed every unit at an empty root
+while unit0's identity domain was built. Today's actuator translates through
+the identity domain and refuses nothing.
+
+Evidence, pull request #700, T14 boots:
+
+- `9be0a8ac8` plus the `FSTS` probe: unit0's one record, before `fault::arm`
+  clears it, is `00:02.0` read `0x9cb80000` reason `0x01` with `FSTS=0x3`
+  (overflowed), from the empty root. Once `TE` is confirmed, `FSTS=0` and the
+  record is clear. Next, `unit0 fault recording overflowed`, then
+  `00:02.0` read `0x9cc39000` reason `0x06`. Every address is inside
+  `rmrr0` (`0x9c000000..0xa07fffff`, scope `00:02.0`).
+- `55a5ccb42`, the same empty-root actuator but handed over before the build,
+  so the refusal and the hand-over are both logged at 0.153 s and `TE` stayed
+  off until 0.155 s: no fault.
+- `25ab31bd5` and `5e8cbb06e`, no refusal, and the hand-over and `translating`
+  logged in the same millisecond: no fault.
+
+- `9be0a8ac8` plus a walk probe: unit0's record before `fault::arm` is again
+  `00:02.0` read `0x9ccc0000` reason `0x01` with `FSTS=0x3`. At the fault,
+  `00:02.0` read `0x9cdb7000` reason `0x06`, raw high word
+  `0xc000000600000010`: `AT=0`, `PP=0`, `EXE=0`, `PRIV=0`, `T2=0`, an
+  untranslated read with no PASID. The unit's walk, from `RTADDR` `0xa18000`,
+  each entry read from DRAM after `clflush`: root `0xa19001`, context
+  `0xa04001:0x102` (`P=1`, `TT=00`, `AW=2`, `DID=1`), then `0xa05003`,
+  `0xa08003` and the 2 MiB leaf `0x9cc00083`, present with read and write over
+  `0x9cdb7000`. The GPU's PCI `STATUS` reads `0x0010`, no master or target
+  abort, before the actuator, before the hand-over, at `TE` and at the fault.
+  The unit refused a read the tables in memory grant.
+- the walk probe plus 3 ms with `TE` off after each hand-over: the same
+  empty-root refusals (`FSTS=0x3`, record `0x01`) before `fault::arm`, then no
+  `DMA FAULT` and `Boot: complete`.
+
+So the tables are not what refuses, and neither is any cached state VT-d Rev.
+4.1 lets the unit keep. The `0x06` needs both a period of refused reads and
+`TE` back on within microseconds of the hand-over: three boots of that
+actuator with no hold faulted (`9be0a8ac8` alone, with the `FSTS` probe and
+with the walk probe), one with a 3 ms hold did not, and `55a5ccb42`,
+`TE` off about 2 ms after its refusals, did not.
+
+**Exit**: on the T14, the reproduction (the selftests boot at `9be0a8ac8`)
+reaches `Boot: complete` with no `DMA FAULT` line, through a hand-over that
+waits on an event the unit or the function reports. A timed hold with `TE`
+off does not close this: it is a flat wait no hardware document mandates, it
+widens the untranslated window
+`issues/kernel/a-unit-left-translating-passes-dma-untranslated-while-programmed.md`
+records, and it rests on one green boot against three reds.

issues/kernel/a-released-function-still-names-its-slots-domain.md

@@ -0,0 +1,19 @@
+---
+status: open
+kind: defect
+opened: 2026-10-03
+---
+
+# A released function still names its slot's domain
+
+`pcidev::release` (`kernel/src/pcidev/mod.rs`, `tear_down`) clears Bus Master
+Enable and resets the function, and leaves its context entry naming the slot's
+domain (`kernel/src/arch/x86_64/vtd/table.rs`, `bind`, is reached from no
+release). The slot's next holder is attached to that same domain
+(`slot_space`), so the released function's requests translate through the
+next holder's grants for as long as it is that holder's. Nothing but the
+released function's own obedience to `BME` stands between them: a function
+that masters the bus with `BME` clear reaches another process's memory.
+
+**Exit**: a released function's context entry names no domain once it is
+quiet, read back off the unit's table on the T14 after its slot's next claim.

issues/kernel/a-unit-left-translating-passes-dma-untranslated-while-programmed.md

@@ -0,0 +1,32 @@
+---
+status: open
+kind: defect
+opened: 2026-10-04
+---
+
+# A unit left translating passes DMA untranslated while it is programmed
+
+`vtd::enable` (`kernel/src/arch/x86_64/vtd/mod.rs`) switches off the
+translation firmware left on (`Unit::hand_over`) once the unit's tables are
+built, and turns it back on through this kernel's root table only after the
+queue, fault reporting, both table pointers and a global invalidation are
+programmed. In between, every function behind that unit reaches all of memory.
+
+Keeping `TE` on instead is what VT-d Rev. 4.1 §6.6 allows only onto tables that
+remap exactly as the ones being walked, which takes firmware's tables read and
+copied first, as Linux's kdump path does (`copy_translation_tables`,
+`drivers/iommu/intel/iommu.c`); Linux outside kdump switches `TE` off as this
+kernel does. Any copy has to read them before the pmm hands their memory out:
+it takes `EfiBootServicesData` as free (`toyos-bootmap/src/lib.rs`,
+`is_usable_type`).
+
+Evidence: no machine here is handed a unit translating; the T14's four units
+read `gsts=0x40000000` (pull request #700's selftests boot). Staged by the
+`iommu-firmware-left` actuator on QEMU 11.1.1, unit0 logs `was handed over
+with translation on` at 0.083 s, just before translation goes off, and
+`translating` at 0.090 s, just after it is back; two of the hand-over's own log
+lines fall between.
+
+**Exit**: under `iommu-firmware-left`, on QEMU and on the T14, no unit logs
+`was handed over with translation on; it goes off first`, and every unit's
+`translating` line reads `tes=y`.

issues/kernel/a-unit-this-kernel-does-not-program-keeps-what-firmware-left-on.md

@@ -0,0 +1,24 @@
+---
+status: open
+kind: defect
+opened: 2026-10-04
+---
+
+# A unit this kernel does not program keeps what firmware left on
+
+`vtd::init` (`kernel/src/arch/x86_64/vtd/mod.rs`) switches off what firmware
+left on (`Unit::hand_over`) only inside `enable`, which a unit reaches only if
+`plan` accepts it and it is among the first `MAX_UNITS` described. A unit
+`plan` refuses, and every unit past `MAX_UNITS`, keeps whatever firmware left
+on while the kernel treats it as absent: translation through firmware's tables,
+and interrupt remapping, which blocks every compatibility-format message this
+kernel then sends through the unit unless firmware also left `CFI` on with
+`EIME` off (VT-d Rev. 4.1 §5.1.4). The module header's "leaves the unit
+switched off" is false of such a unit.
+
+Evidence: read off the tree; no machine here has one. The T14's four units are
+all programmed, and its firmware leaves each at `gsts=0x40000000`.
+
+**Exit**: a boot that leaves a unit translating, remapping and queueing and
+then refuses its plan logs each of those going off and boots on, judged on
+QEMU.

kernel/src/actuator.rs

@@ -112,6 +112,11 @@ actuators! {
     /// Withhold `VIRTIO_F_ACCESS_PLATFORM` from every virtio device but the console, staging a function whose addresses the unit never translates.
     virtio_no_access_platform = "virtio-no-access-platform";
 
+    /// Leave every IOMMU unit queueing, translating and remapping through
+    /// tables of its own, as firmware may hand one over, just before this
+    /// kernel programs it. Judged by `iommu_firmware_left`.
+    iommu_firmware_left = "iommu-firmware-left";
+
     /// Leave every AP holding the CR0/CR4 that INIT left it.
     no_ap_control_regs = "no-ap-control-regs";
 

kernel/src/arch/x86_64/vtd/domain.rs

@@ -168,8 +168,7 @@ pub fn attach(stream: StreamId, id: DomainId) {
     let domain = *domains.at(id);
     let mut units = UNITS.lock();
     for unit in units.iter_mut() {
-        table::bind(&mut TABLES.lock(), unit.root(), stream, &domain);
-        unit.invalidate_context(domain.id(), stream.requester());
+        unit.attach(stream, &domain);
     }
     super::fault::attached(stream, domain.id());
     log!("iommu: {stream} moves to domain{}", domain.id());

kernel/src/arch/x86_64/vtd/interrupt.rs

@@ -40,7 +40,7 @@ pub const IRTA_REG: u64 = 0xB8;
 
 pub const INTERRUPT_REMAPPING_ENABLE: u32 = 1 << 25;
 pub const SET_TABLE_POINTER: u32 = 1 << 24;
-/// Set means compatibility-format messages bypass remapping; `GCMD.CFI` is never written, so it reads clear.
+/// Set means compatibility-format messages bypass remapping.
 pub const COMPATIBILITY_FORMAT: u32 = 1 << 23;
 
 /// `2^(7+1)` entries of 128 bits is exactly the 4 KiB [`Tables::alloc`] hands out.