virtio-gpu leaves the kernel: the compositor drives it through its claim and follows the host's display

[Japabu]

Stage 2 of issues/the-desktop-draws-at-the-panels-scale.md and the virtio-gpu half of stage 1 of issues/every-driver-is-still-in-the-kernel.md, as the design accepted on #895 (comment) has it. Head 9a1bff55e4769b70bcd9c8223913ba678e307d1a, with main at e136c77fd merged (review round 1 at 1ab1d9917; the claim is spelled pci:03:80:00/1af4:1050 in #926's grammar).

Problem and resulting behaviour

The kernel had the only virtio-gpu driver, behind a Gpu trait and four SYS_GPU_* syscalls that the compositor and console called through the framebuffer claim. It asserted on device answers, allocated a second scanout nobody read, never asked GET_DISPLAY_INFO, floored the mode at 1280x720, and could not follow a host window.

Now the compositor holds a pci:1af4:1050 claim (images/system.toml) through the ordinary claim path: BAR, one MSI-X entry, its IOMMU domain, grants and lends. It drives the function in-process. Virtio is taken first and the firmware's framebuffer otherwise. On metal (GOP) the compositor gets the same one-scanout framebuffer as before. No SYS_GPU_* number is left in sdk/abi/src/syscall.rs, and the kernel has no virtio-gpu code.

• system/compositor/virtio-gpu (toyos-virtio-gpu): Virtio 1.2 §5.7's 2D commands over the control and cursor queues, in one grant, on toyos-virtio (no second transport). Pure and host-tested on a device model.
  • Every control command is fenced (§5.7.6.5). An answer missing the flag, with another id, short of its structure or of another type is refused by name, and the compositor ends.
  • RESP_ERR_OUT_OF_MEMORY to RESOURCE_CREATE_2D, RESOURCE_ATTACH_BACKING or SET_SCANOUT declines the mode and the old one stays.
  • A frame's transfers are answered before its flushes are sent, because §5.7.6.5 promises completion and not order.
  • A mode change is a page flip (§5.7.6.3). The new scanout is lent, created, attached and drawn while the old one is on screen. Then: whole transfer, SET_SCANOUT, flush, then a fenced UNREF of the old one, and only after that the old backing is unlent.
  • The cursor queue carries UPDATE_CURSOR/MOVE_CURSOR unfenced. Used len 0 is no answer (QEMU's, measured on The loader hands the panel's EDID to the compositor, which chooses the display's scale #895); 24 is an answer and is checked; anything else is refused.
  • One MSI-X entry serves both queues and configuration. Every wake, including the one inside a command's wait, drains the cursor queue and reads events_read. The output is read after events_clear (§5.7.4.2, §5.7.5).
• Compositor (display.rs, session.rs):
  • At start it takes the device's preferred size, clamped axis by axis into toyos-desktop's new mode bounds (least 642x446, the old Scale floor moved there; largest 7680x4320). A disabled output gives the least mode.
  • Each display event re-reads the output, logs mode, mm and scale, and flips to the new size if mode::follow accepts it.
  • screen WxH takes the same flip on virtio and is refused by name on GOP; it is still the window protocol's MSG_SET_RESOLUTION until the display port of the scale issue's item 1.
  • The hardware cursor is used on virtio, the software cursor on GOP.
  • It logs the device's EDID base block once, and its first frame.
• pcidev: what a claim lends is bounded by its own window, toyos_abi::pci::LEND_WINDOW = 256 MiB, apart from the unchanged 32 MiB grant bound. toyos-desktop asserts at compile time that two scanouts at the largest mode fit in it.
• toyos-virtio: Live::device_read32/device_write32, bounded to the device structure, FAILED on a refusal, and host-tested.
• cargo run on macOS opens the Virtio profile with -display cocoa,zoom-to-fit=on, so a window resize reaches the guest's mode. Gop and Metal keep cocoa.

What goes (measured against the merge base)

• Kernel: 1045 lines deleted, 81 added.
  • Removed: drivers/virtio_gpu.rs (649), gpu.rs (121), the SYS_GPU_* dispatch and sys_gpu_reset_scanout, the claim's cursor region and second scanout, remint, and the panic console's virtio disable.
  • Also removed: VirtioDevice::init, and the virtio-no-access-platform actuator with platform_addressing.
• SDK: 117 deleted, 14 added. The four numbers, their wrappers, toyos::gpu (48), and FramebufferInfo's second scanout, cursor and flags.
• Elsewhere:
  • console's presents (it draws only on a GOP machine now).
  • abuse_gpu_resolution (127), which guarded a syscall that is gone.
  • The decline control declining_is_not_free, per the orchestrator's ruling on The loader hands the panel's EDID to the compositor, which chooses the display's scale #895: with the kernel driving only its console the actuator reached nothing. Profile::HeadlessVirtioGpu stays.
• Added in userland: the driver crate (lib.rs 650 and wire.rs 148 production; tests.rs 787), display.rs 396, mode.rs 176 including its tests, and the VNC client 74 (test harness).
• Whole diff: 2821 insertions and 1709 deletions over 61 files.
• The mint-rollback self-test goes (review round 1): every claim description now installs one handle, so the control could not go red for the batch rollback it was written to catch. It is removed with its actuator, judge, metal row, install_all, stage_room and Region::empty. Its staged T14/AMD run passed and proved nothing.

Tests and evidence (README)

check	result
cargo run -- --build-only	exit 0 (log)
cargo run -- --ci host	exit 0, 83 steps green (log)
the_compositor_drives_the_virtio_gpu (new, x86-64 q35, Profile::HeadlessVirtioGpu)	exit 0 (log)
clipboard boot, iommu_virtio_platform, desktop_without_x2apic (GOP), screen_fatal_halt_composited (GOP), console_image_boots (GOP)	exit 0, 5/5 (log)
host mutations m1–m5	each red, tree restored (log)
guest mutations g1, g2	each red (g1, g2)
metal	staged only, not run (request)

What the new guest row establishes. The compositor drives QEMU's virtio-gpu through its claim, and a host display resize reaches the guest's mode. The row:

1. Boots QEMU's device, waits for the compositor's first frame, and screendumps the scanout at 1280x800 (QEMU's default preferred size).
2. Resizes the display to 1024x768 over a VNC client's SetDesktopSize.
3. Waits for compositor: mode 1024x768 and screendumps the scanout again at that size.

A screendump counts as drawn only if it has at least 64 colours: a scanout no frame reached is one colour, or QEMU's two-colour notice. The log shows the device's event read back as display 1024x768, panel 260x195 mm, scale 1.

Why a guest row: no metal machine has a virtio-gpu, and the host model is this design's own reading of §5.7, so only QEMU's device can say it was read right. It boots separately from the clipboard boot because its VNC display may give QEMU's clipboard a peer that the clipboard test's premise rules out. AArch64 is not covered, because a virt boot hands over no PCI claim yet (issues/an-aarch64-cargo-run-boot-hands-over-no-pci-claim.md).

Negative controls, as checked patches, each built and then reverted:

• Host, each reds its named toyos-virtio-gpu tests:
  • m1: clearing events after the reads.
  • m2: no wake inside a command's wait.
  • m3: UNREF sent unfenced.
  • m4: the old backing unlent before SET_SCANOUT.
  • m5: an error answer accepted.
• Guest, each reds the new row:
  • g1: the compositor ignores the display event (STALLED waiting for the followed mode).
  • g2: the compositor's presents are dropped ("the scanout is 1280x800 in 1 colour(s)").
• The whole change reverted onto the base was not run as a control for the new row; g1 and g2 cover the claims that could otherwise pass unseen.

Metal (round 0, head 883e496f7): staged with --metal-readback for gop_edid_scale (the compositor's display row, metalcase boot, T14 and AMD bench) and leak_rollback_selftest (the adapted mint control, shared-debug boot). Both benches then ran it, green: gop_edid_scale T14 handle 0, 309x173 mm, 1.5; AMD handle 1, 309x174 mm, 1.5 (readbacks). The leak row is deleted. No bench rerun for round 1. On gop_edid_scale's path, the framebuffer mint now uses one install in place of a one-object install_all, with the same effect; the GOP row screen_fatal_halt_composited passes in QEMU at the new head.

Round 1 checks at 1ab1d9917 (README): --build-only exit 0; --ci host exit 0, 83 steps green; the new row, the clipboard row and screen_fatal_halt_composited 3/3. The driver crate is unchanged, so m1–m5 stand.

Records

• Closed with their exits met: issues/the-gpu-driver-asserts-on-what-the-device-answers.md and issues/the-second-scanout-buffer-is-read-by-nobody.md.
• Filed, as the design requires:
  • issues/a-lent-scanout-is-one-contiguous-run.md: a mode change is declined when no contiguous run exists, though §5.7.6.1 takes scatter lists.
  • issues/a-restarted-display-holder-lends-around-its-predecessors-scanouts.md: residue on a function no mechanism resets.
• Rewritten in place:
  • Stage 1 of the drivers issue (decline control and actuator gone by the ruling; the profile stays; the cursor-queue len measured).
  • Item 2 of the scale issue (done).
  • The other issues that cited the deleted driver as an instance, so they stay true.

Risks and open points

• Lend window, first-fit: LEND_WINDOW holds two scanouts at the largest mode only together. Lends are placed first-fit, so a small scanout left where it split the window can leave no run for a mode past half of it. That flip is declined, the old mode kept and logged. This is stated at the assertion and in issues/a-lent-scanout-is-one-contiguous-run.md.
• A GPU fault ends the compositor, and so does a virtio-gpu function it was given but cannot bring up (no fallback to a framebuffer). The supervisor does not restart it. That is by design: the desktop cannot outlive its display.
• DEVICE_ANSWER (10 s) is policy for a wait's ceiling, not a measurement.
• followed() drops an event whose mode equals the current one even if only the EDID's mm changed. The scale is drawn at by nothing yet.
• The panic console shows nothing on a virtio-only machine, as before.

🤖 Generated with Claude Code

https://claude.ai/code/session_016UzbDnM9m6nQWhbjGi6Lv1

[Japabu]

Metal readbacks at head 883e496f7, both benches green (README, evidence commit 1d7eadef67f9ac10e82e40f78a4f98ea07b779f5):

• T14 (judge): 2 passed over 2 boots. gop_edid_scale: GOP handle 0, panel 309x173 mm, scale 1.5. leak_rollback_selftest: device-mint PASS (refused=true installed=0).
• AMD bench (judge): 2 passed over 2 boots. gop_edid_scale: GOP handle 1, panel 309x174 mm, scale 1.5. leak_rollback_selftest: device-mint PASS.

The images that ran are the staged ones (sha256 in the README). On metal, the compositor's GOP path and the adapted mint control are unchanged in result.

[Japabu]

Review of #925 at 883e496f7 (main 2c3f61a3e merged)

Architecture

This is the right move and it matches the accepted design (#895 comment 6158220132), with the display port deferred to the scale issue's item 1 as the brief allows.

• What is gone. The kernel has no virtio-gpu code left: no Gpu trait, no SYS_GPU_* number, no remint, no second scanout, no cursor region, and no panic-console disable. rg finds no 0x1050/1af4:1050 anywhere in kernel/, loader/ or build/.
• The claim is ordinary. The compositor holds pci:1af4:1050 through the same claim path netstack and soundserver use, with no privilege beyond it.
• The driver is a pure, forbid(unsafe) crate on toyos-virtio, owned by the one program that uses it.
• Line counts. I checked them against the merge base with git diff --shortstat. They match the body:
  • kernel: 81 added, 1045 deleted
  • SDK: 14 added, 117 deleted
  • whole diff: 2821 insertions, 1709 deletions
  • Production roughly breaks even. About 1.2k kernel/SDK/console lines go. About 1.2k userland lines arrive (driver 798, display.rs 396, mode.rs about 100 plus its tests). Tests grow by 787 host lines and 74 for the VNC client.

What I checked

The driver against Virtio 1.2 §5.7, word by word:

• Header layout. Payloads for RESOURCE_CREATE_2D, ATTACH_BACKING (one mem_entry), SET_SCANOUT, TRANSFER_TO_HOST_2D (rect, le64 offset, id) and RESOURCE_FLUSH, UNREF.
• GET_DISPLAY_INFO's pmodes[0] at header+8/12/16, GET_EDID's 32-byte offset, and the update_cursor layout. Format codes 1 and 2.
• Fencing: every control command carries FLAG_FENCE and a fresh id. An answer missing the flag or with another id is refused before its type is read.
• A frame's transfers are settled before its flushes.
• Flip order: lend, create, attach, whole transfer settled, SET_SCANOUT, flush settled, fenced UNREF, then unlend.
• ERR_OUT_OF_MEMORY handling at create, attach and set: it unrefs and unlends the new scanout and keeps the old one.
• Events: events_read then events_clear(events) happen before the reads (§5.7.4.2). Every wake, including the one inside reap's wait, drains the cursor queue and re-reads events_read. follow_display runs after present on the same pass, so an event a frame's wait consumed is not lost.
• Cursor queue: len 0 is no answer, 24 is checked, anything else is refused. Past 24 is toyos-virtio's Written.

Failure handling:

• Every device refusal is a Refusal by name, and display.rs turns it into a compositor panic. That ends the compositor with the reason and returns the claim.
• No fixed delay anywhere.
• VirtioClaim::wait is bounded by DEVICE_ANSWER. The site calls it "policy, not physics … a hang's ceiling", which is an adequate justification for 10 s.
• The compositor is not restarted after a fault. That is the design's accepted roast ("the desktop cannot outlive its display"), and before this change a device error was a kernel assert!. I don't consider it a gap needing a new record.

pcidev:

• LEND_WINDOW replaces MAX_GRANT_TOTAL only where the window is created and searched (function_space, lend_room).
• dma_map no longer adds lends to the grant sum, and dma_alloc now sums only non-Mapped grants.
• A lend still needs SharedMemObject::ram(): memory this kernel allocated that the holder already holds. The 256 MiB is address space in the function's own domain (OwnSpace::create reserves addresses, not memory or tables up front).
• The only authority that grows is how much of its own memory a holder may pin in its own function's view. It is enforced at the same boundary as before (with_bound). This is sound and minimal.

Evidence (pr-925 at d528eab97). It is all at head 883e496f7, and the lib.rs hash recorded in the mutations log (000a4cb3…) matches mine.

• Host controls m1–m5: each patch applies, builds, reds its named tests, and the tree is restored to 18/18.
• Guest controls: g1 STALLED waiting for the followed mode. g2's first attempt did not build; that is recorded, and the corrected patch reds with "1 colour(s)".
• New row: passes. The guest log shows the device's EDID, display 1280x800, the first frame, then display 1024x768, panel 260x195 mm, then mode 1024x768, and must_be_clean.
• Related rows: 5/5.
• GOP rows: unchanged, and desktop_without_x2apic, screen_fatal_halt_composited and console_image_boots pass.
• Host check: I re-ran cargo run -- --ci host in my own worktree at 883e496f7: exit 0, 83 steps green, matching yours.

Required

1. kernel/src/object/device.rs:186-208 (with :40-55, kernel/src/leak_selftest.rs, kernel/src/object/handle.rs:200-218,425-431, kernel/src/object/shm.rs Region::empty, tests/toyos.rs:989 and :1273, tests/metal/src/lib.rs:750, and the leak-rollback-selftest actuator): the adapted mint-rollback control no longer tests anything.

   • The control exists to show that a batch install refused part-way leaves nothing behind (install_all's all-or-nothing rule).
   • After this change the only caller, install_buffers, always passes exactly one object, for the framebuffer scanout and for the PCM.
   • Minting one handle into a table with room 0 passes for any implementation, rollback or none. It cannot go red for the defect it was written to catch.
   • The body admits the premise ("no description installs more than one handle any more"). Yet a T14 shared-debug row is staged to run it. That spends bench time to confirm a tautology.
   • Fix: make install_buffers a single install and drop the batch comment at :53. Delete mint_rollback_selftest, leak_selftest.rs, the actuator, its judge, its metal row, both param-list entries, stage_room and Region::empty. Delete install_all too if nothing else needs it.
   • The "description bytes unset on refusal, so the next read re-mints" property stays, and inspection establishes it.
   • Then remove leak_rollback_selftest from the metal request.

2. system/compositor/desktop/src/mode.rs:57-58 against kernel/src/pcidev/mod.rs:308-319: the compile-time "two of the largest fit" guarantee is weaker than it reads.

   • lend_room is first-fit on 2 MiB leaves, so a small scanout can split the 128-leaf window. Concrete case:
     • boot at 1280x800 (2 leaves, at 0–1);
     • flip to 7680x4320 (64 leaves, at 2–65), which frees 0–1;
     • flip to 7680x4300 (63 leaves).
   • The free runs are now 2 and 62 leaves, so the lend is refused ResourceExhausted. The flip is declined by name, although 2 * LARGEST.scanout_bytes() <= LEND_WINDOW holds.
   • The failure is clean (old mode stays, logged), and only modes above about 62 leaves reach it. But the assertion and its comment promise more than the placement gives, and the design's own fit argument rests on it.
   • Fix, cheapest first: say so at the assertion and in issues/a-lent-scanout-is-one-contiguous-run.md (the same owner and exit would cover it). Alternatively, make the assertion true by bounding the window at 2 * LARGEST plus the least mode's leaves.

3. kernel/src/drivers/virtio.rs:121: stale doc link.

   • /// Why [VirtioDevice::init] refused the device points at the deleted init, which is now a broken intra-doc link.
   • Fix: point it at offer/accept.

4. issues/a-refused-framebuffer-reads-as-no-display.md:31-40: the edited paragraph is no longer true.

   • The kernel no longer "rescues" the Blt-only shape. Your own guest-virtio-gpu.log:218 shows GPU: the firmware handed over no framebuffer on exactly the -vga none -device virtio-gpu-pci shape the issue describes.
   • So "That inference is from the branch, not measured: QEMU cannot stage a Blt-only GOP on a device this kernel has no driver for" is now false.
   • Fix: the kernel line is identical for both shapes, measured on HeadlessVirtioGpu. The compositor's claim draws the desktop, but it does not tell the kernel's record apart.

Suggestions (optional)

• tests/toyos.rs:452-454: the committed comment says as fact that VNC "gives QEMU's clipboard a peer the clipboard test's boot must not have"; the body says "may". The defensible reason is simply that -vnc changes the display configuration the clipboard row's premise (-display none) depends on. Say that, or measure the peer claim.
• The row's 1280x800 is QEMU's default xres/yres. Passing them explicitly in HeadlessVirtioGpu's -device would make the input controlled rather than inherited from a QEMU default.
• display.rs:294-295: Live::device_read32 bounds every access itself, so the num_scanouts read no longer guarantees anything about the event fields. The comment can go, and the read is only a log line.
• display.rs:57-65: when a virtio-gpu function is claimed but refused, the compositor falls back to a framebuffer if one exists. On a machine with both, that hides a broken driver behind an eprintln!. Consider ending instead, since the claim was given.
• mode.rs:133-138: two_of_the_largest_scanouts_fill_the_lend_window restates the compile-time assertion for constants. Not needed.
• panic_console/mod.rs:350-364: detach/rearm lost their only outside caller (the old gpu::set_resolution window). They can become private, and their doc ("until the next rearm") describes a window that no longer exists.
• issues/a-restarted-display-holder-lends-around-its-predecessors-scanouts.md names an exit but no owner (pcidev / the small-kernel track).

Verdict

SEND BACK. Items 1–4 are small. Item 1 also removes the leak_rollback_selftest metal row from the request, leaving gop_edid_scale as the one metal row before landing.

[Japabu]

Review round 1 at 1ab1d9917 (main 2c3f61a3e, still current):

1. Mint-rollback control removed: install_buffers is a single install. Also removed: mint_rollback_selftest, leak_selftest.rs, the leak-rollback-selftest actuator and both param-list entries, the judge and its metal row, install_all, stage_room and Region::empty. The staged leak_rollback_selftest run passed on both benches and proved nothing; the body says so.
2. Fit assertion: the reachable refusal is stated at mode.rs's assertion, and the sequence (1280x800 → 7680x4320 → 7680x4300) is recorded in issues/a-lent-scanout-is-one-contiguous-run.md, whose exit now also covers a window that serves any lend no larger than its free total.
3. Stale doc link: InitRefusal's doc names VirtioDevice::offer and Offered::accept.
4. Refused-framebuffer issue: now says what the virtio-gpu boot measured. The kernel line is identical for both shapes, and the compositor's claim draws the desktop without telling the kernel's record apart.

Suggestions taken: the VNC-boot comment now gives the configuration reason; HeadlessVirtioGpu passes xres=1280,yres=800; the num_scanouts comment and the redundant fit test are gone; a virtio-gpu function the compositor was given and cannot bring up now ends it; detach/rearm are private; the restarted-holder issue names its owner.

Rerun:

• --build-only, exit 0.
• --ci host, exit 0, 83 steps green.
• the_compositor_drives_the_virtio_gpu, the clipboard row and screen_fatal_halt_composited, 3/3 pass.

The driver crate is unchanged, so m1–m5 stand.

No bench rerun: gop_edid_scale stands from 883e496f7; on its path the only change is the mint's equivalent single install. The deleted row needs none.

Evidence: README.

[Japabu]

T14 gop_edid_scale at head 1ab1d9917: PASS, GOP handle 0, panel 309x173 mm, scale 1.5 (judge, readback, README).

The image sha256 is b266c38d…bb65949. This boot reads the round-1 framebuffer mint on metal. The AMD bench row stands from 883e496f7. Evidence commit 2b548798da4ce7a14385c1d46966bb196c7007f3.

[Japabu]

Merged main e136c77fd (#926, #927) at 9a1bff55e4769b70bcd9c8223913ba678e307d1a.

• Conflict: the only one was pcidev's imports, and LEND_WINDOW now sits beside A device function is claimed by its arrival, and a drives row has one process per function (lifecycle stage 1.2) #926's Arrival.
• Claim spelling: the compositor's claim is now in A device function is claimed by its arrival, and a drives row has one process per function (lifecycle stage 1.2) #926's grammar, pci:03:80:00/1af4:1050 (QEMU's virtio-gpu enumerates as class 0380), looked up with endow::claim. This applies in images/system.toml and tests/images/clipboardcase.toml.
• Checks: --build-only exit 0; --ci host exit 0 with 86 steps green; the_compositor_drives_the_virtio_gpu, usbd_drives_the_spare, bar_map_again, lent_device_memory_refused and a_drives_row_holds_one_process_per_function passed 5/5 (README, evidence 44a3d73603080a5ee1de1687cf1d8edd0d1439c9).
• No bench rerun: the merge does not touch the framebuffer mint path (object/device.rs, gop.rs and handle.rs are unchanged against 1ab1d9917; device.rs changes only the PCI arm), so the T14 gop_edid_scale result stands.