Repository navigation
pcidev: a BAR behind a bridge is placed only where every bridge on its path forwards - #928
Conversation
…s path forwards A claim moves each BAR it maps onto a 2 MiB window of its own, because a 2 MiB page is the only one this kernel maps into a process: at firmware's address the page holds other functions' registers too. The candidates came from the root's free runs alone, and those exclude every range a bridge forwards. So a function behind a bridge was only ever offered addresses its bridge does not forward. On the AMD bench, 03:00.4 (the xHCI behind the port that forwards 0xd0000000..0xd0700000) was moved onto eight addresses from 0x450200000 up and answered 0xffffffff at each. A q35 virtio NIC behind a pcie-root-port reproduces it on main: eight candidates above its port's 0xc000000000..0xc020000000, none answering, "NOT HANDED OVER". A function on a bus a bridge leads to now takes its candidates from the windows of the bridge whose secondary bus it is, of the BAR's kind (a non-prefetchable BAR only in the memory window, PCI-to-PCI Bridge Architecture §3.2.5.6 to §3.2.5.8), below 4 GiB for a 32-bit BAR, inside the root's declaration, and free of every BAR, every window cut or being tried, and the windows of every bridge not on the path. A candidate some bridge on the path does not forward is passed over unread. Where nothing is free the BAR stays where firmware put it and is handed to nobody, named NoRoutedRun: there it shares its page, so the reason for moving it still holds. `bar_map_behind_a_bridge` claims the NIC behind a root port on a new q35 profile and checks the placed address is inside what the port forwards; it fails on main's kernel. Not caused by #916, which changed no candidate: the root runs never held a bridge's windows. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016UzbDnM9m6nQWhbjGi6Lv1
… read it The host test's AMD bus 3 had the root's window for its port, and lacked 03:00.7's BARs; both are now the bench's own reading. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016UzbDnM9m6nQWhbjGi6Lv1
The interim places a BAR behind a bridge inside what its path forwards and names the case with no room; the move itself and the exit stay with P1. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016UzbDnM9m6nQWhbjGi6Lv1
|
Evidence: |
Review of #928 at
|
…it, and the issue names what remains - A bridge records the bus it sits on, and `bridge::path` walks from a function's bus to its root bus: each step is the one bridge whose secondary bus is the one reached, sitting on a lower bus, with the function's bus in its subordinate range. A bridge firmware never configured (both bus numbers 0) or one claiming every bus is no step, so no bus-0 function leaves the root runs on its word; two bridges naming one secondary bus refuse that bus's functions alone (`PathAmbiguous`). Host tests: the T14's bus-0 shape with such bridges, a two-level climb, the duplicate, and `Bridge::forwards`'s kind rule. - Candidates behind a bridge are cut from the firmware memory map too, as the root runs are. - A prefetchable BAR tries its port's prefetchable window first. - `trying` holds a candidate while it is probed and no longer; `cut` and `place_bar` say what they do for a candidate from either source; `NoRoutedRun` says only what is known. - The issue the interim refuted is renamed to the defect left, a BAR behind a bridge with no free 2 MiB of its windows, and `there-is-no-wifi` cites it by its new name. Before this branch every candidate behind a bridge lay outside it, since the root runs exclude every forwarded range. Merges origin/main e136c77 (#926), keeping both new q35 profiles. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016UzbDnM9m6nQWhbjGi6Lv1
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016UzbDnM9m6nQWhbjGi6Lv1
|
Review round 1 answered at
Evidence: |
Re-review of #928, round 2, at
|
The AMD bench's 03:00.4 and the T14's AX210 hold their MSI-X table in their one BAR, which a claim withholds, so each is refused NoMappableBar before placement; NoRoutedRun is what a claim handing that BAR over would meet. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016UzbDnM9m6nQWhbjGi6Lv1
|
Re-review answered at
|
pcidev: #928's bridge windows (`Bridge`, `msix`) beside this branch's table BAR placed like any other. On the AMD bench `03:00.4`'s one BAR, which holds its MSI-X table, now meets `NoRoutedRun` behind `00:08.1` and the claim is refused `NoMappableBar`; the issue #928 opened says so. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016UzbDnM9m6nQWhbjGi6Lv1
Status: draft. Head
317b1bb52(maine136c77fdmerged). Review round 1 is answered. Both bench rows passed at5321afa0c, before this round, and still stand (see "Metal"). This is the interim fix underissues/a-bar-behind-a-bridge-with-no-free-2-mib-of-its-windows-is-not-handed-over.md(renamed frompcidev-moves-a-bar-behind-a-bridge-out-of-the-range-the-bridge-forwards.md, which this change refutes); its exit stays with P1 ofissues/there-is-no-wifi.md.Problem
Before a hand-over, a claim moves each BAR it maps onto a 2 MiB window of its own. A 2 MiB page is the only page this kernel maps into a process, and at firmware's address that page also holds other functions' registers.
The candidates come from the root's free runs. Those runs exclude every range a bridge forwards (
publishpushesforwarded()into what is taken). So a function behind a bridge is only ever offered addresses its bridge does not forward. It answers0xffffffffat all eight, and the claim is refused.03:00.4behind00:08.1, which forwards0xd0000000..0xd0500000and, prefetchable,0xfc90000000..0xfca0200000): candidates0x450200000..0x451000000, none answering (#921's log).pcie-root-port(https://github.com/ToyOSOrg/evidence/blob/3f22d59a59e483946e0bfaf4f21d46a9ef5aa63c/r1/manual-boot-control-serial.log): main's kernel tries eight candidates from0xc020400000, all above the port's0xc000000000..0xc020000000, and refuses the NIC.Not #916's regression.
git diff 5b48a3d17 cc1a44c9b -- kernel/pci/src/placement.rs kernel/pci/src/bridge.rsis empty, and inpcidevit touched only the windows argument's spelling (&windowstowindows). The windows list is the loader's, as before.mainfor the AMD bench's AX200 (amdvi: the firmware-left actuator invalidates with one INVALIDATE_IOMMU_ALL, and its metal row judges AMD-Vi units #884's E13 run).amdvi_usbd_drives_its_own_entrydid not pass atc22745327. AMD-Vi stage 6: interrupt remapping, and claims only where every unit filters SMIs #921's own review records that the flash tool refused thexhci-leaveimage then, and only thetestcasesboot ran.main,03:00.4never reaches placement at all: its MSI-X table is in its one BAR, which a claim withholds, so it is refusedNoMappableBar. It reaches placement only on AMD-Vi stage 6: interrupt remapping, and claims only where every unit filters SMIs #921's branch, which maps that BAR.What changes
The path.
bridge::pathwalks from the function's bus up to its root bus (§3.2.5.3, §3.2.5.4).PathAmbiguous.Placement behind a bridge. A BAR whose path is non-empty takes its candidates from
placement::behind_bridges. A candidate there:Bridge::forwards):The probe that proves a placement is unchanged. A root-bus function takes its candidates from the root runs, as before.
One behaviour change on the root-bus path. A successful placement now gives back every candidate tried before it that the function did not answer at. Before this PR, such a root-run candidate stayed spent once a later one answered. Behind a bridge this keeps
tryingholding only what is being probed.placement::release, which returns only the address its run handed out last. So it can return a refused address only when that address came from a different run than the one that answered.No routed room. Where nothing is free, the BAR stays where firmware put it and is handed to nobody, logged as
keeps BAR n where firmware put it and hands it to nobody — no 2 MiB the bridges above it forward is free. At firmware's address it shares its 2 MiB page, so the reason for moving it still holds and it cannot be mapped. A function with no other mappable BAR is then refusedNoMappableBar.What stays until P1. The move itself stays until P1 (4 KiB user mappings for device windows, stage 3 of
issues/process-memory-is-2-mib-pages-and-that-caps-the-process-count.md). On the AMD bench, bus 3's window has no free 2 MiB, so03:00.4stays unclaimable under the 2 MiB rule, now with that reason named.Checks
Evidence:
pr-928at882142b5fcargo run -- --ci host503d19599usbd_drives_the_spare,bar_map_again,lent_device_memory_refused,iommu_virtio_platform,bar_map_behind_a_bridge503d19599usbd_drives_the_type_c_controller24ed5a756amdvi_translates_the_stick_controller24ed5a756cargo run -- --ci host88dab83e0usbd_drives_the_spare,bar_map_again,lent_device_memory_refused,iommu_virtio_platform, and the newbar_map_behind_a_bridge88dab83e0kernel/change reversed,bar_map_behind_a_bridge88dab83e0minus patchNOT HANDED OVER(log)main0xc000200000inside the port's window, handed over; main: refused (fix, main)--build-onlyx86_64 and aarch6488dab83e0behindafter24ed5a756(the bench's measured port window in the test data only)24ed5a756Why a guest test.
bar_map_behind_a_bridgeruns on a new q35 profile,HeadlessNicBehindPort. It is the only check that the kernel reads a real bridge's bus numbers and windows and lands the BAR inside them, and it fails onmain's kernel. The T14's claimed functions are on bus 0. The AMD bench's function behind a bridge has no room and holds its MSI-X table in its one BAR.Metal. Both rows passed at
5321afa0c, and this round does not change what they exercise.5321afa0chas aBAR n left 0x…line in its kernel log, so no probe failed and that path was not reached there.What judges the fix on metal.
mainclaims a function behind a bridge on either bench. The AMDtestcasesboot shows only that nothing regressed.03:00.4's one BAR holds its MSI-X table, which a claim withholds, so it is refusedNoMappableBarbefore placement. AMD-Vi stage 6: interrupt remapping, and claims only where every unit filters SMIs #921 hands that BAR over, so on top of this fix itsamdvi_usbd_drives_its_own_entryshould show03:00.4 keeps BAR 0 where firmware put it and hands it to nobody — no 2 MiB the bridges above it forward is free. It stays red until P1, by that name, not by candidates outside the bridge.Related records, not closed here
issues/nothing-reds-when-the-bar-survey-drops-an-assigned-bar.mdissues/no-boot-reaches-the-retry-that-undoes-a-placement-nothing-answered.mdBoth still rest on P1's placement.
🤖 Generated with Claude Code
https://claude.ai/code/session_016UzbDnM9m6nQWhbjGi6Lv1