Metal judges read the channel their line crosses on

[Japabu]

The full T14 run of main at 7e151819 (233 passed, 19 failed) reddened eleven rows. The fault in each was a judge or the harness being wrong about the machine. The triage in #615 found no OS defect. This branch fixes the seven causes the evidence makes certain and closes their issues.

Each fix has a host check fed readback lines from a T14 run, and a mutation that turns that check red. The recorded T14 run is the independent oracle, and the T14 run at f5c80264f is in the table below.

What changed, per cause

1. Readbacks' kernel records did not count as kernel output (klogd_hosted, loader_watchdog_arms, hda_tone). A /log readback's records open [<date> <time> <secs> cpuN], and only [kernel heads were counted. So every must_not_say and must_be_clean refused before it judged anything.
   • qemu::is_kernel_line is the harness's one definition of a kernel line. It now reads both heads: the console's [kernel …], and any line toyos_logstream::record_ms parses.
   • Serial::kernel_lines and died() both call it. So a readback's kernel PANIC: now reads as Died::Kernel.
   • Check: serial::self_check gains three /log readback rows (one of them programs-only, which must still refuse). It also gains two died() rows: a dated kernel PANIC: and a program's.
2. Judges read kernel.log for records only the page carries (machine_reboot, log_poll_outlives_a_close, usb_reset_records_the_phase_it_cut, boot_deadline_ends_a_wedge). The kernel writes these lines after init had the file made whole: Rebooting., wedge: staged, the arrived-deaf line and usb-load: sweeping. They cross only on the sealed page, in the pass after the reset.
   • The two reboot judges now ask bootlog::handed_back of that pass.
   • deadline_wedge_chain and usb_load_chain assert those records after Previous boot's panic:. The absences beside them move to the page too.
   • hard_lockup_chain no longer asserts that kernel.log lacks Rebooting., which never reaches /log.
   • Check: metal_judges_read_the_page_for_what_only_the_page_carries.
3. toyos-metal refused the hang the foreign-record arm stages (blackbox_foreign_record). Its pass after the reset clears the record as another image's and hands the machine back. The verdict is now a pure boot_verdict(armed, loader, log).
   • An image armed with FOREIGN_RECORD_ARM is not refused for HUNG_WITHOUT_A_RECORD.
   • In place of handed_back, it owes bootlog::FOREIGN_DONE: the loader's held a DONE record another image left in this memory. The kernel seals every state under the foreign identity, so only DONE says the stop finished. A stop that panicked or wedged is refused as Unfit::NoForeignDone.
   • The registration's judge asserts the same line.
   • That page is cleared, so its three boot.foreignrecord.* census rows are deleted.
   • Checks: metal::tests::the_foreign_record_arm_reaches_its_judge_through_the_hang_it_stages gives Ok(1171) armed and HungWithoutARecord unarmed. It gives NoForeignDone for a PANIC or WEDGED record. checks::the_foreign_record_judge_demands_the_stop_sealed_done checks the registered judge the same way.
   • The line is owed in the pass after the reset. bootlog::foreign_done reads from bootlog::SEPARATOR on, through bootlog::after_the_reset, and Readback::after_the_reset makes its cut with that function too. The pass before the handoff clears a stale foreign DONE record with the same line, so reading the whole file passed a stop that the pass after the reset read as PANIC.
   • metal::clears_its_own_page says which arm's page the pass after its reset clears. boot_verdict asks it, and so does the loop's metal::owes_nothing.
   • The QEMU half reads bootlog::FOREIGN_DONE as well. the_loader_writes_the_lines_the_host_reads holds that line to the loader's format, with State::Done's own word in its hole.
   • power::done_line spelled bootlog::HANDED_BACK a second time, from State::Done's word. It is deleted, and its four callers read the constant.
   • More checks: boot_verdict and the judge both refuse a stale DONE in the pass before the handoff when the pass after it read PANIC or WEDGED. The judge refuses a pass without HUNG_WITHOUT_A_RECORD. a_cleared_page_owes_no_fact_off_it checks owes_nothing.
   • boot_verdict refuses a chain that never went round: a stale foreign DONE and the handoff's last line, with no pass after the reset (unreturned).
4. xhci_xecp named no line the T14 prints, and judged one controller (xhci_xecp_walk). The judge now pairs every take_ownership outcome with the controller reset after it.
   • Each outcome must be one that hands the controller over: no capability, never claimed, or released.
   • Each must precede its own reset, and there must be as many outcomes as resets.
   • The three outcomes that leave firmware holding it red wherever they appear: unusable, runs past the register window, and still owns.
   • Check: the_xecp_judge_reads_the_t14s_handoff. One and two T14 controllers pass. A second controller that firmware kept reds, and so do a handoff with no reset and a reset with no handoff.
   • Two more cases red there. One is a controller that firmware kept and that was never reset, after a controller that was handed over ({t14}{kept}). The other is a handoff with no reset, followed by a paired one ({unreset}{t14}).
   • A third refuses a log in which the judge recognises no handoff at all (silent): the T14's lines with neither an outcome nor a reset, and the controller started.
5. allocator_stress bounded total memory to 2..=9 GB, which is a QEMU guest's size. The range is deleted, and 0 < used < total stays.
6. A shared metal chunk staged only the RUST_SKIP helpers its text names (dlopen_dedup). metal::reached now stages every binary a chunk's text spells test_rs_<name> that is not already one of its jobs. Check: a_shared_chunk_stages_every_binary_its_members_name.
   • It stages a binary only where the text names it whole: test_rs_std_tls_dlopen does not name test_rs_std_tls.
7. The guest C comparator compared the TinyCC warning line the host drops (03_struct). c_expectation drops it once, and both comparators read the result. Check: the_c_corpus_stages_the_expectation_the_host_compares.

Issues

This closes eight files:

• the-metal-wedge-judge-reads-a-channel-the-wedge-cannot-write
• a-readbacks-kernel-records-never-count-as-kernel-output
• a-metal-judge-reads-the-log-file-for-a-record-written-after-it-was-made-whole
• the-metal-loop-refuses-the-hang-the-foreign-record-arm-stages
• the-xecp-judge-names-no-line-the-t14s-handoff-prints
• allocator-stress-bounds-total-memory-by-a-qemu-guests-size
• a-shared-metal-chunk-stages-no-test-binary-another-member-reads
• the-guest-c-comparator-keeps-the-tinycc-warnings-the-host-one-drops

Seven of them name rows that PASS or exit 0 in the T14 run below. The eighth, a-readbacks-kernel-records-never-count-as-kernel-output, named four rows and asked that a T14 run reach each judge's own assertions. klogd_hosted and loader_watchdog_arms PASS. hda_tone and hda_client_stall now reach their own assertions and red on them, and each red is filed without a cause:

• hda_client_stall reds on soundd resumed 1 time(s). At 6737a442 it read soundd resumed 0 time(s). It is filed as issues/audio/hda-client-stall-reads-one-resume-where-its-judge-wants-two.md, with the lines measured.
• hda_tone reds on soundd filled 104 period(s) the tone client had not covered, on a client that keeps its ring full. On main the row reddened before it judged. It is filed as issues/audio/hda-tone-reads-underruns-on-the-t14-where-its-judge-wants-none.md, with the window measured.

The branch also files issues/build/allocator-stress-bounds-a-failed-reservation-by-sixteen-gib.md, and does not fix it.

These are left red, with their own issues: the LAN rows, fs_large_file, home_backing_revoked and the usbload boot's unpriced panel_* numbers.

T14 run at f5c80264f (the orchestrator's, full metal set)

The run ended 252 passed, 6 failed, 27 boot(s), and the toyos-build test binary run with --metal exited with exit status: 1 (EXIT=1). The six are lan_dhcp_lease, lan_talk, hda_tone, hda_client_stall, test_rs_fs_large_file and test_rs_home_backing_revoked. The lantalkcase boot and the usbload boot's two panel_* rows red as well.

row	result
klogd_hosted, loader_watchdog_arms	PASS
machine_reboot, log_poll_outlives_a_close, usb_reset_records_the_phase_it_cut	PASS
boot_deadline_ends_a_wedge, hard_lockup_ends_a_deaf_cpu	PASS
blackbox_foreign_record	PASS, and the foreignrecord boot has no FAIL
xhci_xecp_walk	PASS
allocator_stress, dlopen_dedup, 03_struct	===TEST_END … exit=0===, no FAIL
hda_tone	FAIL: soundd filled 104 period(s) the tone client had not covered, on a client that keeps its ring full (issue filed above)
hda_client_stall	FAIL: soundd resumed 1 time(s) — the second stream did not find a suspended daemon (issue filed above)
LAN rows, fs_large_file, home_backing_revoked, boot.usbload.panel_*	red, known, their own issues

The commit after f5c80264f adds host cases, deletes a doc clause, replaces power::done_line() by the bootlog::HANDED_BACK it spelled, and files an issue. It changes no line a metal judge asserts, so this run stands for the head.

Gates at 7b0ee7fd9

• cargo run -- --ci host: EXIT=0, Host: 54 step(s), all green.
• cargo test --test toyos-build -- --list: EXIT=0.
• cargo test --test toyos-checks: EXIT=0, 18 passed.

Negative controls

Each mutation below was a checked patch, applied onto a clean committed tree. It was built with --no-run (EXIT=0), run by the command shown, and then reversed. Exit 101 means the named test went red. The ten patches of the f5c80264f table and the two of the 7b0ee7fd9 table are in full in comments on this PR.

At 7b0ee7fd9:

mutation	command	exit	red with
xhci_xecp without its handoffs.is_empty() refusal	cargo test --test toyos-checks	101	tests/checks.rs:804, xhci_xecp(&silent).is_err(); the other 17 pass
foreign_done reads the whole loader.log: after_the_reset(loader).or(Some(loader))	cargo test --lib -- metal::tests::the_foreign_record_arm	101	src/metal.rs:3453, a chain that never went round gets Ok(1171)

The same two commands on the restored tree are EXIT=0.

At f5c80264f:

mutation	command	exit	red with
foreign_done reads the whole loader.log again	cargo test --lib -- metal::tests::the_foreign_record_arm	101	a stale DONE before the handoff gets Ok(1171)
xhci_xecp without the KEPT check	cargo test --test toyos-checks -- the_xecp_judge	101	{t14}{kept} is Ok
pending = Some(line);	cargo test --test toyos-checks -- the_xecp_judge	101	{unreset}{t14} is Ok
reached matches a substring	cargo test --test toyos-checks -- a_shared_chunk	101	test_rs_std_tls_dlopen stages bin/test_rs_std_tls
owes_nothing without its cleared-page arm	cargo test --test toyos-checks -- a_cleared_page	101	the foreign-identity boot owes panel_us
clears_its_own_page answers armed.is_empty()	cargo test --lib -- metal::tests::the_foreign_record_arm, and cargo test --test toyos-checks -- a_cleared_page	101, 101	the armed boot gets HungWithoutARecord; the foreign-identity boot owes panel_us
the loader's line says left in memory	cargo test --lib -- bootlog::tests::the_loader_writes_the_lines_the_host_reads	101	the loader formats no held a {} record another image left in this memory
State::Done is named FINISHED	cargo test --lib -- bootlog::tests::the_loader_writes_the_lines_the_host_reads	101	the loader formats no held a DONE record …
the judge without must_say(HUNG_WITHOUT_A_RECORD)	cargo test --test toyos-checks -- the_foreign_record_judge	101	a pass without that line is Ok
the judge reads b[0].loader()	cargo test --test toyos-checks -- the_foreign_record_judge	101	a stale DONE before the handoff is Ok

At b4e752c12, each mutation put back the base's code at the site:

mutation	command	exit	red with
boot_verdict back to if !staged_hang { handed_back }	cargo test --lib -- metal::tests::the_foreign_record_arm	101	the PANIC pass gets Ok(1171)
foreign judge back to must_say("record another image left in this memory")	cargo test --test toyos-checks -- the_foreign_record_judge	101	the PANIC pass is Ok
xhci_xecp whole, as at 6737a442	cargo test --test toyos-checks -- the_xecp_judge	101	xhci_xecp(&format!("{t14}{held}")).is_err() fails
is_kernel_line without the record_ms arm	cargo test --test toyos-checks -- serial_vocabulary	101	the test capture carried no kernel output at all

The round before it, on sites that round did not touch, built each mutation with cargo test --no-run (EXIT=0), and each run went red with EXIT=101:

• machine_reboot and log_close_survived back to kernel().must_say(REBOOTING).
• deadline_wedge_chain and usb_load_chain with their call sites back to the base's.
• The hang refusal without the arm's exemption.
• reached gated on RUST_SKIP again.
• The corpus staging the raw .expect again.

🤖 Generated with Claude Code

[Japabu]

Review of #616 at 6737a4427. CI host is green at this head (run 36572268983, conclusion success). There is no T14 reading at this head yet; the orchestrator's run is owed (see BLOCKER 3).

Net: 18 files, +352 −461. Code is +331 −129. About +183 of that is tests (tests/checks.rs +134, metal::tests +32, serial::self_check +16). The harness's production code is about +148 −128. Issues are +21 −330.

BLOCKER

• src/metal.rs:2149-2152 + tests/toyos.rs:1916-1926. The foreign-record arm is exempted from handed_back, and nothing replaces what that check proved: that the stop sealed DONE.
  • kernel/src/blackbox.rs:229 seals every state (panic, wedge, done) under the foreign identity.
  • The loader's line names the state it found: held a {state} record another image left (bootloader/src/blackbox.rs:153).
  • The QEMU half demands DONE in that line (tests/common/power.rs:1666). The metal judge and boot_verdict do not.
  • So a stop that panics, or that the deadline ends, after init's STOPPING line passes both.
  • Patch that must go red: in the_foreign_record_arm_reaches_its_judge_through_the_hang_it_stages, add let panicked = loader.replace("held a DONE record", "held a PANIC record"); assert!(boot_verdict(&armed(&[FOREIGN_RECORD_ARM, "boot-deadline=120000"]), &panicked, log).is_err());. It is Ok(1171) today.
  • Fix: the staged-hang branch demands held a DONE record another image left in place of handed_back, and the judge asserts the same line.
• tests/toyos.rs:14601. xhci_xecp judges only the first handoff line and the first controller reset. Every T14 boot prints two controllers (main's T14 log, selftests boot, lines 12949 and 12970).
  • A second controller that firmware keeps (still owns … resetting it anyway), or whose list is unusable, or that runs past the register window, passes.
  • Patch that must go red: in the_xecp_judge_reads_the_t14s_handoff, add assert!(xhci_xecp(&format!("{t14}{held}")).is_err());. It is Ok(()) today.
  • Fix: every take_ownership outcome line is one of HANDED_OVER and precedes its own reset, and there are as many of them as there are resets.
• issues/ (eight deleted files). Every exit condition reads "a T14 run … then this file is deleted", and no T14 run exists at this head. Each deletion stands only on its rows being PASS in the orchestrator's T14 run at the head that lands. Any file whose rows are not PASS is restored:
  • a-readbacks-kernel-records…: klogd_hosted, loader_watchdog_arms, hda_tone, hda_client_stall.
  • a-metal-judge-reads-the-log-file…: machine_reboot, log_poll_outlives_a_close, usb_reset_records_the_phase_it_cut.
  • hardware/the-metal-wedge-judge…: boot_deadline_ends_a_wedge.
  • the-metal-loop-refuses-the-hang…: blackbox_foreign_record PASS, and the foreignrecord boot facts with no FAIL.
  • the-xecp-judge…: xhci_xecp_walk.
  • allocator-stress-bounds-total-memory…: allocator_stress exits 0 on its shared chunk.
  • a-shared-metal-chunk…: dlopen_dedup exits 0.
  • the-guest-c-comparator…: 03_struct exits 0 on ccorpus.
  • The two fixes above change the foreignrecord and xhci_xecp_walk code, so those two rows must be read again at the fixed head. Keeping that run's readback directory lets the judges be replayed on it without another boot.

NOTE

• tests/common/metal.rs:1145 and src/metal.rs:2123. Whether an arm clears its own page is decided twice from FOREIGN_RECORD_ARM (cleared and staged_hang). One pub fn beside stages_a_wedge should answer it for both.
  • Deleting cleared || leaves every host check green. Only the T14 foreignrecord boot facts would catch it.
  • The deleted boot.foreignrecord.{panel_max_us,panel_us,park_open_operations} rows cost no measurement. That page is cleared before its census is read, and the same image's jobcase boot still prices all three.
• tests/common/metal.rs:695 (reached). The match is a substring, so test_rs_std_tls rides every chunk whose text says test_rs_std_tls_dlopen. Refusing a match followed by [a-z0-9_] makes it exact with one check.
  • Reach is not transitive: a staged helper's own source is never read. A miss shows up only as entity not found on the machine. That was already true before this branch.
• tests/common/serial.rs:244 vs :140. kernel_lines now reads record_ms, while died() still picks its column with is_kernel_line. The DEATHS doc calls is_kernel_line "the harness's one definition of that".
  • On a readback, a kernel PANIC: therefore reads as Died::Panicked.
  • must_be_clean is unaffected, because it reads the by_kernel column for every line. Still, one module now has two definitions of a kernel line.
• tests/toyos-rust-tests/src/bin/allocator_stress.rs:267. 0 < used < total still catches accounting that stopped (used 0) or wrapped (used ≥ total). A wildly wrong total now passes. Nothing on the guest side knows the machine's size, so I accept that.
  • No gate compiles this edit at 6737a44. The T14 image build is its first compile.
• tests/toyos.rs:1938 and :2427. In live runs, handed_back in machine_reboot and log_close_survived repeats a refusal boot_verdict already made. It adds a check only in the offline readback mode. machine_reboot's own claim is reset_register_decoded.

REMOVE

• src/metal.rs:2151: "The staged hang's pass reads no DONE". That is false: the pass reads a DONE record that another image left.
• tests/checks.rs:742-743: "and main's T14 run cut the shared list so that std_tls rode the other chunk". This is investigation history.
• tests/checks.rs:658, :721, :759, src/metal.rs:3393 and tests/common/serial.rs:426-427: the "verbatim from main's T14 run" wording and "whose klogd_hosted was refused …". This is provenance and belongs in the commit message.
• tests/toyos.rs:1907-1909: a second comment directly under :1904 that says the same thing.

SEND BACK

🤖 Generated with Claude Code

https://claude.ai/code/session_01U6SVYFkdvV2t38KzNrESxs

[Japabu]

Re-review of #616 at 0d289ec78 (ace064f9d merged into b4e752c12). CI host at this head is green (run 36618203190, conclusion success); it runs cargo test --lib and --test toyos-checks, which hold every check this branch adds.

Net: 21 files, +503 −481. Harness code is +207 −148; the +59 net pays for seven fixes, and I accept it. Host checks (tests/checks.rs, tests/checks/serial.rs, metal::tests) are +225 −1. Issues are +71 −332.

The merge's port is whole. tests/checks/serial.rs gains exactly what b4e752c12 carried in tests/common/serial.rs: the two /log captures, the three cases and the two died() rows. tests/common/serial.rs is now main's byte for byte, and is_kernel_line keeps this branch's body under its own doc.

Earlier round (6737a4427)

• BLOCKER 1 (the foreign arm owed nothing in place of handed_back): CLOSED. Reverting boot_verdict went red, and so did reverting the judge, each with exit 101. The commands were cargo test --lib -- metal::tests::the_foreign_record_arm and cargo test --test toyos-checks -- the_foreign_record_judge. The new scope hole is BLOCKER 1 below.
• BLOCKER 2 (xhci_xecp judged one controller): CLOSED. xhci_xecp as at 6737a442 reds the_xecp_judge_reads_the_t14s_handoff with exit 101. Its new arms are BLOCKER 2 below.
• BLOCKER 3 (the eight deletions stand on landing-head rows):
  • CLOSED for the rows the T14 at 6737a442 read PASS. hda_client_stall is re-filed under its new cause, which the old file's exit condition ("reaches each judge's own assertions") allows.
  • OPEN for blackbox_foreign_record and xhci_xecp_walk. Both judges changed after that run. The body's replay of them names no command, and the offline replay on record exited 2: it staged images and judged nothing, because its readback directory was empty.
  • Both close on the orchestrator's T14 at the landing head.
• NOTE, cleared and staged_hang decided twice: OPEN. The body lists it as unsure; it is neither fixed nor filed.
• NOTE, reached matches a substring: OPEN. Same as above.
• NOTE, kernel_lines vs died(): CLOSED. There is one is_kernel_line, and a dated PANIC: row pins it.
• NOTE, no gate compiled allocator_stress: CLOSED. cargo run -- --build-only exits 0 at b4e752c12.
• NOTE, handed_back repeated in two judges: stands, and nothing is asked. It is the one hand-back check when readbacks are judged offline.
• REMOVEs src/metal.rs:2151, tests/checks.rs:742-743, the "verbatim from main's T14 run" and "whose klogd_hosted was refused" provenance, and tests/toyos.rs:1907-1909: CLOSED, each deleted.

BLOCKER

• src/bootlog.rs:369 + src/metal.rs:2152: foreign_done searches the whole loader.log, so the pass before the handoff can satisfy it.
  • That pass prints FOREIGN_DONE for a stale foreign DONE record it clears, which is the defect this arm stages. tests/common/metal.rs:388 already says an earlier chain's report can sit in that pass.
  • With it in that pass, a stop whose own record the pass after the reset read as PANIC or WEDGED gets Ok, and toyos-metal exits 0.
  • The registered judge reads after_the_reset() and still reds. toyos-metal's own claim fails ("a stop that panicked or wedged is refused as NoForeignDone").
  • handed_back is not exposed the same way. Its DONE can only be this image's own record, and a pass that finds one ends the chain before any handoff.
  • Patch that must go red, inside the for state loop of metal::tests::the_foreign_record_arm_reaches_its_judge_through_the_hang_it_stages: let stale = format!("Black box: 0x8000000 held a DONE record another image left in this memory\n{ended}"); assert_eq!(boot_verdict(&armed(&[FOREIGN_RECORD_ARM, "boot-deadline=120000"]), &stale, log), Err(Refusal::Log(bootlog::Unfit::NoForeignDone)));. Today it is Ok(1171).
  • Fix: foreign_done reads from bootlog::SEPARATOR on, the same cut after_the_reset makes.
• tests/toyos.rs:14752 and :14756: no test can fail on two arms of the rewritten xhci_xecp. Every held case reds through the pairing, not through KEPT.
  • Delete the KEPT check (:14752-14754). cargo test --test toyos-checks -- the_xecp_judge stays green. Case that must go red, a controller that firmware kept and whose reset was refused: let kept = held.replace("xHCI: controller reset", "xHCI: 34 scratchpad buffers configured"); assert!(xhci_xecp(&format!("{t14}{kept}")).is_err());
  • Replace if let Some(earlier) = pending.replace(line) { return Err(..) } with pending = Some(line);. The check stays green. Case that must go red: assert!(xhci_xecp(&format!("{unreset}{t14}")).is_err());. Only the trailing unreset handoff is tested today, not one in the middle of the log.

NOTE

• tests/common/metal.rs:1145 + src/metal.rs:2123: two readers still decide which arm clears its own page. One pub fn beside stages_a_wedge should answer for both. Deleting cleared || leaves every host check green.
• tests/common/metal.rs:708: reached still matches a substring, so test_rs_std_tls rides any text that says test_rs_std_tls_dlopen. Refuse a match followed by [a-z0-9_].
• tests/common/power.rs:1665-1670 vs src/bootlog.rs:123: the QEMU half reads the same loader line with its own needle plus State::Done.named().
  • the_loader_writes_the_lines_the_host_reads (src/bootlog.rs:644) holds every loader phrase except FOREIGN_DONE.
  • So the two halves can now disagree about one log, which bootlog's module doc forbids.
  • Fix: the QEMU half reads bootlog::FOREIGN_DONE, and the gate holds FOREIGN_DONE to the loader's format and State::Done.named().
• tests/toyos.rs:1935: no test pins the judge's HUNG_WITHOUT_A_RECORD assertion. the_foreign_record_judge_demands_the_stop_sealed_done never feeds it a pass without that line. Add a case: loader.replace(bootlog::HUNG_WITHOUT_A_RECORD, "") must be Err.
• tests/common/power.rs:1993: hard_lockup_chain asserts REBOOTING is absent from kernel.log, where src/bootlog.rs:13-16 says it never appears. This branch removed the same vacuous absence from the chain's two siblings. Move it to after, or delete it.
• tests/toyos.rs:12831: the QEMU registration shares the rewritten xhci_xecp. It is weekly tier, so no CI run reaches it, and the body shows no QEMU run of it since the rewrite.
• Metal timings are recorded per machine and judged against that machine's own record #630 (head db5e59bc4) will merge this branch. Four places need care:
  • Its boot-facts loop fails any missing value that is not path-taken (tests/common/metal.rs:1099), and it has no cleared arm. This branch is what lets foreignrecord reach that loop at all. Without a ported cleared, that boot reds on boot.foreignrecord.panel_max_us and panel_us. The pub fn from the first NOTE would make this one call.
  • Its machine and numbers fields (:838-840) have to move into Readback::new, which the new checks call.
  • It still passes helpers through build and run (:624, :886); reached replaces that.
  • Its boot_file edit meets boot_verdict at the tail of metal::run.

REMOVE

• tests/toyos.rs:1929: "Named and cleared". Nothing in the judge asserts the clear, and the line was rewritten rather than deleted.
• PR body, "Two judges changed after that run…" and its table: a replay with no command and no code.
• PR body, "What I am unsure of": two of its bullets are open NOTEs, which get fixed or filed in issues/, not carried in main's merge commit. The tree answers the third: klogd's only console head is [kernel , and no loader line opens with [.

SEND BACK

[Japabu]

Negative controls for the third review, at f5c80264f. Each patch below was checked with git apply --check, applied with git apply onto the clean committed tree, and built with the target's --no-run, which exited 0 every time. It was then run by the command shown and reversed with git apply -R. git status --porcelain was empty after each reversal. Exit 101 means the named test went red on the assertion shown.

patch	command	build	run	red at
01 foreign_done reads the whole loader.log (its body at 0d289ec78)	cargo test --lib -- metal::tests::the_foreign_record_arm	0	101	src/metal.rs:3443, the review's stale-pass case: left: Ok(1171), right: Err(Log(NoForeignDone))
02 xhci_xecp without the KEPT check	cargo test --test toyos-checks -- the_xecp_judge	0	101	tests/checks.rs:795, xhci_xecp(&format!("{t14}{kept}")).is_err()
03 pending = Some(line);	cargo test --test toyos-checks -- the_xecp_judge	0	101	tests/checks.rs:800, xhci_xecp(&format!("{unreset}{t14}")).is_err()
04 reached matches a substring	cargo test --test toyos-checks -- a_shared_chunk	0	101	tests/checks.rs:823, left: ["bin/test_rs_std_tls", "lib/libfoo.so"], right: ["lib/libfoo.so"]
05 owes_nothing without its cleared-page arm (the review's cleared ||)	cargo test --test toyos-checks -- a_cleared_page	0	101	tests/checks.rs:768, the foreign-identity boot owes panel_us
06 clears_its_own_page answers armed.is_empty()	cargo test --lib -- metal::tests::the_foreign_record_arm and cargo test --test toyos-checks -- a_cleared_page	0 and 0	101 and 101	src/metal.rs:3423, left: Err(HungWithoutARecord), right: Ok(1171); tests/checks.rs:768
07 the loader's line says left in memory	cargo test --lib -- bootlog::tests::the_loader_writes_the_lines_the_host_reads	0	101	src/bootlog.rs:679, formats no "held a {} record another image left in this memory"
08 State::Done is named FINISHED	cargo test --lib -- bootlog::tests::the_loader_writes_the_lines_the_host_reads	0	101	src/bootlog.rs:679, formats no "held a DONE record another image left in this memory"
09 the judge without after.must_say(bootlog::HUNG_WITHOUT_A_RECORD)	cargo test --test toyos-checks -- the_foreign_record_judge	0	101	tests/checks.rs:760, the pass without that line
10 the judge reads b[0].loader() in place of after_the_reset()	cargo test --test toyos-checks -- the_foreign_record_judge	0	101	tests/checks.rs:757, the stale-pass case

Patch 07's build compiles the host lib, whose gate reads the loader's source as text; it does not compile the loader. Patch 08's build compiles the mutated toyos-blackbox.

01-foreign-done-reads-the-whole-loader-log.patch

diff --git a/src/bootlog.rs b/src/bootlog.rs
index 8ac0fe2fa..c3b810f8b 100644
--- a/src/bootlog.rs
+++ b/src/bootlog.rs
@@ -367,10 +367,10 @@ pub fn handed_back(loader: &str) -> Result<(), Unfit> {
 /// The loader's half of a passing foreign-identity boot, whose own chain the
 /// loader ends as a hang: the record it cleared was sealed `DONE`.
 pub fn foreign_done(loader: &str) -> Result<(), Unfit> {
-    // The pass before the handoff clears a stale foreign record the same way.
-    match after_the_reset(loader) {
-        Some(after) if after.contains(FOREIGN_DONE) => Ok(()),
-        _ => Err(Unfit::NoForeignDone),
+    if loader.contains(FOREIGN_DONE) {
+        Ok(())
+    } else {
+        Err(Unfit::NoForeignDone)
     }
 }
 

02-xhci-xecp-without-the-kept-check.patch

diff --git a/tests/toyos.rs b/tests/toyos.rs
index 70dc6817c..d5b2eefb1 100644
--- a/tests/toyos.rs
+++ b/tests/toyos.rs
@@ -14747,9 +14747,6 @@ fn xhci_xecp(log: &str) -> Result<(), String> {
         let mut handoffs = Vec::new();
         let mut pending: Option<&str> = None;
         for line in log.lines() {
-            if KEPT.iter().any(|said| line.contains(said)) {
-                return Err(format!("a controller was never handed over: {line}\n{log}"));
-            }
             if HANDED_OVER.iter().any(|said| line.contains(said)) {
                 if let Some(earlier) = pending.replace(line) {
                     return Err(format!("a handoff with no reset of its own: {earlier}\n{log}"));

03-xhci-xecp-overwrites-a-pending-handoff.patch

diff --git a/tests/toyos.rs b/tests/toyos.rs
index 70dc6817c..d632876c1 100644
--- a/tests/toyos.rs
+++ b/tests/toyos.rs
@@ -14751,9 +14751,7 @@ fn xhci_xecp(log: &str) -> Result<(), String> {
                 return Err(format!("a controller was never handed over: {line}\n{log}"));
             }
             if HANDED_OVER.iter().any(|said| line.contains(said)) {
-                if let Some(earlier) = pending.replace(line) {
-                    return Err(format!("a handoff with no reset of its own: {earlier}\n{log}"));
-                }
+                pending = Some(line);
             } else if line.contains("xHCI: controller reset") {
                 let Some(handoff) = pending.take() else {
                     return Err(format!("a controller reset before its handoff: {line}\n{log}"));

04-reached-matches-a-substring.patch

diff --git a/tests/common/metal.rs b/tests/common/metal.rs
index c2254eeb0..87245e8ad 100644
--- a/tests/common/metal.rs
+++ b/tests/common/metal.rs
@@ -701,13 +701,6 @@ pub fn reached(
     jobs: &[String],
     rust_bins: &[(String, Vec<u8>)],
 ) -> Vec<(String, Vec<u8>)> {
-    // Whole: a longer name that opens with this one is another binary's.
-    let names = |staged: &str| {
-        reachable.match_indices(staged).any(|(at, _)| {
-            !reachable[at + staged.len()..]
-                .starts_with(|c: char| c.is_ascii_lowercase() || c.is_ascii_digit() || c == '_')
-        })
-    };
     let mut out = Vec::new();
     for (name, data) in rust_bins {
         let staged = format!("test_rs_{name}");
@@ -716,7 +709,7 @@ pub fn reached(
         // together are a fraction of one helper binary.
         if name.ends_with(".so") {
             out.push((format!("lib/{name}"), data.clone()));
-        } else if !jobs.contains(&staged) && names(&staged) {
+        } else if !jobs.contains(&staged) && reachable.contains(&staged) {
             out.push((format!("bin/{staged}"), data.clone()));
         }
     }

05-a-cleared-page-owes-every-fact.patch

diff --git a/tests/common/metal.rs b/tests/common/metal.rs
index c2254eeb0..5c0d4a6d7 100644
--- a/tests/common/metal.rs
+++ b/tests/common/metal.rs
@@ -35,7 +35,7 @@ const PATH_TAKEN: &[&str] =
 /// none and the profile prices none: a path it did not take, or a field that
 /// crosses only on a page the pass after its reset cleared.
 pub fn owes_nothing(field: &str, params: &[&str]) -> bool {
-    PATH_TAKEN.contains(&field) || toyos_build::metal::clears_its_own_page(params)
+    PATH_TAKEN.contains(&field)
 }
 
 /// One boot a metal test needs.

06-clears-its-own-page-answers-no.patch

diff --git a/src/metal.rs b/src/metal.rs
index 3b7966c28..3a6dfa7eb 100644
--- a/src/metal.rs
+++ b/src/metal.rs
@@ -879,7 +879,7 @@ pub fn stages_a_wedge(armed: &[String]) -> bool {
 /// as another image's: nothing crosses on the page but the record's state, and
 /// that pass hands the machine back as a hang.
 pub fn clears_its_own_page(armed: &[impl AsRef<str>]) -> bool {
-    armed.iter().any(|name| name.as_ref() == FOREIGN_RECORD_ARM)
+    armed.is_empty()
 }
 
 /// [`FLASHABLE`]'s ruling on `name`, or `None` where nobody has made one.

07-the-loader-rewords-the-foreign-record-line.patch

diff --git a/bootloader/src/blackbox.rs b/bootloader/src/blackbox.rs
index 0f7498103..87ad2c441 100644
--- a/bootloader/src/blackbox.rs
+++ b/bootloader/src/blackbox.rs
@@ -150,7 +150,7 @@ pub fn harvest(
         return (
             None,
             Some(alloc::format!(
-                "{HEAD} {PHYS:#x} held a {} record another image left in this memory \
+                "{HEAD} {PHYS:#x} held a {} record another image left in memory \
                  ({was:02x?}, and this stick is {identity:02x?}), armed at {}. It has been \
                  cleared and this pass boots its kernel",
                 state.named(),

08-state-done-is-another-word.patch

diff --git a/toyos-blackbox/src/lib.rs b/toyos-blackbox/src/lib.rs
index b2e5653a1..ccf34beef 100644
--- a/toyos-blackbox/src/lib.rs
+++ b/toyos-blackbox/src/lib.rs
@@ -165,7 +165,7 @@ impl State {
         match self {
             Self::Armed => "ARMED",
             Self::Panic => "PANIC",
-            Self::Done => "DONE",
+            Self::Done => "FINISHED",
             Self::Wedged => "WEDGED",
             Self::Fault => "FAULT",
         }

09-the-judge-asks-no-hang.patch

diff --git a/tests/toyos.rs b/tests/toyos.rs
index 70dc6817c..185c5b7f1 100644
--- a/tests/toyos.rs
+++ b/tests/toyos.rs
@@ -1929,8 +1929,6 @@ const METAL: &[(&str, metal::Metal)] = &[
                 let said = after.must_say(bootlog::FOREIGN_DONE)?.to_string();
                 power::says_nothing_of(&after, bootlog::PREVIOUS_PANIC)?;
                 power::says_nothing_of(&after, "the last boot read")?;
-                // The hang `toyos-metal` admits for this arm, and only this one.
-                after.must_say(bootlog::HUNG_WITHOUT_A_RECORD)?;
                 eprintln!("  [power] {}", said.trim());
                 Ok(())
             },

10-the-judge-reads-the-whole-loader-log.patch

diff --git a/tests/toyos.rs b/tests/toyos.rs
index 70dc6817c..76ad2fc8c 100644
--- a/tests/toyos.rs
+++ b/tests/toyos.rs
@@ -1925,7 +1925,7 @@ const METAL: &[(&str, metal::Metal)] = &[
                 &[],
             )],
             judge: |b| {
-                let after = b[0].after_the_reset()?;
+                let after = b[0].loader();
                 let said = after.must_say(bootlog::FOREIGN_DONE)?.to_string();
                 power::says_nothing_of(&after, bootlog::PREVIOUS_PANIC)?;
                 power::says_nothing_of(&after, "the last boot read")?;

[Japabu]

Delta re-review of #616, 0d289ec78..f5c80264f. CI host at f5c80264f is green: run 36624818667, job host conclusion success, its step cargo run -- --ci host success.

Net +570 −492 over 21 files: src/ outside its test modules +66 −14, harness +169 −145, host checks +264 −1, issues +71 −332. This delta is +85 −29: src/ +18 −5, harness +23 −19, checks +44 −5. Every production line it adds is a fix the last review asked for, and I accept the growth.

Re-planted in a scratch worktree detached at f5c80264f, made and removed with cargo run -- --worktree. Each patch is the implementer's, taken byte for byte from the patch comment. For each: git apply --check 0, applied, --no-run build 0, run, git apply -R, git status --porcelain empty.

patch	command	exit	red at
01	cargo test --lib -- metal::tests::the_foreign_record_arm	101	src/metal.rs:3443, left: Ok(1171), right: Err(Log(NoForeignDone))
02	cargo test --test toyos-checks -- the_xecp_judge	101	tests/checks.rs:795, xhci_xecp(&format!("{t14}{kept}")).is_err()
05	cargo test --test toyos-checks -- a_cleared_page	101	tests/checks.rs:768, metal::owes_nothing("panel_us", …)

Earlier findings

• The 0d289ec78 review's BLOCKER, foreign_done read the whole loader.log: CLOSED. Patch 01, its body at 0d289ec78, reds above. The judge's twin, patch 10, is 101 in the implementer's run.
• The 0d289ec78 review's BLOCKER, two xhci_xecp arms no check could fail: CLOSED. Patch 02 reds above, and patch 03 is 101 at tests/checks.rs:800 in the implementer's run. The rewrite has a third such arm, the BLOCKER below.
• The 6737a4427 review's BLOCKER 3, for blackbox_foreign_record and xhci_xecp_walk: OPEN until the orchestrator's T14 reads both rows PASS. Nothing asked below changes what a judge does, so a run at f5c80264f stands for the landing head.
• NOTE, two readers decide which arm clears its page: CLOSED. clears_its_own_page answers boot_verdict and the loop, and patch 05 reds above. f5c80264f belongs here: it closes the half a18220a06 left, the loop's cleared || that no check reached.
• NOTEs on reached matching a substring, the QEMU half's own needle, and the unpinned HUNG_WITHOUT_A_RECORD: CLOSED. Patches 04, 07, 08 and 09 are each 101 in the implementer's run.
• NOTE, hard_lockup_chain's vacuous absence: CLOSED, deleted.
• NOTE, no QEMU run of the rewritten xhci_xecp (tests/toyos.rs:12829): OPEN. It is the orchestrator's QEMU landing run.
• REMOVEs "Named and cleared", the body's replay and its table, and "What I am unsure of": CLOSED, each deleted.

BLOCKER

• tests/toyos.rs:14767-14769 — no check can fail on the rewrite's third arm, if handoffs.is_empty(). It sits outside this delta; the 0d289ec78 review named two of the three. It is the judge's one refusal of a log in which it recognises no handoff, the successor of the "no line about the handoff at all" the deleted issue measured. Deleting the three lines keeps cargo test --test toyos-checks at exit 0 (18 passed), measured at f5c80264f. Add after tests/checks.rs:802: let silent = unhanded.replace("xHCI: controller reset", "xHCI: 34 scratchpad buffers configured"); assert!(xhci_xecp(&silent).is_err());. Measured: 101 at :804 with the lines deleted, 0 with them in place.

NOTE

• src/bootlog.rs:371 — no case reaches foreign_done's None arm. With match after_the_reset(loader).or(Some(loader)), cargo test --lib (401 passed) and --test toyos-checks (18 passed) both exit 0. toyos-metal then passes a chain that never went round, on the stale DONE its pass before the handoff cleared. Add after the for state loop of the_foreign_record_arm_reaches_its_judge_through_the_hang_it_stages: let unreturned = format!("Black box: 0x8000000 held a DONE record another image left in this memory\n{}\n", bootlog::LOADER_LAST_LINE); assert_eq!(boot_verdict(&armed(&[FOREIGN_RECORD_ARM, "boot-deadline=120000"]), &unreturned, log), Err(Refusal::Log(bootlog::Unfit::NoForeignDone)));. Measured: 101 at src/metal.rs:3452 under that patch, 0 without it.
• tests/common/power.rs:968 — done_line() spells bootlog::HANDED_BACK a second time, from State::Done.named(). That is the two-halves split this branch closed for FOREIGN_DONE. Its four callers read bootlog::HANDED_BACK, and it is deleted.

REMOVE

• tests/common/metal.rs:35-36 — ": a path it did not take, or a field that crosses only on a page the pass after its reset cleared" — false. The cleared arm ignores field: owes_nothing("complete_ms", &[FOREIGN_RECORD_ARM]) is true.
• PR body, cause 3 — ", which excuses the boot facts that cross only on that page" — false, for the same reason.

SEND BACK

[Japabu]

Negative controls for the fourth review, at 7b0ee7fd9. Each patch below was checked with git apply --check (exit 0), applied with git apply onto the clean committed tree, and built with the target's --no-run, which exited 0 both times. It was then run by the command shown and reversed with git apply -R (exit 0). git status --porcelain was empty before each patch and after each reversal. Exit 101 means the named test went red on the assertion shown. The green arm is the same command on the restored tree.

patch	command	build	run	red at	green arm
01 xhci_xecp without its handoffs.is_empty() refusal (the review's BLOCKER)	cargo test --test toyos-checks	0	101	tests/checks.rs:804, assertion failed: xhci_xecp(&silent).is_err(); the other 17 checks pass	0, 18 passed
02 foreign_done reads the whole loader.log where the chain never went round, after_the_reset(loader).or(Some(loader)) (the review's NOTE)	cargo test --lib -- metal::tests::the_foreign_record_arm	0	101	src/metal.rs:3453, left: Ok(1171), right: Err(Log(NoForeignDone))	0, 1 passed

01-xecp-without-the-no-handoff-refusal.patch

--- a/tests/toyos.rs
+++ b/tests/toyos.rs
@@ -14764,9 +14764,6 @@
         if let Some(unreset) = pending {
             return Err(format!("a handoff with no reset of its own: {unreset}\n{log}"));
         }
-        if handoffs.is_empty() {
-            return Err(format!("no controller was handed over and reset:\n{log}"));
-        }
         // A controller that still enumerates its bus afterwards.
         if !log.contains("xHCI: controller started") {
             return Err(format!("the controller did not come up:\n{log}"));

02-foreign-done-reads-the-whole-loader-when-the-chain-never-went-round.patch

--- a/src/bootlog.rs
+++ b/src/bootlog.rs
@@ -368,7 +368,7 @@
 /// loader ends as a hang: the record it cleared was sealed `DONE`.
 pub fn foreign_done(loader: &str) -> Result<(), Unfit> {
     // The pass before the handoff clears a stale foreign record the same way.
-    match after_the_reset(loader) {
+    match after_the_reset(loader).or(Some(loader)) {
         Some(after) if after.contains(FOREIGN_DONE) => Ok(()),
         _ => Err(Unfit::NoForeignDone),
     }