Delete the test tiers: a run is the whole suite or its filter

[Japabu]

Delete the Fast / Nightly / Weekly / Local test tiers. A run now takes the whole suite, or whatever its filter names; each nightly guest shard runs everything, on one daily schedule.

What went

• src/tiers.rs: Tier, Reach, Schedule, for_filter and their tests.
• The Tier column on every MACHINE_TESTS and SCREEN_TESTS row in tests/toyos.rs. Also gone:
  • SHARED_TIER
  • the assert that a boot group shares one tier
  • the reach loop in check_shard_partition
  • the held-back / local-tier NOT run lines
  • Tally::holding_back and its relegated field, with the not run without --nightly blocks and the held back for suffix on the result line
• --nightly / --weekly in src/testargs.rs, and their two refusals (--nightly beside --weekly, either one beside --metal). Their tests are deleted too, and the flags are dropped from the fixture argv lists. A --nightly typed now is refused as an unknown flag.
• src/ci.rs: guest_reach, reach_of_event, reach_of_schedule, NIGHTLY_CRON / WEEKLY_CRON, and their three tests, including the one that parsed nightly.yml's crons. guest <i>/<n> now runs --shard i/n --jobs 1.
• .github/workflows/nightly.yml: the two crons become one, 0 3 * * *.
• tests/checks.rs nightly_tier_is_announced.
• Comments and docs that described tiers were deleted, not reworded:
  • the tier-justifying row comments
  • the faults.rs "Nightly." paragraph
  • build.rs's paragraph on what a CI shard charges
  • nightly.yml's header clause on when it runs
  • the tier pointers in CLAUDE.md (briefed) and tests/CLAUDE.md
• issues/kernel/toyos-runs-on-arm64.md stage 8 no longer names a tier or an aarch64 CI lane. Its exit is one manual whole-suite run on the M4 host, after the track's stages.

What replaced Local

Each SCREEN_TESTS row names the qemu::Profile it boots, where the Tier column was. The screen task hands that profile to run_screen_test, and every body boots it rather than a literal of its own. virt_job and virt_selftest take it as a parameter, and virt_job's cached test binary is virt_copyout(arch): one cache per architecture behind an exhaustive match, so a boot cannot be served the other architecture's binary. MACHINE_TESTS rows carry only (name, Sched).

Selection is select(shared, filter, sharded) in tests/toyos.rs:

• A run takes every test that kept(filter, name) passes, meaning the filter matches and no redlist row disables it.
• On a shard, a screen row is dropped when profile.arch() is not ci::GUEST_ARCH.
• GUEST_ARCH is the constant the guest job's instrument step reads too.

The architecture is stated once, by Profile::arch(), and read off the profile the body actually boots. A row cannot say aarch64 while its body boots x86. The one body that boots a second machine is screen_gop_firmware_mode: its row names Gop, and it also boots Metal by literal. A different-arch literal there would fail that boot on a shard. It could not be skipped.

Every shard prints one line naming the screen rows it drops for their architecture (arch_drop_line: the rows select(.., false) takes and select(.., true) does not), whether or not anything is left to run. An empty selection has the one message it always had, "No enabled test matches filter".

What stayed

The refusal of a name registered twice stays as registered(), which now answers a Result so the refusal is testable. --list prints bare names: 517, 12 of them virt_.

What the prompts and issues say now

A search of every CLAUDE.md, .claude/agents/*.md, README.md and issues/ for tier, --nightly, --weekly, Fast/Nightly/Weekly/Local, src/tiers.rs, in_tier and Tier:: found no prompt, CLAUDE.md or README.md that names a test tier; the tier left in those is model tiers and the compiler's target tiers. In all, 29 issue files and one prompt line changed.

• .claude/agents/orchestrator.md, "Runs": one line, "A run is the whole suite, or a positional filter that reaches any enabled test." The orchestrator is the only role that runs guest tests, and "enabled" is there because a src/redlist.rs row still disables a test in every run.
• Ten issue files, earlier in the branch, lost the clauses that named a tier, src/tiers.rs or FAST_CEILING_MS; one of them is the stage 8 bullet above.
• A tier stated as a test's own property, the word deleted: issues/boot-media/a-disk-whose-port-went-away-panics-the-boot-at-roots-hold.md, issues/boot-media/screen-diag-boot-leaves-no-i8042-line-on-the-panel.md and issues/filesystem/home-budget-refusal-retried-is-red-on-every-nightly.md ("(nightly tier)"); issues/kernel/a-log-rings-owner-is-named-only-when-logd-reads-its-registration.md ("(fast tier)"); issues/filesystem/blockd-survives-its-death-reds-on-a-replacement-that-reorders-acknowledged-writes.md ("The nightly-tier test" is "The test"); issues/hardware/tearing-is-what-gop-cannot-give-back.md (", Nightly" in a registration citation).
• "in any tier" and "in every tier" deleted from a coverage claim: issues/kernel/a-claims-own-refusals-are-read-by-nothing.md, issues/kernel/nothing-asserts-that-a-claim-answers-no-configuration-write.md, issues/kernel/nothing-reaches-the-msi-arm-of-a-claimed-function.md.
• An exit condition that asked for a tier or typed --nightly:
  • "or a fast-tier test" deleted: issues/build/which-pass-drain-irqs-is-entered-from-is-gated-only-by-a-nightly-guest.md and issues/build/a-settled-thread-join-taking-the-table-lock-again-is-gated-by-nothing.md, which also says "the guest fpu_isolation" where it said "the nightly guest".
  • A whole-suite run named for what it is: issues/build/lan-talk-host-closes-judges-a-fixed-ten-second-window.md ("a loaded suite"), issues/build/parallel-tests-red-under-other-suites.md ("a loaded full suite"), issues/build/qemu-drops-console-output-the-harness-is-slow-to-read.md ("a full suite"), issues/build/the-console-loses-typed-keystrokes-under-host-load.md ("runs" for "runs of the Fast tier").
  • "either re-tiered or fixed at the cause" is "fixed at the cause": issues/build/log-reserve-window-negative-times-out-beside-other-guests.md and issues/build/process-stats-exits-101-beside-other-guests.md.
  • --nightly removed from the command: issues/build/usb-transport-break-flushedstick-can-break-after-the-reboot.md.
• A mechanism that is gone: issues/build/the-shard-split-prices-a-boot-and-not-the-image-behind-it.md (the clause tying the profile to the ten-second Fast ceiling, and the sentence arguing a second profile from that ceiling and the tier gate) and issues/build/idle-stack-guard-price-nearly-doubled-since-its-return.md (the closing sentence on relegation and the per-PR gate).
• issues/build/a-metal-only-row-cannot-be-disabled.md: schedule is registered in its three citations, the function this branch replaced.
• Left as they are: sightings of runs that happened, which stay true of the run (17 lines in 15 issue files record a --nightly command that ran and its outcome, and others call the run a "fast tier"); the audio gate's own fast and thorough modes (issues/audio/gate-a-suspend-structure-verdict-unread.md, issues/audio/hda-has-no-jack-detection-volume-or-keys.md, issues/audio/desktop-session-put-26ms-of-silence.md); and issues/build/there-is-no-network-gate.md's plan for a Fast and a Thorough split, which is the owner's to decide.

Size

Net −348: 41 files changed, +678/−1026 (git diff --shortstat origin/main...HEAD at head 68340f068, origin/main 3af470121). Production (src/, .github/, CLAUDE.md): 7 files, +17/−343. Tests (tests/): 4 files, +611/−609. Prompts and issues (.claude/, issues/): 30 files, +50/−74.

Gates (head 68340f0, origin/main 3af4701 merged)

• cargo run -- --ci host exits 0: [ci] Host: 54 step(s), all green.
• cargo test --test toyos-build -- --list exits 0: 517 names, 12 of them virt_, the same 517 names and the same redlist rows as at 11e543ae1.

The merge of origin/main

origin/main at 3af470121 (#616, the metal judges) is merged. The one conflict was issues/build/a-metal-only-row-cannot-be-disabled.md: main cut the line citing an issue #616 deleted, and this branch renamed schedule to registered on the line after it. Both edits stand, and no file cites any of the eight documents #616 deleted, by path or by bare name. The merge changes none of the branch's hunks and drops none of main's: over git diff -U0, merge minus main and the branch's own patch (11e543ae1 against the ace064f9d it had last merged) have one git patch-id --stable, 22f8051e, and merge minus branch and main's own patch have one, 7ebcbfca. No file the branch touched is a file main deleted, so no file is excluded. Altering one line of the conflicted file's hunk in either diff text moves its id (c38d66df, 90da2bbb), so the equality is not vacuous. tests/toyos.rs and tests/checks.rs are the two files both sides edited and merged without a conflict; their hunks do not overlap, and cargo test --test toyos-checks runs both sides' new cases in one binary, 19 passed. tests/common/faults.rs, src/testargs.rs and src/build.rs are byte-identical to 7d98ba5b. #616 adds no tier, reach-flag or schedule wording: a sweep of every CLAUDE.md, .claude/agents/*.md, README.md and issues/ for tier, --nightly, --weekly, Fast, Nightly, Weekly, Local and the removed names finds the same lines before and after the merge, so this round edits nothing else.

Tests and negative controls

checks::a_run_selects_by_filter_and_shard runs against the real tables plus one fixture shared test. It checks:

• with no filter, an unsharded run takes every registered, non-redlisted name
• with no filter, a sharded run takes the same minus the rows whose profile is not of GUEST_ARCH
• a filter takes only the matching rows, on or off a shard, including a non-prefix filter (el2_drop)
• a sharded virt_el2 filter takes nothing
• a shard's drop line names exactly the foreign rows and their count with no filter, only virt_el2_drop under el2_drop, and nothing under sshd_

checks::a_name_registered_twice_is_refused checks three things:

• two distinct shared rows plus the declared tables register
• two shared rows under one name are refused
• a shared row named like a screen row (virt_el2_drop) is refused

Each mutation below was applied as a checked patch (git apply --check), built (build exit 0 each time), run, and reverted, leaving the tree clean.

At 7d98ba5b:

Mutation	Test exit	Failing case
arch_drop_line: select(shared, filter, false) → select(shared, filter, true)	101	a shard that drops the rows of another architecture said nothing

At 60e667d2 (kept, registered and select are unchanged since):

Mutation	Test exit	Failing case
kept: name.contains(f) → name.starts_with(f)	101	el2_drop, unsharded
registered: the refusal dropped (names.insert(name);)	101	shared_one twice answered Ok(305)
select: the shard's arch filter dropped	101	sharded, no filter: the 12 virt_ rows taken
virt_el2_drop's row names Profile::Gop	101	the premise: virt_el2_drop no longer foreign

From the first round, all red (exit 101):

• a shard keeps the foreign arch
• the foreign arch is dropped even unsharded
• machine rows ignore the filter
• shared rows ignore the filter
• screen rows ignore the filter
• kept ignores the redlist

A whole-change revert is not a usable negative control here: the test and select do not exist on the base.

Measured cost of running everything

• The orchestrator's local run of the whole suite, unfiltered and unsharded, at 7d98ba5b: 495 passed of 495.
• The nightly dispatch on this branch, run 36600425263 at 2d45623a: host, build, tcg, both portability jobs and 11 of the 12 guest jobs passed. guest (4) failed on screen_loader_lines alone, 21 of its 22 tests passing: the panel carried 26 rows at the GOP query and 41 at the loader's last line, a growth of 15, where the loader printed 14 lines between them, 14 rows at 240 columns. The same message reds main's nightly (run 36550208853 at 7e151819, guest (11)), and screen_loader_lines is one of the names in issues/build/parallel-tests-red-under-other-suites.md.
• In that run the twelve shards' suite totals are 227, 22, 21, 22, 21, 22, 24, 24, 23, 24, 29 and 24. They sum to 483, the 495 less the 12 virt_ rows, and no shard log names a virt_ row.
• The longest guest job, guest (4), took 27 m 53 s, against the 60-minute job timeout.

Unsure / not measured

• A MACHINE_TESTS row carries no profile and is always kept on a shard. No machine test boots virt today. One that did would fail on a shard, loudly.
• "Tier" wording that means metal against QEMU (testargs.rs a_readback_directory_alone_selects_no_tier, toyos.rs "no other tier" in the metal registration doc) is not the tier system and stays.
• No nightly has run at or after 7d98ba5b, so no CI shard has printed the drop line; only the host test checks it.
• Neither guest run above ran the hunks of main's that this branch has merged (Metal judges read the channel their line crosses on #616, Delete forkcheck; the fork rules live in the agent prompts #620, sourcegate's prose ledgers become review rules; BANS become clippy disallowed-methods #624, Delete the meta tracks; fold the shared-cache track into the toolchain track #626, The orchestrator's run procedure is written down #627, The actuator-state gate goes; its rule is the reviewer's #628); on the merged head only the gates above have run. Metal judges read the channel their line crosses on #616 includes check_c_result's expectation reader, which an ordinary guest run exercises.

🤖 Generated with Claude Code

https://claude.ai/code/session_016t9wjdQkB8SH7bmfUoiy6L

[Japabu]

Review of #625 at c9ef173b

Gate: CI host passed at c9ef173b: run 36592180458 concluded success, [ci] Host: 54 step(s), all green, and checks::a_run_selects_every_test_its_host_boots ... ok is in its log. This change touches no hardware. Net size is −414: production (src/, workflow) is +19/−339, and the harness (tests/) is +459/−553.

BLOCKER

• tests/toyos.rs:492-537 + tests/common/qemu.rs:1389-1391: the new Arch column is a second declaration of something the tree already declares once. Profile::arch() already says which architecture each boot is. Nothing checks that a row's Arch matches the profile its body boots. The PR body concedes this: "an x86 row marked Aarch64 would silently skip CI". It is a known compromise and the branch neither removes it nor records it. Fix it in one of two ways:
  • Refuse the disagreement where the screen task boots: the task knows its row and the profile knows its arch, so a mismatch reds on any unsharded run.
  • Or file it in issues/ with an owner, the evidence and an exit condition.

NOTE

• Selection is exactly what the brief asks. An unsharded run takes every registered, non-redlisted name (kept), including the 12 virt_ rows. A shard takes the same set minus the rows whose Arch is not GUEST_ARCH.
• Which hosts boot which rows is unchanged from Local:
  • On this aarch64 Mac, Profile::Virt gets HVF (Arch::accel, macOS and same arch), VirtEl2 is always TCG (virt_el2_drop and the virt_job rows), and x86 rows get TCG.
  • On an x86-64 Linux host, x86 rows get KVM and every virt_ row gets TCG. That needs qemu-system-aarch64 and AAVMF, exactly as the old Local tier did.
• Arch is not a tier under another name. It is a fact with one consumer, and the rule lives in one line tied to the constant the instrument step also reads. Its only defect is the duplication in the BLOCKER.
• The deletion that would remove the column altogether is installing qemu-system-aarch64 and AAVMF in the guest lane and dropping the shard filter. That is unmeasured on the runner and belongs to issues/kernel/toyos-runs-on-arm64.md stage 8, not this branch.
• tests/checks.rs:570: the test name …_its_host_boots overclaims. Selection never asks what the host can boot; it keys on --shard. A local --shard 1/2 on the Mac drops virt_ rows the Mac can boot.
• tests/checks.rs:600-606: every filter case is a prefix, so this mutation stays green: in kept, name.contains(f) → name.starts_with(f). The case that turns it red is (Some("el2_drop"), false, names(&["virt_el2_drop"])).
• tests/toyos.rs:17238-17246: registered()'s duplicate refusal lost its only test, which was tiers.rs every_registered_test_has_exactly_one_tier. The mutation names.insert(name); (the assert dropped) stays green in every host suite. Either pull the check into a Result the checks can call on a two-row fixture, or accept that it is untested.
• tests/toyos.rs:17424: a sharded run whose filter matches only virt_ rows now says "No enabled test matches filter". The rows do match; the shard dropped them for their arch, and the old local-tier line said so.
• The nightly guest timeout (60 min) is already measured, not "unmeasured". Nightly run 36306830048 at 16d2e645 predates 4505f872d (the Weekly tier), when --nightly selected everything but Local, so it is this branch's shard selection:
  • guest (6) ran 08:40:56→09:10:16, 29 m 20 s, suite line (1173.6s), the longest of the 12.
  • Headroom is about 2×, less whatever landed since.
• The owed guest measurements are unchanged: the unfiltered local wall time (the orchestrator's run), and one nightly dispatch on the branch to confirm the shard drops the virt_ rows.
• Past-run evidence that ran --nightly X stays: those lines record a command that ran and exited, not an instruction. They are:
  • a-syscall-panic-reset…:11
  • log-flush-retry-deadman-arm.md:9
  • screen-diag-boot…:18
  • a-disk-whose-port…:22
  • usb-transport-break-flushedstick…:22,30
  • home-budget…:22
  • blockd-survives…:19
  • logd-flushes…:21
  • kernel-log-file…:10
  • a-log-rings-owner…:24
  • a-holders-queued-line…:42
  • a-held-disk…:27
  • a-shutdown-on-a-held-usb-disk…:9,37
  • a-loaded-suite…:114,156
  • the-shard-split…:51
• The same goes for the "fast tier at " sighting records.
• issues/kernel/toyos-runs-on-arm64.md:317-324: stage 8's heading and exit still say "aarch64 tier", "the fast tier runs on aarch64 locally" and "the nightly runs the aarch64 tier". The owner should restate that stage; this branch should not rewrite it.
• No metal code read tiers; the metal path has no Tier reference left. tests/test-durations is keyed name ms, with no tier column.

REMOVE

• src/ci.rs:44: "A shard, which only a guest lane runs, selects no other." False: --shard 2/4 is accepted locally (testargs.rs fixtures).
• tests/checks.rs:566-568: ", which only a CI guest lane runs," is the same false claim.
• src/ci.rs:795-796: "Every red is adjudicated into a fix or a src/redlist.rs row" is prose rewritten instead of deleted, and nothing here enforces it.
• src/build.rs:33-35: "Without this distinction, each CI shard charges … tens of seconds; the next run then charges …" is rewritten, and it is a story with an unsourced number.
• .github/workflows/nightly.yml:3-5: ": on main every night, and on any branch by gh workflow run …" is rewritten, and it restates the on: block.
• PR body: "The nightly guest shards now also carry what used to be Weekly, inside a 60-minute job timeout, and that is unmeasured too." Run 36306830048 measured it.
• issues/kernel/desktop-window-child-freeze.md:137-139: "The test is Tier::Nightly (src/tiers.rs) … -- --nightly desktop_window_child does." It is now false, and that command is refused.
• issues/kernel/toyos-runs-on-arm64.md:319: "src/tiers.rs gains the arch axis."
• issues/isolation/the-supervisor-is-host-tested-and-owns-the-stop.md:13-15: "registered at Tier::Fast or Tier::Nightly …"
• issues/kernel/no-kernel-address-is-drawn-per-boot.md:17: the Exit cites "a Tier::Weekly test".
• issues/build/parallel-tests-red-under-other-suites.md:68-69: "is Tier::Nightly and so never in a pull request's parallel phase".
• issues/boot-media/kernel-log-file-reds-beside-other-guests-and-is-green-alone.md:43-45: "kernel_log_file is Nightly for Why::Cost … (src/tiers.rs)."
• issues/build/the-tooling-is-a-review-prompt-and-three-workflows.md:13: src/tiers.rs is in the list of files that go, and it is gone.
• issues/build/the-shard-split-prices-a-boot-and-not-the-image-behind-it.md:97: "FAST_CEILING_MS (src/tiers.rs:84, 10,000 ms)".
• issues/build/a-shards-boot-width-does-not-price-its-tests.md:14,63: "src/tiers.rs's ceiling" and "by src/tiers.rs relegation class".
• issues/build/smp-ap-hole-and-log-reserve-window-red-under-a-loaded-host.md:13: "src/tiers.rs's RELEGATED".
• issues/build/defect-events.md:156-157: "src/tiers.rs reded a Tier::Fast name measured over FAST_CEILING_MS …"

SEND BACK

🤖 Generated with Claude Code

[Japabu]

No CI host exists at 60e667d, and none can run while the PR stays DIRTY. origin/main moved to b846743 (#623, merged 16:22:44Z), and git merge-tree --write-tree HEAD origin/main reports CONFLICT (content): Merge conflict in tests/toyos.rs. The ci workflow ran nothing at this head between 16:25 and 16:34Z. Merge origin/main, resolve tests/toyos.rs, and push. The next round needs host green at that head, plus the orchestrator's whole-suite run and the nightly dispatch green at that same head, not at 60e667d.

NOT READY FOR REVIEW

🤖 Generated with Claude Code

[Japabu]

Review of #625 at 2d45623a, round 3.

Gate: CI host at 2d45623a (run 36600384661) concluded success. Its log has [ci] Host: 54 step(s), all green, and checks::a_run_selects_by_filter_and_shard ... ok and checks::a_name_registered_twice_is_refused ... ok (test result: ok. 13 passed). This change touches no hardware.

Net size is −383 (+603/−986):

• production (src/, nightly.yml, CLAUDE.md): +19/−343
• the harness (tests/): +568/−603
• issues: +16/−40

Two runs must be green at the landing head, both the orchestrator's:

• The whole suite, unsharded and unfiltered: exit 0 and test result: ok. 495 passed, 495 total. That is 517 registered less 22 redlisted, and the 12 virt_ rows must be among the passes.
• The nightly dispatch: host, build, tcg and 12/12 guest jobs success, each inside its 60-minute timeout. No shard log may name a virt_ row, and the shards' totals must sum to 483.

Earlier BLOCKER

• CLOSED — the duplicate Arch column.
  • The column is gone from SCREEN_TESTS.
  • Each row's qemu::Profile is the literal its body booted on origin/main. All 30 rows were checked arm by arm.
  • Every body boots profile. The one exception is the comparison arm of screen_gop_firmware_mode (see the first NOTE).
  • select reads profile.arch().
  • The implementer's mutation (virt_el2_drop's row changed to Profile::Gop) exits 101, and the test is green in CI at this head.

The earlier NOTEs are closed as well: the el2_drop case, registered() answering a Result with its own test, the message naming dropped rows, the test's rename, and the measured timeout.

BLOCKER

None.

NOTE

• tests/toyos.rs:3822,3834 — screen_gop_firmware_mode's Profile::Metal literal is accepted, and needs no change:
  • It is the second machine of a two-machine comparison, and a row can name only one.
  • Metal and Gop are both x86_64. A literal of another arch would fail its boot on a shard; it could not be skipped.
  • if gop == metal (:3869) reds if the row itself names Metal.
• tests/toyos.rs:17433-17451 — an unfiltered shard drops the 12 virt_ rows without a line. main printed local tier: 12 test(s) NOT run … on every shard; this branch names the dropped rows only when the selection ends empty. Fix it with one path instead of two:
  • on any shard, print the names that select(&all_tests, filter, false) takes and the sharded selection does not;
  • keep the plain "No enabled test matches filter" as the only empty-selection exit.
• tests/toyos.rs:17439 — no test covers the dropped-rows message. The mutation select(&all_tests, filter, false) → select(&all_tests, filter, true) stays green in every host suite, because both arms exit 1.
• tests/toyos.rs:3570-3573 — the COPYOUT OnceLock in virt_job ignores the profile the function now takes. The first caller's profile.arch() builds the binary that every later caller boots. All six callers pass VirtEl2 today. Key the cache by arch, or assert the arch matches the cached build's.
• tests/toyos.rs:5331-5333 — the reason for Profile::Metal stays in the body: keys reach the i8042 only without a usb-kbd. The choice itself now sits on the row at :524. Move the comment to that row.
• issues/kernel/toyos-runs-on-arm64.md:314-318 — stage 8 is consistent with the owner's ruling: run by hand, once, on the M4, with no CI lane.
  • Its exit ("red on only one arch") needs a per-test arch axis, which one-profile rows cannot express.
  • Its body ("tests/common/qemu.rs takes an Arch") is the only statement of that work.
  • That is adequate for a parked track.
• issues/build/there-is-no-network-gate.md:18,39,50 — this track plans a Fast/Thorough two-tier split, against the ruling this branch lands. That is the owner's call and outside this branch's fence.

REMOVE

• src/ci.rs:34-35 — GUEST_ARCH's doc. The first clause restates the constant. The second ("no hosted runner has been measured for an aarch64 one") implies a runner measurement that stage 8 no longer plans.
• tests/checks.rs:577 — "A shared-boot row under name, for the checks below." restates the function's name.
• PR body, the second "Unsure / not measured" bullet: it repeats the last three sentences of "What replaced Local".
• PR body, the first "Unsure / not measured" bullet: it is false once the dispatch reports. That run's record replaces it.

LAND AFTER NAMED CHANGES

🤖 Generated with Claude Code