Repository navigation
toyos-sha2-hw: SHA-256 on x86-64's SHA extensions where CPUID offers them, for every SHA-256 the tree takes, the loader's included - #833
Conversation
…them, for every SHA-256 the tree takes The owner's ruling after #812 ("Yes, in a separate crate"): the instruction path lives in a crate of its own, and toyos-sha2 stays scalar, pure and forbid(unsafe_code). - toyos-sha2 takes a compression function: `Sha256::with(Compress256)` hashes with a caller's block function and buffers and pads around it as before; `new()` keeps the scalar one. `K256` is public, so the instruction path reads the one table. - toyos-sha2-hw's `sha256()` and `sha256_digest()` ask CPUID (leaf 7 EBX bit 29, leaf 1 ECX bit 9) as each hash begins and hand toyos-sha2 the SHA-NI compression where both bits are set, its scalar one where not. The compression is one `asm!` block, AT&T syntax, SHA256RNDS2 / SHA256MSG1 / SHA256MSG2 in the arrangement of Linux's arch/x86/crypto/sha256_ni_asm.S. It saves and restores the eleven XMM registers it uses itself and declares none, because the loader's target (x86_64-unknown-uefi) is soft-float and has no XMM register class to clobber, and the UEFI calling convention keeps XMM6-XMM15 the firmware's. Every target runs that one block. - AArch64 stays scalar: its SHA2 instructions are announced only in ID_AA64ISAR0_EL1, which ToyOS's EL0 cannot read and its kernel does not report. - Every SHA-256 caller moves to toyos-sha2-hw: toyos-update's `sha256` (the loader's section and ROOT hashes, update's), its repository `Archive`, update's streamed ROOT, pkg, swap, the build and the harness. SHA-512 (the SSHSIG header) stays toyos-sha2's. - The SHAVS readers move to toyos-sha2/src/tests/shavs.rs, which toyos-sha2-hw's tests include by path, so both crates drive the same CAVP files the same way. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017cSFvbD35xJ2kGANVdm23C
|
Evidence for #833 at head UEFI harness:
|
|
T14 reading, run by the orchestrator. Both images are
|
|
Review of #833 at Net lines ( Judgements asked for
BLOCKERNone. NOTE
LAND AFTER NAMED CHANGES |
…lar reason is the missing metal Review round 1 of #833 (NOTE 1 and the AArch64 judgement). - issues/x86-64-hashing-runs-scalar-where-the-cpu-has-sha-instructions.md was a `kind: question` the owner has answered, and its slug and body ("runs scalar", "the loader is not among them") are refuted by this branch. What remains is a measurement, so it is renamed to issues/no-t14-reading-of-updates-root-hash-before-and-after-sha-ni.md, `kind: tooling`, `status: open`, with the T14 reading of `update`'s ROOT hash before and after as its exit. The loader's reading is a different target and does not meet it. No file cited the old slug. - toyos-sha2-hw's module doc said AArch64 is scalar because EL0 cannot read ID_AA64ISAR0_EL1. The loader, at EL1/EL2, could; the reason that holds is that ToyOS has no AArch64 metal to measure a gain on. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017cSFvbD35xJ2kGANVdm23C
|
Round 2 gates at build-x86.logbuild-aarch64.log |
|
NOTE 3, read by the orchestrator: CI run 38048034073 at The x86-64 |
…toyos-sha2-hw #826 moved update's stream_root onto the block service (a `&mut dyn Disk`, STREAM_BLOCKS bytes per read and write); this branch had routed its hasher through toyos-sha2-hw. The resolution keeps #826's function whole and takes the branch's `toyos_sha2_hw::sha256()` for its hasher, as every other SHA-256 caller on this branch does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017cSFvbD35xJ2kGANVdm23C
|
Merge of The conflict was in
Gates at
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017cSFvbD35xJ2kGANVdm23C
, #826, #835, #833, #831 and #822, into wt/toyos-netperf No hunk conflicted. userland/netstack/src/main.rs took both sides: main's removal of `mod device` and the branch's batched `node.receive` and its module-doc line. The TCP window-scaling and loss-probe commits main carries were already in the branch from #820, so their files merged to main's text plus the branch's own delta. Both lockfiles are main's and pass `cargo metadata --locked`. The branch's new issue still cites `VirtioNet::poll_rx` and `toyos_i219::RX_BUDGET` as they stand on main. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017cSFvbD35xJ2kGANVdm23C
#846, #843, #849, #850, #840, #839, #837) into the batch: the icons' and wallpaper's digests hash with toyos-sha2-hw, and the loader's wall clock reads through its own UEFI bindings The batch's merge of main at f72d53d moved #813's wallpaper and #814's icon digest tests onto `toyos_sha2`. #833, already on main, had replaced the root package's `toyos-sha2` dependency with `toyos-sha2-hw`, so CI's merge of the two compiled no `toyos-build` lib test and both the build system's tests and clippy went red with E0432. Both tests now hash through `toyos_sha2_hw`, as every other SHA-256 the build takes does. #842's `bootloader/src/wallclock.rs` was written on the `uefi` crate, which #815 removes; it now calls `efi`'s `RuntimeServices::get_time`, whose `Time` fields are plain and whose error is the `Status` itself. `start_kernel` takes main's `wall_clock` and the batch's `SystemTable`; the batch's `armed_at` goes, as main replaced it with `wallclock::now`. Cargo.lock takes main's `ntapi`; `nonempty` goes with `gix`, which the batch removed and which was its only user (`cargo metadata --offline`). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017cSFvbD35xJ2kGANVdm23C
The owner's ruling after #812, "Yes, in a separate crate": SHA-256 runs on x86-64's SHA extensions where the CPU has them, in a new crate
toyos-sha2-hw, andtoyos-sha2stays scalar, pure andforbid(unsafe_code). Head873d96529, which mergesmainat60dc7fcec(#826) into round 1's1bec21be5, round 2's5878befa6and the merge ofmainata09dce906(ff80f9ba5).The ruling covers the loader. The owner's question came from the loader's measured ROOT-hash regression after #812, so "Yes, in a separate crate" is a yes for the loader too; the orchestrator so ruled on review round 1's NOTE 2. The loader is where most of the measured win is (Speed, below).
What changed, per decision
toyos-sha2takes a compression function.Sha256::with(Compress256)hashes with a caller's block function, and buffers and pads around it exactly as before.new()keeps the scalar compression.K256is public, so the instruction path reads the one table.toyos-sha2-hwchooses by CPUID as each hash begins.sha256()andsha256_digest()check leaf 7 EBX bit 29 (SHA) and leaf 1 ECX bit 9 (SSSE3, forPSHUFBandPALIGNR).toyos-sha2the SHA-NI compression; where either is clear, its scalar one.Somethatcompress()returns after that check: nothing can call it on a CPU without the instructions.asm!block in AT&T syntax:SHA256RNDS2,SHA256MSG1andSHA256MSG2, arranged as in Linux'sarch/x86/crypto/sha256_ni_asm.S. The hash value stays in XMM1/XMM2 across a whole run of blocks.x86_64-unknown-uefi, is soft-float (-mmx,-sse,+soft-float), so it has no XMM register class to clobber and no intrinsics.toyos_update::sha256.ID_AA64ISAR0_EL1, which the loader at EL1/EL2 could read. Only ToyOS's EL0 cannot. The crate's shape would take an AArch64 arm unchanged.toyos-sha2-hw.toyos-update'ssha256: the loader's kernel, cmdline, ROOT and signed-digest hashes, andupdate's section hashes.Archive.update's streamed ROOT, in A block connector opens only the partitions its badge grants, and a write its grant does not make is refused ReadOnly; update and the slot table reach diskserver's disk through sessions #826's block-servicestream_root: the merge at873d96529keeps that function whole and changes only its hasher, totoyos_sha2_hw::sha256()(note).pkgandswap.release,sourcegate,sysrootandimage.https_fetchhash.toyos-sha2's.toyos-sha2/src/tests.rstotoyos-sha2/src/tests/shavs.rs.toyos-sha2-hw's tests include that file by#[path], so both crates drive the same CAVP files the same way and the reader is not duplicated.NOTICE'stests/cavp/paragraph names both crates' tests.issues/the-build-runs-host-tools-outside-rust-and-qemu.md'sshasumrow namestoyos-sha2-hw, whichsrc/release.rsnow hashes with.issues/x86-64-hashing-runs-scalar-where-the-cpu-has-sha-instructions.mdwas akind: questionthe owner has answered, and this branch refutes its slug and body. It is renamed toissues/no-t14-reading-of-updates-root-hash-before-and-after-sha-ni.md,kind: tooling,status: open. Its exit is the T14 reading ofupdate's ROOT hash with the scalar compression and withtoyos-sha2-hw's.updatereports no hash time, and no metal row runs it. The loader's reading is a different target and does not meet that exit. No file cited the old slug (git grepat the head: no hit).Checks of high-risk code (the signed-image trust boundary)
toyos-sha2-hw's own tests (src/tests.rs,cfg(target_arch = "x86_64")):toyos-sha2's scalar compression over every length 0..=4096 bytes of random bytes, whole and in three seeded random splits each.x86_64::compress().expect(..). A host without the instructions reds by name ("this host's CPU has no SHA extensions") rather than passing on the scalar path.arch -x86_64 sysctl machdep.cpu.featureshas no SHA), and here--ci hostreadstoyos_sha2_hw:running 0 tests. The x86-64 CIhostjob runs them; it is skipped while the PR is a draft. Its log, with 3 passed, and the KVMguest / suiteat this head are the orchestrator's to read before landing.ciguest job's CPU on 2026-10-10, 23 jobs in the poolhostruns in: AMD EPYC 7763, 9V74 and 9V45, and Intel Xeon Platinum 8573C and 8370C. All have SHA-NI. A SHA-less SKU, or an Intel client CPU from Skylake to Comet Lake, redshostby name: a failure of the host, loud, not a scalar green over an untested path.x86_64-unknown-uefiapplication (source in the PR comment) builttoyos-sha2-hwwith[profile.toyos]'s settings and booted under QEMU 11.1.1 TCG with-cpu maxand edk2. CPUID leaf 7 EBX there was0x219c43a9, SHA set. It ran:toyos-sha2scalar and RustCryptosha20.10.9 (force-soft).SHANI-TEST ALL PASS, script EXIT=0..efidisassembles to 32sha256rnds2, 12sha256msg1and 12sha256msg2(64 rounds; schedule words 16–63).sha2's code, and QEMU's TCG implementation of the instructions from the SDM.git apply --checkthengit apply, built (EXIT=0), booted (EXIT=1), and restored, with the tree clean after each. The patches, the script and the logs are in the PR comment.palignr $4→$8(the W(t-7) tap):ShortMsg Len = 0red.SHA256MSG1dropped (< 52→< 48):ShortMsg Len = 0red.ShortMsg Len = 0red.d5bc4df9d(git diff d5bc4df9d be305c87eis 0 bytes) staged<scratch>/metal-base; the branch was then reset to1bec21be5. The asm's byte sequence0f38cbd1 660f6dc0 0f38cbcaoccurs 8 times in the branch'stestcasesimage and 0 times in the base's.qemu64, which has no SHA. The suite below is that arm, green.Speed
testcasesimage, with each boot's supervisor line naming its commit:d5bc4df9d(be305c87e):Slot A: ROOT hashed in 3,262,200,513 ticks. The loader took 9365 ms and the ROOT read 6392 ms.1bec21be5: 555,939,562 ticks. The loader took 8231 ms and the ROOT read 6390 ms.boot:testcasesovertestcasesandtestcases-watchdogat1bec21be5: EXIT=0,249 passed, 0 failed, 2 boot(s).main, and873d96529mergesmainagain. None of them changes the compression or the loader's hash, so the reading was not repeated atff80f9ba5or873d96529.testcasesimages weref3aaffb5268712b1b2108ad43e64efa35e31f5354d496ffdfa4e8c478853cc7c(branch) and5935489468e2656937c3f658ca2633bb6a39d7f616c5295097ae1daeb65b3f97(base). Staging exited EXIT=2 (staged) for each.[profile.toyos]'s settings took the following, best of 21 runs' thread CPU time, in four runs at load averages 33–46:main'stoyos-sha2: 98.31, 98.66, 98.53 and 97.88 ms.Gates
At
873d96529, the merge of #826. The logs are whole across PR comments, with home paths replaced by<checkouts>/and~/:cargo run -- --ci host: EXIT=0, "Host: 77 step(s), all green". It also reads "the licences of what ships: 6 exception(s) stand, and nothing else is refused" and "clippy, warnings denied: clean".toyos_sha2_hwreadsrunning 0 testson this aarch64 host. Log parts 1, 2, 3, 4, 5, 6, 7, 8.cargo run -- --build-only: EXIT=0.cargo run -- --build-only --arch aarch64: EXIT=0. Log.cargo test --test toyos-build: EXIT=0, "54 passed, 54 total" over 58 guests. That includes A block connector opens only the partitions its badge grants, and a write its grant does not make is refused ReadOnly; update and the slot table reach diskserver's disk through sessions #826'supdate_writes_the_idle_slot_through_the_block_service, which runs the mergedstream_rooton the scalar arm, andhttps_fetch.uptimeload averages were 34.57 / 44.72 / 43.71 before and 35.28 / 41.35 / 42.49 after. Log.At
1bec21be5, round 1:cargo test -p toyos-sha2 -p toyos-sha2-hw -p toyos-update -p toyos-swap: EXIT=0.toyos-sha2: 5 passed.toyos-sha2-hw: 0 tests on this aarch64 host (they are x86-64's).toyos-swap: 5 passed.toyos-update: 42 passed, plusrepo_oracleandsshsig_oracle.cargo clippy -p toyos-sha2-hw -p toyos-sha2 --all-targets --target x86_64-unknown-linux-gnu: EXIT=0, no warnings. This is the x86-64 arm and its tests, which--ci hostlints only on an x86-64 host. The bootloader shape lints the arm onx86_64-unknown-uefiin every run.cargo test --test toyos-build: EXIT=0, "50 passed, 50 total" over 53 guests,https_fetchamong them.uptimeload averages were 70.32 / 68.32 / 55.52 before and 55.86 / 61.40 / 55.26 after. Every x86-64 boot verifies its image through the loader's hash.Not run by this branch: CI's x86-64
hostjob and the KVMguest / suite, which the orchestrator reads.No new guest test. The hash is a pure function: the host tests above cover it, and so does the loader-target harness, which needs a CPU with SHA. Every boot in the suite already runs the selector.
Net lines over
main(git diff --shortstat origin/main...873d96529): 28 files, +486/−218.toyos-sha2-hwis 193 lines (lib.rs33,x86_64.rs148, manifest 12), andtoyos-sha2/src/lib.rsis +24/−9.toyos-sha2-hw/src/tests.rs+62.shavs.rs+145 is moved out oftoyos-sha2/src/tests.rs(−141).Unsure
-cpu hostpasses SHA through, and in the CIhosttests.CpuExceptionHandlerLibdoes withFXSAVE. That is read, not measured on the T14's firmware. A corrupted register would give a digest that does not match, and the slot would be refused loudly; it would not be accepted.🤖 Generated with Claude Code
https://claude.ai/code/session_017cSFvbD35xJ2kGANVdm23C