diff --git a/.changes/secret-bearing-hmac-hkdf-8528.md b/.changes/secret-bearing-hmac-hkdf-8528.md new file mode 100644 index 00000000..d0f66149 --- /dev/null +++ b/.changes/secret-bearing-hmac-hkdf-8528.md @@ -0,0 +1,17 @@ +--- +"rscrypto" = "minor" +--- + +Secret-bearing HMAC, HKDF, KMAC, and PBKDF2 states no longer implement +`Clone`. Keyed BLAKE2 parameter builders and the variable-output Blake2b state +also no longer do, because they can retain a MAC key. The `Mac` trait no longer +requires `Clone`; reuse or share one owner, or construct another keyed state +explicitly. + +Secret-bearing SHA/HMAC, Ed25519, Keccak, BLAKE2, and BLAKE3 temporaries now +clear at their actual finalization, reset, transfer, or drop boundary, including +keyed XOF and parallel heap scratch. The optimized zeroization gate covers +success, error, early-return, move, reuse, and drop shapes. Secret-key hex and +generic ECDSA random-source errors no longer echo input or payload bytes through +`Debug`, `Display`, or the standard error-source chain. Their public fields and +variants still allow explicit recovery when callers deliberately inspect them. diff --git a/THREAT_MODEL.md b/THREAT_MODEL.md index 377230cd..f899a98c 100644 --- a/THREAT_MODEL.md +++ b/THREAT_MODEL.md @@ -5,8 +5,11 @@ crate defends, what the caller owns, and where an audit should focus. Related documents: [`SECURITY.md`](SECURITY.md) for vulnerability reporting, [`docs/constant-time.md`](docs/constant-time.md) for the exact constant-time -claim model, [`docs/compliance.md`](docs/compliance.md) for regulatory -positioning, and [`ct.toml`](ct.toml) for the machine-readable CT claim set. +claim model, [`docs/secret-ownership.md`](docs/secret-ownership.md) for the +secret-bearing type and heap inventory, +[`docs/secret-lifecycle.md`](docs/secret-lifecycle.md) for cleanup and redaction +evidence, [`docs/compliance.md`](docs/compliance.md) for regulatory positioning, +and [`ct.toml`](ct.toml) for the machine-readable CT claim set. ## Audit Scope @@ -114,17 +117,17 @@ Ordered by exposure to untrusted input: | Wrong output from accelerated kernels | Portable path is the byte-for-byte authority | Portable-vs-accelerated differential tests and native CI | | Timing leakage | Constant-time coding rules on claimed paths | `ct.toml` evidence gate: timing tests, generated-code review, binary checks where supported | | Oracle behavior | Opaque errors, failed-open output wipe, single-bit failure shape | AEAD and verification tests, fuzz targets | -| Secret exposure at rest | Zeroize on drop, masked `Debug`, and sealed fixed-size comparison only on semantic secret owners | `src/secret.rs`, `ct.toml`, and per-owner tests | +| Secret exposure at rest | Zeroize at the last owned use and on drop, masked `Debug` and errors, and sealed fixed-size comparison only on semantic secret owners | [`docs/secret-ownership.md`](docs/secret-ownership.md), [`docs/secret-lifecycle.md`](docs/secret-lifecycle.md), `scripts/check/zeroize-evidence.sh`, and `tests/secret_redaction.rs` | | Supply chain | Minimal optional runtime dependencies, `cargo deny`, `cargo audit`, signed tags, Trusted Publishing, release attestations | `deny.toml`, `.github/workflows/release.yaml`, `docs/release.md` | ## Known Gaps - No third-party security audit yet. -- Secret containers and parsing temporaries use volatile source-level wipes, - and `scripts/check/zeroize-evidence.sh` checks representative optimized MIR, - LLVM IR, and assembly paths. This does not prove that every compiler-created - register copy or spill is erased on every target. The crate does not lock - pages, prevent swapping, or replace hardware-backed key storage. +- Named secret owners and explicit temporaries use volatile source-level wipes, + and `scripts/check/zeroize-evidence.sh` checks optimized lifecycle shapes in + MIR, LLVM IR, and host assembly. This does not prove that every + compiler-created register copy or spill is erased on every target. The crate + does not lock pages, prevent swapping, or replace hardware-backed key storage. - Miri covers portable paths only; sanitizer and interpreter coverage do not execute every native SIMD or assembly kernel. - Constant-time evidence is produced by CI and release workflows. Consumers diff --git a/docs/secret-lifecycle.md b/docs/secret-lifecycle.md new file mode 100644 index 00000000..4c5e9e5e --- /dev/null +++ b/docs/secret-lifecycle.md @@ -0,0 +1,106 @@ +# Secret Lifecycle And Diagnostic Redaction + +This document records where rscrypto-owned secret storage is cleared and how +that source-level policy is checked after release optimization. The type and +capability boundary is defined in +[`secret-ownership.md`](secret-ownership.md); this document follows those +owners through success, failure, early return, transfer, reuse, and drop. + +## Claim Boundary + +The lifecycle claim covers named owner storage and explicit source-level +temporaries: inline arrays and words, initialized `MaybeUninit` regions, +heap-backed vectors and boxes, reusable scratch, finalized hash snapshots, and +parser or generator staging buffers. Cleanup uses volatile writes plus a +compiler fence. + +The claim does not extend to caller-owned input, a secret after an explicit +export into ordinary bytes, compiler-created register or spill copies, swapped +pages, crash dumps, or hardware-backed storage. Protocol-visible tags, keyed +outputs, signatures, public keys, nonces, and ciphertexts are not confidential +owners merely because they are produced by secret-bearing operations. + +## Source Ownership And Cleanup + +| Owner or flow | Retained secret state | Cleanup boundary | +|---|---|---| +| `SecretBytes`, `SecretVec`, typed keys, private keys, and shared secrets | Fixed or variable-length key material | Concrete `Drop`; consuming export either clears the source allocation or explicitly transfers responsibility to the returned ordinary bytes | +| AEAD contexts and AES backend schedules | Expanded encryption keys and authentication subkeys | Context and nested schedule `Drop`; operation-local subkeys and authentication state are cleared after use; failed open and private-output paths clear rejected output | +| HMAC-SHA-2 | Live SHA state, keyed inner/outer prefixes, oversized-key digests, and inner-digest finalization snapshots | Secret-specific SHA finalization clears copied state and padding blocks; reset clears the replaced live state; `Drop` clears the live state and both saved prefixes | +| HKDF and PBKDF2 | PRK or password-derived HMAC prefix words and derivation scratch | Prefix-owner `Drop`; oversized-key/password digests and per-block working values are cleared on every return path | +| Ed25519 signing | Expanded scalar, nonce prefix, nonce hash state, digest, and scalar intermediates | Expanded-secret `Drop`; secret-specific SHA-512 digest/finalization clears hash state and padding snapshots; signing clears scalar and digest temporaries before return | +| HMAC-SHA-3 and KMAC | Keyed Keccak state, initial snapshots, and finalized inner state | Secret-mode Keccak owners clear on replacement and `Drop`; fixed-output finalized sponge copies and inner digests are cleared after absorption | +| Keyed BLAKE2 | Stored key, chaining state, block buffer, and finalization copies | Core and parameter-owner `Drop`; finalized chaining words and partial-block copies are cleared before return | +| Keyed and derive-key BLAKE3 | Key words, chunk/output/root state, CV stack, XOF root, and per-state, reduction, or thread-local parallel CV vectors | Conditional nested `Drop` follows the mode flags; emitted output blocks are cleared after copying; reusable and reduction vectors are cleared after their last keyed use, before logical `Vec::clear`, and on owner drop | +| ECDSA, X25519, ML-KEM, and RSA private operations | Secret scalars, decapsulation arithmetic, private components, blinding values, limbs, encoded-message buffers, and reusable private scratch | Typed and nested owner `Drop`; operation wrappers clear local arrays and initialized heap regions; rejected private outputs are cleared | +| Argon2 and scrypt | Password/pepper-derived matrix or ROMix working set | Owning matrix/state `Drop` clears every initialized block; allocation and parameter failures retain the same RAII boundary | + +Backend call records and fixed-size array arguments may be copied into registers +or ABI spill slots by the compiler. Their durable source owners and explicit +stack/heap temporaries are covered above; the compiler-created copies remain an +explicit limitation rather than an unprovable erasure claim. + +## Path Audit + +| Path | Source audit result | Optimized evidence | +|---|---|---| +| Success | Finalized HMAC/Keccak/BLAKE copies, oversized-key digests, emitted BLAKE3 blocks, parser staging, and private-operation scratch are cleared after the last read | Fixed stack, secret hex success, HMAC-SHA-2/SHA-3 finalization, and keyed BLAKE3 wrappers retain volatile zero stores | +| Error | `ZeroizingBytes` and RAII owners cover parser/generator failure; AEAD and RSA clear rejected plaintext/private output | The secret hex error wrapper reaches the same audited parser as success, and its release IR contains both cleanup paths | +| Early return | Scope-owned fixed and heap secrets retain `Drop` cleanup across `return` and `?`; explicit cleanup precedes returns from manual scratch paths | `diag_zeroize_early_return` retains zero stores in release MIR, LLVM IR, and assembly | +| Move or transfer | `SecretBytes::expose` clears its source before returning ordinary bytes; `SecretVec::into_unprotected_vec` transfers the allocation and responsibility; keyed XOF moves transfer one root owner whose destination clears on drop | Fixed-owner move and keyed BLAKE3 XOF move/consume wrappers retain source and destination cleanup | +| Reuse | HMAC clears replaced live SHA state; secret-mode Keccak assignment drops the replaced state; BLAKE3 replacement drops the old owner; BLAKE3 parallel vectors are wiped before reuse | The BLAKE3 reset wrapper contains separate production `Drop` calls for the replaced and final owners | +| Drop | Every confidential public owner in the ownership inventory reaches a concrete or nested cleanup implementation; heap owners traverse initialized storage before deallocation | The gate follows the keyed BLAKE3 wrapper into its production `drop_in_place` and requires retained owner and heap-scratch zero stores | + +## Release-Binary Inspection + +Run: + +```text +just check-zeroize-evidence +``` + +[`scripts/check/zeroize-evidence.sh`](../scripts/check/zeroize-evidence.sh) +builds an optimized, single-codegen-unit library with the diagnostic entry +points needed to make each lifecycle shape observable. It requires every entry +point in release MIR, LLVM IR, and assembly, then checks volatile LLVM zero +stores and host-architecture zero-store instructions. + +The gate maps evidence to production behavior as follows: + +| Evidence entry point | Boundary made observable | +|---|---| +| `diag_zeroize_fixed_stack`, `diag_zeroize_variable_heap` | Inline and heap owner drop | +| `diag_zeroize_fixed_move`, `diag_zeroize_early_return` | Ownership transfer and early return | +| `diag_zeroize_hex_success`, `diag_zeroize_hex_error` | Shared secret-parser success and error cleanup | +| `diag_zeroize_hmac_sha256_finalize`, `diag_zeroize_hmac_sha3_finalize` | SHA-2 and Keccak keyed finalization, temporary cleanup, and owner drop | +| `diag_zeroize_blake3_drop`, `diag_zeroize_blake3_reuse` | Production keyed owner drop and replaced-state cleanup | +| `diag_zeroize_blake3_xof_move`, `diag_zeroize_blake3_xof_consume` | Keyed XOF ownership transfer and destination drop | +| `diag_zeroize_blake3_thread_scratch`, `diag_zeroize_blake3_parallel_scratch` | Thread-local and per-state heap CV wipe before reuse or deallocation | + +This is host-binary evidence, not a universal machine-code proof. The gate must +run on each target whose generated cleanup is being claimed; unsupported +architectures retain the source audit only. + +## Formatting And Error Audit + +[`tests/secret_redaction.rs`](../tests/secret_redaction.rs) holds exact `Debug` +and error snapshots for generic secret wrappers, AEAD keys and contexts, +HMAC/HKDF/KMAC/PBKDF2 state, keyed BLAKE2/BLAKE3 state and XOF readers, +ECDSA/Ed25519 keypairs, X25519 and ML-KEM shared secrets, prepared ML-KEM +decapsulation state, Argon2 context, and representative secret-input errors. +RSA private components, the public-key-only private-key view, private scratch, +`SecretVec`, and Ed25519 expanded state have adjacent unit snapshots where +their internal fixtures are available. + +Secret-key hex errors retain the public offending-byte field for programmatic +inspection but omit that byte from both `Debug` and `Display`. Generic ECDSA +random-source errors omit the caller's payload from `Debug`, `Display`, and the +standard error-source chain; callers can still recover it by explicitly +matching the public `Random` variant. Other reviewed errors contain only +discriminants, public sizes, or opaque verification failures. + +`DisplaySecret` is the deliberate exception: constructing it explicitly opts +into rendering borrowed secret bytes. Feature-gated diagnostic functions may +return their declared result bytes, but they do not implicitly format owning +keys, seeds, nonce material, intermediate state, or unmasked shares through +`Debug` or an error value. diff --git a/docs/secret-ownership.md b/docs/secret-ownership.md new file mode 100644 index 00000000..43636dca --- /dev/null +++ b/docs/secret-ownership.md @@ -0,0 +1,123 @@ +# Secret Ownership And Generic Capabilities + +This inventory identifies named types that retain secret material or +secret-derived state, then records where generic capabilities can duplicate, +format, serialize, or allocate that state. It is the review baseline for +secret-lifetime work; it is not proof that every compiler-created copy is +erased. See [`constant-time.md`](constant-time.md) for timing claims and the +[`THREAT_MODEL`](../THREAT_MODEL.md) for the system boundary. + +The inventory treats keys, private-key state, shared secrets, password/pepper +state, MAC/KDF state, key schedules, and explicitly keyed hash/XOF state as +secret-bearing. Authentication tags and keyed outputs are included because +their concrete owner controls comparison, but they are protocol-visible values +rather than confidential key material. + +Ordinary unkeyed digest/XOF states, public keys, signatures, nonces, +ciphertexts, PHC records, password-hash parameters, and public RSA scratch are +not semantic secret owners. They can still receive sensitive caller data. +Operation-local arrays, scalars, limbs, plaintext buffers, and individual +backend transfer records are grouped here only where their type-level +`Clone`/`Copy` capability affects ownership. Their complete data flow belongs +to the temporary-flow review. + +In the tables below, “explicit duplicate” means a method named +`duplicate_secret()` rather than `Clone`. “Masked” means `Debug` can identify +the type or show public metadata but does not print the owned secret bytes. +The use column explains why a capability exists; it does not pre-approve that +capability for permanent retention. + +## Confidential Public Owners And Views + +| Type or family | Clone / Copy | Debug | Serialization or export | Storage | Capability use | +|---|---|---|---|---|---| +| `SecretBytes` | Neither | Masked | Consuming `expose()` returns a plain array | Inline | Fixed-size integration boundary whose owned source is cleared on extraction | +| `SecretVec` | Neither | Masked | Consuming `into_unprotected_vec()` returns an ordinary allocation | Heap `Vec` | Variable-size RSA private-key export and explicit transfer to APIs that cannot borrow | +| `DisplaySecret<'a>` | Neither | Intentionally prints bytes through both `Display` and `Debug` | Hex formatting only | Borrowed | Explicit opt-in escape hatch for integrations that must render a key; never use it in logs | +| AEAD `*Key` types | Explicit duplicate; no `Clone` or `Copy` | Masked | `SecretBytes` export, hex opt-in, and `serde-secrets` | Inline | Lets an owned cipher context coexist with a caller-retained typed key while making the extra lifetime visible | +| AEAD cipher contexts | Neither | Masked | None | Inline, except boxed RISC-V fixslice AES schedules | Reusable expanded key and authentication subkey state without exposing a generic duplication path | +| ECDSA P-256/P-384 secret keys and keypairs | Explicit duplicate; no `Clone` or `Copy` | Secret keys are masked; keypairs show only the public half | `SecretBytes` export and hex opt-in; no Serde | Inline | Caller-controlled key/keypair duplication for independent signing owners | +| Ed25519 secret key and keypair | Explicit duplicate; no `Clone` or `Copy` | Secret key is masked; keypair shows only the public half | Secret-key `SecretBytes` export, hex opt-in, and `serde-secrets`; no keypair Serde | Inline | Independent signing owners; keypair duplication also copies its expanded secret state deliberately | +| X25519 secret key and shared secret | Explicit duplicate; no `Clone` or `Copy` | Masked | `SecretBytes` export, hex opt-in, and `serde-secrets` | Inline | Explicit transfer of key-agreement material into a separately owned protocol or KDF context | +| ML-KEM decapsulation keys and shared secrets | Explicit duplicate; no `Clone` or `Copy` | Masked | `SecretBytes` export, hex opt-in, and `serde-secrets` | Inline | Explicit transfer of decapsulation or established key material into another owner | +| ML-KEM prepared decapsulation keys | Explicit duplicate; no `Clone` or `Copy` | Masked | `SecretBytes` export; no Serde | Inline | Reuse of validated private arithmetic without making implicit copies | +| `RsaPrivateKey` | Neither | Public key plus a redacted private-components field | PKCS#1/PKCS#8 DER into `SecretVec`; no Serde | Heap-backed big integers and Montgomery state | Standards-compatible private-key storage/export while keeping the returned allocation typed as secret | +| `RsaPrivateScratch` | Neither | Public sizing metadata only | None | Reusable heap buffers and limb vectors | Amortizes private-operation allocation while keeping intermediate ownership with the caller | +| `RsaPrivateKeyParts<'a>` | `Clone + Copy` | No `Debug` | Borrowed import fields; no Serde | Borrowed | Pass-by-value import description; copying duplicates references, not private bytes | +| `RsaSignatureSigner<'a>` | `Clone + Copy` | Signature profile only | None | Borrowed | Reusable profile-bound signing handle; copying duplicates a private-key reference, not the key | +| HMAC-SHA-2 and HMAC-SHA-3 states | Neither | Masked | None | Inline | Reuse through `reset`; independent streamed owners require explicit keyed construction rather than an implicit state copy | +| HKDF-SHA-2 states | Neither | Masked | None | Inline | Repeated expansion borrows one extracted PRK owner, which can also be shared by reference | +| KMAC128/256 states | Neither | Masked | None | Inline | Reuse through `reset`; private cSHAKE snapshots implement non-consuming finalization and reset without exposing a public keyed-state copy | +| PBKDF2-SHA-2 states | Neither | Masked | None | Inline | Repeated derivation borrows one password-prefix owner, which can also be shared by reference | +| `Poly1305OneTimeKey` and `Poly1305` | Neither | Masked | `SecretBytes` key export; no Serde | Inline | Enforces one-time key consumption and consuming finalization | +| `Argon2Context<'a>` | `Clone + Copy` | Secret bytes redacted; secret and associated-data lengths shown | None | Borrowed | Pass-by-value optional pepper/associated-data configuration without duplicating either byte string | +| BLAKE2 keyed parameter builders and variable-output state | Neither | Key bytes masked; builders show key length, salt, and personalization | None | Inline | Reuse by borrowing; independent keyed owners require explicit construction | +| Fixed-output BLAKE2 states | `Clone` | Masked | None | Inline | The same type serves unkeyed `Digest` and keyed modes, so the public `Digest: Clone` contract also permits keyed prefix forks | +| `Blake3` | `Clone` | Masked | None | Inline, plus `Vec` scratch with `parallel` | Fork a streamed common prefix in keyed or derive-key mode; cloning also duplicates initialized parallel scratch | +| `Blake3XofReader` | `Clone` | Masked | None | Inline | Checkpoint or fork an output cursor; the reader is secret-bearing when created from keyed or derive-key state | + +## Protocol-Visible Authentication Owners + +These owners provide constant-work comparison semantics, but their bytes are +normally transmitted or stored with the protected message. Copying, rendering, +and ordinary serialization therefore do not duplicate a confidential key. + +| Type or family | Clone / Copy | Debug | Serialization | Storage | Capability use | +|---|---|---|---|---|---| +| AEAD tags | `Clone + Copy` | Raw hex | `serde` | Inline | Detached-tag wire formats and cheap by-value API use | +| HMAC-SHA-2 and HMAC-SHA-3 tags | `Clone + Copy` | Raw hex | `serde` | Inline | Protocol transport plus sealed `ct_eq` verification | +| `Poly1305Tag` | `Clone + Copy` | Raw hex | None | Inline | Detached one-time authenticator transport and verification | +| `Blake3KeyedHash` | `Clone + Copy` | Raw hex | None | Inline | Protocol-visible keyed output with sealed `ct_eq` verification | + +## Internal And Operation-Scoped Owners + +| Owner | Clone / Copy | Debug / serialization | Storage | Capability use | +|---|---|---|---|---| +| `ZeroizingBytes` | Neither | Neither | Inline | Generation and parsing scratch that cannot escape as a generic clone | +| AES expanded schedules | Neither | Neither | Inline; boxed only for the large RISC-V fixslice schedule | Retained by an AEAD context and borrowed by block operations; unused private `Clone` derives were removed during this inventory | +| AEAD authentication working state | Private copies only where a backend finalizer consumes a value | Neither | Inline | Bound, intra-operation snapshot needed by consuming backend finalization | +| HMAC-SHA-3 and KMAC Keccak/cSHAKE snapshots | Private use of `Clone` | Neither | Inline | Implement non-consuming finalization and reset inside one public keyed owner | +| Ed25519 `ExpandedSecret` | Private `Clone` | Masked; no serialization | Inline | Implements the public keypair's explicit `duplicate_secret()` operation | +| X25519 clamped scalar and ECDSA secret scalar/word wrappers | No generic duplication on the owning wrappers | Neither | Inline | Bound one-operation arithmetic ownership | +| ML-KEM prepared decapsulation arithmetic | Private `Clone` | Neither | Inline | Implements the prepared key's explicit `duplicate_secret()` operation | +| RSA private components, buffers, limbs, and key-generation DRBG | Neither | Private integers are masked; no serialization | Heap for variable-width integers/scratch; DRBG inline | Variable-width private arithmetic, reusable scratch, and generated-key construction | +| Argon2 matrix | `MemoryBlock` and borrowed `MatrixView` are `Clone + Copy`; `Matrix` is not | Neither | Heap `Vec` | Algorithm-defined whole-block mixing and disjoint parallel lane views; copying the view duplicates only a pointer/length pair | +| scrypt working state | `SalsaBlock` is `Clone + Copy`; owning states are not | Neither | Heap vectors | Algorithm-defined whole-block ROMix copies while one owner retains the complete work area | +| BLAKE3 parallel scratch | Cloned with `Blake3` | Neither | Per-state vectors plus thread-local vectors | Avoid repeated allocation during parallel subtree reduction; keyed modes make stored chaining values secret-derived | +| Private-key and AEAD backend transfer records | Private `Clone`/`Copy` only where passed by value or snapshotted by a consuming kernel | Neither | Inline | Fixed-layout, call-scoped handoff to portable, SIMD, or assembly code; no public capability | + +## Review Consequences + +- `serde` alone covers protocol-visible values. `serde-secrets` is the explicit + opt-in for AEAD keys, Ed25519/X25519 secrets, and ML-KEM decapsulation/shared + secrets. ECDSA and RSA private keys do not implement Serde. +- `DisplaySecret` is the only generic formatting wrapper that intentionally + makes secret bytes visible through `Debug`; constructing it is the disclosure + decision. +- Heap-backed secret state is confined to `SecretVec`, RSA private-key import, + generation, keys, and scratch, Argon2 and scrypt work areas, BLAKE3 parallel + scratch, and the RISC-V fixslice AES schedule. Caller-owned plaintext and + derived-output buffers are outside this named-owner inventory. +- HMAC, HKDF, KMAC, PBKDF2, keyed BLAKE2 parameter builders, and variable-output + BLAKE2 state do not expose `Clone`; `Mac` does not require it. Their reusable + operations borrow or reset one owner, and independent keyed state must be + constructed explicitly. +- Fixed-output BLAKE2 and BLAKE3 hash/XOF states retain `Clone` because the same + types serve ordinary unkeyed `Digest`/`Xof` use. A clone made in keyed or + derive-key mode is another secret lifetime even though the type cannot encode + that distinction. +- Private-key owners remain non-`Clone`. Borrowed RSA views copy references + only, while the private clones behind Ed25519 and ML-KEM + `duplicate_secret()` make the requested owned duplication explicit at the + public call site. +- Wiping success, failure, early-return, move, reuse, and drop paths—including + BLAKE3 XOF and reusable heap scratch—is mapped to source and optimized-binary + evidence in [`secret-lifecycle.md`](secret-lifecycle.md). + +When this surface changes, review the public re-exports and the implementations +under [`src/secret.rs`](../src/secret.rs), [`src/hex.rs`](../src/hex.rs), +[`src/aead`](../src/aead), [`src/auth`](../src/auth), and +[`src/hashes/crypto`](../src/hashes/crypto). Search derives and manual trait +implementations for `Clone`, `Copy`, and `Debug`; search Serde macros and +implementations under both feature gates; and search owned `Vec`/`Box` fields. +Every new capability needs a concrete use in this document or must be removed. diff --git a/docs/types.md b/docs/types.md index 91e85231..12787bf3 100644 --- a/docs/types.md +++ b/docs/types.md @@ -246,8 +246,12 @@ Generic secret wrappers deliberately do not implement equality. Fixed-size keys, shared secrets, authentication tags, and keyed outputs compare only through their concrete semantic owner types. -Secret keys, shared secrets, keypairs, and AEAD cipher contexts do not implement -`Clone`. Where duplication is necessary, call the concrete type's -`duplicate_secret()` method so the additional secret lifetime is visible at the -call site. RSA private-key DER exports return `SecretVec`; borrow the encoded -bytes for parsers and writers whenever possible. +Secret keys, shared secrets, keypairs, AEAD cipher contexts, and keyed +HMAC/HKDF/KMAC/PBKDF2 states do not implement `Clone`. Where key duplication is +necessary, call the concrete type's `duplicate_secret()` method so the +additional secret lifetime is visible at the call site. RSA private-key DER +exports return `SecretVec`; borrow the encoded bytes for parsers and writers +whenever possible. See +[`secret-ownership.md`](secret-ownership.md) for the complete Clone, Copy, +Debug, serialization, and heap-ownership inventory, and +[`secret-lifecycle.md`](secret-lifecycle.md) for cleanup and redaction evidence. diff --git a/scripts/README.md b/scripts/README.md index c52fb7ec..10ad987f 100644 --- a/scripts/README.md +++ b/scripts/README.md @@ -49,6 +49,10 @@ its caller. | `ct/python.sh` | CT recipes and Python-backed CT, check, and release scripts | | `update/update-all.sh` | `just update`, `just update-check` | +The optimized secret-lifecycle inspection performed by +`check/zeroize-evidence.sh` is mapped to its source ownership and host-binary +claim in [`docs/secret-lifecycle.md`](../docs/secret-lifecycle.md). + ## Cross-platform Check Helpers | Script | Callers | diff --git a/scripts/check/zeroize-evidence.sh b/scripts/check/zeroize-evidence.sh index 09298de4..0e0fdeb2 100755 --- a/scripts/check/zeroize-evidence.sh +++ b/scripts/check/zeroize-evidence.sh @@ -10,7 +10,7 @@ RUSTC_WRAPPER="" CARGO_TARGET_DIR="$TARGET_DIR" cargo rustc \ --release \ --lib \ --no-default-features \ - --features alloc,aes-gcm,diag \ + --features alloc,aes-gcm,blake3,hmac,hmac-sha3,parallel,diag \ -- \ -Ccodegen-units=1 \ --emit=mir,llvm-ir,asm @@ -40,9 +40,19 @@ fi for symbol in \ diag_zeroize_fixed_stack \ + diag_zeroize_fixed_move \ + diag_zeroize_early_return \ diag_zeroize_variable_heap \ diag_zeroize_hex_success \ - diag_zeroize_hex_error; do + diag_zeroize_hex_error \ + diag_zeroize_blake3_drop \ + diag_zeroize_blake3_reuse \ + diag_zeroize_blake3_xof_move \ + diag_zeroize_blake3_xof_consume \ + diag_zeroize_blake3_thread_scratch \ + diag_zeroize_blake3_parallel_scratch \ + diag_zeroize_hmac_sha256_finalize \ + diag_zeroize_hmac_sha3_finalize; do if ! grep -q "@$symbol" "$LLVM_IR"; then echo "zeroize LLVM evidence missing symbol: $symbol" >&2 exit 1 @@ -57,7 +67,21 @@ for symbol in \ fi done -for symbol in diag_zeroize_fixed_stack diag_zeroize_variable_heap; do +for symbol in \ + diag_zeroize_fixed_stack \ + diag_zeroize_fixed_move \ + diag_zeroize_early_return \ + diag_zeroize_variable_heap \ + diag_zeroize_hex_success \ + diag_zeroize_hex_error \ + diag_zeroize_blake3_drop \ + diag_zeroize_blake3_reuse \ + diag_zeroize_blake3_xof_move \ + diag_zeroize_blake3_xof_consume \ + diag_zeroize_blake3_thread_scratch \ + diag_zeroize_blake3_parallel_scratch \ + diag_zeroize_hmac_sha256_finalize \ + diag_zeroize_hmac_sha3_finalize; do FUNCTION_IR="$(sed -n "/define .*@$symbol(/,/^}/p" "$LLVM_IR")" VOLATILE_STORES="$(grep -c 'store volatile .* 0' <<<"$FUNCTION_IR" || true)" if [[ "$VOLATILE_STORES" -lt 1 ]]; then @@ -66,6 +90,76 @@ for symbol in diag_zeroize_fixed_stack diag_zeroize_variable_heap; do fi done +BLAKE3_DROP_WRAPPER="$(sed -n '/define .*@diag_zeroize_blake3_drop(/,/^}/p' "$LLVM_IR")" +BLAKE3_DROP_SYMBOL="$(sed -n 's/.*call .*@\([^ (]*drop_in_place[^ (]*Blake3[^ (]*\).*/\1/p' \ + <<<"$BLAKE3_DROP_WRAPPER" | head -n 1)" +if [[ -z "$BLAKE3_DROP_SYMBOL" ]]; then + echo "zeroize LLVM evidence does not route BLAKE3 cleanup through its production Drop" >&2 + exit 1 +fi + +BLAKE3_DROP_IR="$(sed -n "/define .*@$BLAKE3_DROP_SYMBOL(/,/^}/p" "$LLVM_IR")" +BLAKE3_DROP_STORES="$(grep -c 'store volatile .* 0' <<<"$BLAKE3_DROP_IR" || true)" +if [[ "$BLAKE3_DROP_STORES" -lt 8 ]] || ! grep -q "$BLAKE3_DROP_SYMBOL" "$ASSEMBLY"; then + echo "zeroize release evidence does not retain BLAKE3 owner and heap-scratch cleanup" >&2 + exit 1 +fi + +BLAKE3_REUSE_IR="$(sed -n '/define .*@diag_zeroize_blake3_reuse(/,/^}/p' "$LLVM_IR")" +BLAKE3_REUSE_DROPS="$(grep -c '^[[:space:]]*call .*drop_in_place.*Blake3' <<<"$BLAKE3_REUSE_IR" || true)" +if [[ "$BLAKE3_REUSE_DROPS" -lt 2 ]]; then + echo "zeroize release evidence does not wipe both replaced and final BLAKE3 state" >&2 + exit 1 +fi + +BLAKE3_MOVE_IR="$(sed -n '/define .*@diag_zeroize_blake3_xof_move(/,/^}/p' "$LLVM_IR")" +if ! grep -q '@diag_zeroize_blake3_xof_consume' <<<"$BLAKE3_MOVE_IR"; then + echo "zeroize release evidence does not retain the keyed XOF ownership move" >&2 + exit 1 +fi + +BLAKE3_THREAD_SCRATCH_IR="$(sed -n '/define .*@diag_zeroize_blake3_thread_scratch(/,/^}/p' "$LLVM_IR")" +if [[ "$(grep -c 'store volatile .* 0' <<<"$BLAKE3_THREAD_SCRATCH_IR" || true)" -lt 10 ]]; then + echo "zeroize release evidence does not clear both BLAKE3 thread-local CV vectors" >&2 + exit 1 +fi + +BLAKE3_PARALLEL_SCRATCH_IR="$(sed -n '/define .*@diag_zeroize_blake3_parallel_scratch(/,/^}/p' "$LLVM_IR")" +if [[ "$(grep -c 'store volatile .* 0' <<<"$BLAKE3_PARALLEL_SCRATCH_IR" || true)" -lt 9 ]]; then + echo "zeroize release evidence does not clear BLAKE3 per-state heap scratch" >&2 + exit 1 +fi + +HMAC_SHA3_IR="$(sed -n '/define .*@diag_zeroize_hmac_sha3_finalize(/,/^}/p' "$LLVM_IR")" +HMAC_SHA3_STORES="$(grep -c 'store volatile .* 0' <<<"$HMAC_SHA3_IR" || true)" +if [[ "$HMAC_SHA3_STORES" -lt 25 ]]; then + echo "zeroize release evidence does not retain HMAC-SHA3 finalization cleanup" >&2 + exit 1 +fi + +HMAC_SHA256_IR="$(sed -n '/define .*@diag_zeroize_hmac_sha256_finalize(/,/^}/p' "$LLVM_IR")" +HMAC_SHA256_STORES="$(grep -c 'store volatile .* 0' <<<"$HMAC_SHA256_IR" || true)" +if [[ "$HMAC_SHA256_STORES" -lt 25 ]]; then + echo "zeroize release evidence does not retain HMAC-SHA256 finalization cleanup" >&2 + exit 1 +fi +HMAC_SHA256_FINALIZE_SYMBOL="$(sed -n 's/.*call .*@\([^ (]*HmacSha256[^ (]*Mac8finalize[^ (]*\).*/\1/p' \ + <<<"$HMAC_SHA256_IR" | head -n 1)" +HMAC_SHA256_FINALIZE_IR="$(sed -n "/define .*@$HMAC_SHA256_FINALIZE_SYMBOL(/,/^}/p" "$LLVM_IR")" +HMAC_SHA256_SECRET_FINALIZE_CALLS="$(grep -c 'call .*finalize_secret' <<<"$HMAC_SHA256_FINALIZE_IR" || true)" +HMAC_SHA256_SECRET_FINALIZE_SYMBOL="$(sed -n 's/.*call .*@\([^ (]*finalize_secret[^ (]*\).*/\1/p' \ + <<<"$HMAC_SHA256_FINALIZE_IR" | head -n 1)" +if [[ -z "$HMAC_SHA256_FINALIZE_SYMBOL" || "$HMAC_SHA256_SECRET_FINALIZE_CALLS" -lt 2 || \ + -z "$HMAC_SHA256_SECRET_FINALIZE_SYMBOL" ]]; then + echo "zeroize release evidence does not route both HMAC-SHA256 snapshots through secret finalization" >&2 + exit 1 +fi +HMAC_SHA256_SECRET_FINALIZE_IR="$(sed -n "/define .*@$HMAC_SHA256_SECRET_FINALIZE_SYMBOL(/,/^}/p" "$LLVM_IR")" +if [[ "$(grep -c 'store volatile .* 0' <<<"$HMAC_SHA256_SECRET_FINALIZE_IR" || true)" -lt 2 ]]; then + echo "zeroize release evidence does not clear SHA-256 finalization snapshots" >&2 + exit 1 +fi + HEX_ERROR_IR="$(sed -n '/define .*@diag_zeroize_hex_error(/,/^}/p' "$LLVM_IR")" HEX_SUCCESS_IR="$(sed -n '/define .*@diag_zeroize_hex_success(/,/^}/p' "$LLVM_IR")" HEX_FROM_STR="$(sed -n 's/.*call .*@\([^ (]*FromStr8from_str\).*/\1/p' <<<"$HEX_ERROR_IR" | head -n 1)" @@ -81,11 +175,17 @@ if [[ "$HEX_VOLATILE_STORES" -lt 2 ]]; then exit 1 fi -FIXED_ASSEMBLY="$(awk ' - $0 == "diag_zeroize_fixed_stack:" || $0 == "_diag_zeroize_fixed_stack:" { found = 1 } - found && emitted && $0 ~ /^[[:space:]]*\.globl[[:space:]]/ { exit } - found { print; emitted = 1 } -' "$ASSEMBLY")" +function_assembly() { + local symbol="$1" + awk -v plain="$symbol:" -v apple="_$symbol:" ' + $0 == plain || $0 == apple { found = 1 } + found && emitted && $0 ~ /^[[:space:]]*\.globl[[:space:]]/ { exit } + found { print } + found { emitted = 1 } + ' "$ASSEMBLY" +} + +FIXED_ASSEMBLY="$(function_assembly diag_zeroize_fixed_stack)" HOST_ARCH="$(rustc -vV | sed -n 's/^host: \([^-]*\).*/\1/p')" case "$HOST_ARCH" in aarch64) @@ -94,12 +194,46 @@ case "$HOST_ARCH" in echo "zeroize assembly evidence does not show the fixed-size stack spill and wipe" >&2 exit 1 fi + for symbol in \ + diag_zeroize_fixed_move \ + diag_zeroize_early_return \ + diag_zeroize_variable_heap \ + diag_zeroize_hex_success \ + diag_zeroize_hex_error \ + diag_zeroize_blake3_xof_consume \ + diag_zeroize_blake3_thread_scratch \ + diag_zeroize_blake3_parallel_scratch \ + diag_zeroize_hmac_sha256_finalize \ + diag_zeroize_hmac_sha3_finalize; do + FUNCTION_ASSEMBLY="$(function_assembly "$symbol")" + if ! grep -Eq 'st(p|r)(b|h)?[[:space:]].*(wzr|xzr)' <<<"$FUNCTION_ASSEMBLY"; then + echo "zeroize assembly evidence has no zero store in $symbol" >&2 + exit 1 + fi + done ;; x86_64) if ! grep -Eq '%rsp' <<<"$FIXED_ASSEMBLY" || ! grep -Eq "mov[bql]?[[:space:]]+\\\$0" <<<"$FIXED_ASSEMBLY"; then echo "zeroize assembly evidence does not show the fixed-size stack spill and wipe" >&2 exit 1 fi + for symbol in \ + diag_zeroize_fixed_move \ + diag_zeroize_early_return \ + diag_zeroize_variable_heap \ + diag_zeroize_hex_success \ + diag_zeroize_hex_error \ + diag_zeroize_blake3_xof_consume \ + diag_zeroize_blake3_thread_scratch \ + diag_zeroize_blake3_parallel_scratch \ + diag_zeroize_hmac_sha256_finalize \ + diag_zeroize_hmac_sha3_finalize; do + FUNCTION_ASSEMBLY="$(function_assembly "$symbol")" + if ! grep -Eq "mov[bql]?[[:space:]]+\\\$0" <<<"$FUNCTION_ASSEMBLY"; then + echo "zeroize assembly evidence has no zero store in $symbol" >&2 + exit 1 + fi + done ;; esac diff --git a/src/aead/aes.rs b/src/aead/aes.rs index defe4a89..aefb1623 100644 --- a/src/aead/aes.rs +++ b/src/aead/aes.rs @@ -108,12 +108,10 @@ pub(crate) struct X86GcmTables<'a> { /// raw 32-byte key for the KM instruction; on powerpc64 with POWER8 crypto, /// stores round keys as 128-bit vectors. Otherwise stores 60 big-endian /// u32 words for the portable path. Zeroized on drop. -#[derive(Clone)] pub(crate) struct Aes256EncKey { inner: KeyInner, } -#[derive(Clone)] enum KeyInner { #[allow(dead_code)] PortableRoundKeys([u32; EXPANDED_KEY_WORDS]), @@ -192,12 +190,10 @@ impl Drop for Aes256EncKey { /// /// Per-backend representation mirrors [`Aes256EncKey`] but stores 11 round /// keys (176 bytes) instead of 15. Zeroized on drop. -#[derive(Clone)] pub(crate) struct Aes128EncKey { inner: Key128Inner, } -#[derive(Clone)] enum Key128Inner { #[allow(dead_code)] PortableRoundKeys([u32; EXPANDED_KEY_WORDS_128]), diff --git a/src/aead/aes/aarch64_ce.rs b/src/aead/aes/aarch64_ce.rs index 014ec25e..f0a128ef 100644 --- a/src/aead/aes/aarch64_ce.rs +++ b/src/aead/aes/aarch64_ce.rs @@ -5,7 +5,6 @@ use core::arch::aarch64::*; mod asm; /// AES-256 round keys stored as 15 × 128-bit NEON vectors for AES-CE. -#[derive(Clone)] #[repr(C, align(64))] pub(in crate::aead) struct CeRoundKeys { rk: [uint8x16_t; 15], @@ -1400,7 +1399,6 @@ pub(super) unsafe fn encrypt_block(keys: &CeRoundKeys, block: &mut [u8; 16]) { /// AES-128-GCM-SIV path in `aes128gcmsiv.rs` can hold these round keys /// across the `#[target_feature(enable = "aes,neon")]` scope via /// `aes::aarch64_expand_key_128_inline` / `aes::aarch64_encrypt_block_128_inline`. -#[derive(Clone)] #[repr(C, align(64))] pub(in crate::aead) struct Ce128RoundKeys { rk: [uint8x16_t; 11], diff --git a/src/aead/aes/powerpc64_ppc.rs b/src/aead/aes/powerpc64_ppc.rs index 61ecd4e3..50c785c1 100644 --- a/src/aead/aes/powerpc64_ppc.rs +++ b/src/aead/aes/powerpc64_ppc.rs @@ -4,7 +4,6 @@ use core::{arch::asm, simd::i64x2}; /// /// POWER8 vcipher expects round keys in big-endian byte order, which /// matches our portable key schedule (stored as big-endian u32 words). -#[derive(Clone)] #[repr(C, align(64))] pub(in crate::aead) struct PpcRoundKeys { rk: [i64x2; 15], @@ -354,7 +353,6 @@ pub(super) unsafe fn encrypt_block(keys: &PpcRoundKeys, block: &mut [u8; 16]) { // AES-128 (11 round keys, 10 rounds) /// AES-128 round keys stored as 11 × 128-bit vectors for POWER8 vcipher. -#[derive(Clone)] #[repr(C, align(64))] pub(in crate::aead) struct Ppc128RoundKeys { rk: [i64x2; 11], diff --git a/src/aead/aes/riscv64_aes.rs b/src/aead/aes/riscv64_aes.rs index 898c0dae..69c3fb18 100644 --- a/src/aead/aes/riscv64_aes.rs +++ b/src/aead/aes/riscv64_aes.rs @@ -4,7 +4,6 @@ use core::arch::asm; /// /// Round keys are loaded from memory into vector registers per call /// since RISC-V `vreg` is clobber-only (cannot be used as input/output). -#[derive(Clone)] #[repr(C, align(64))] pub(super) struct RvRoundKeys { rk: [[u8; 16]; 15], @@ -252,7 +251,6 @@ pub(super) unsafe fn encrypt_4blocks(keys: &RvRoundKeys, blocks: &mut [[u8; 16]; // AES-128 (11 round keys, 10 rounds) /// AES-128 round keys stored as 11 × 16-byte arrays for Zvkned. -#[derive(Clone)] #[repr(C, align(64))] pub(super) struct Rv128RoundKeys { rk: [[u8; 16]; 11], diff --git a/src/aead/aes/riscv64_fixslice_aes.rs b/src/aead/aes/riscv64_fixslice_aes.rs index e856cb4e..19002124 100644 --- a/src/aead/aes/riscv64_fixslice_aes.rs +++ b/src/aead/aes/riscv64_fixslice_aes.rs @@ -14,7 +14,6 @@ use super::{BLOCK_SIZE, KEY_SIZE, KEY_SIZE_128}; type State = [u64; 8]; -#[derive(Clone)] pub(super) struct RvFixsliceRoundKeys { keys: [u64; 120], } @@ -39,7 +38,6 @@ impl RvFixsliceRoundKeys { } /// Bitsliced AES-128 round keys (10 rounds → 11 round keys × 8 u64 per group). -#[derive(Clone)] pub(super) struct RvFixslice128RoundKeys { keys: [u64; 88], } diff --git a/src/aead/aes/riscv64_scalar_aes.rs b/src/aead/aes/riscv64_scalar_aes.rs index 36bc804d..13831556 100644 --- a/src/aead/aes/riscv64_scalar_aes.rs +++ b/src/aead/aes/riscv64_scalar_aes.rs @@ -4,7 +4,6 @@ use core::arch::riscv64::{aes64es, aes64esm}; /// /// Each round key is kept in canonical AES byte order and split into the /// upper/lower 64-bit halves consumed by the RV64 scalar AES instructions. -#[derive(Clone)] #[repr(C, align(64))] pub(super) struct RvScalarRoundKeys { rk: [(u64, u64); 15], @@ -152,7 +151,6 @@ pub(super) unsafe fn encrypt_4blocks(keys: &RvScalarRoundKeys, blocks: &mut [[u8 // AES-128 (11 round keys, 10 rounds) /// AES-128 round keys stored as 11 pairs of 64-bit halves for scalar AES. -#[derive(Clone)] #[repr(C, align(64))] pub(super) struct RvScalar128RoundKeys { rk: [(u64, u64); 11], diff --git a/src/aead/aes/s390x_km.rs b/src/aead/aes/s390x_km.rs index 6daecc3b..43a49a92 100644 --- a/src/aead/aes/s390x_km.rs +++ b/src/aead/aes/s390x_km.rs @@ -6,7 +6,6 @@ use super::{EXPANDED_KEY_WORDS, EXPANDED_KEY_WORDS_128}; /// alongside the full expanded schedule. This avoids 8 BE serializations /// per `encrypt_block` call — critical for GCM-SIV which does 7 AES /// calls for even a 0-byte message. -#[derive(Clone)] #[repr(C, align(8))] pub(in crate::aead) struct KmKey { /// Raw 32-byte AES-256 key, ready for the KM parameter block. @@ -182,7 +181,6 @@ pub(super) unsafe fn encrypt_blocks(key: &KmKey, blocks: &mut [u8], count: usize // AES-128 (KM function code 18) /// AES-128 key for the KM (Cipher Message) instruction. -#[derive(Clone)] #[repr(C, align(8))] pub(in crate::aead) struct Km128Key { /// Raw 16-byte AES-128 key, ready for the KM parameter block. diff --git a/src/aead/aes/x86_64_ni.rs b/src/aead/aes/x86_64_ni.rs index aa4eacb3..f441cc19 100644 --- a/src/aead/aes/x86_64_ni.rs +++ b/src/aead/aes/x86_64_ni.rs @@ -1,7 +1,6 @@ use core::arch::x86_64::*; /// AES-256 round keys stored as 15 × 128-bit values for AES-NI. -#[derive(Clone)] #[repr(C, align(16))] pub(super) struct NiRoundKeys { rk: [__m128i; 15], @@ -309,7 +308,6 @@ pub(super) unsafe fn encrypt_block(keys: &NiRoundKeys, block: &mut [u8; 16]) { // AES-128 (11 round keys, 10 rounds) /// AES-128 round keys stored as 11 × 128-bit values for AES-NI. -#[derive(Clone)] #[repr(C, align(16))] pub(super) struct Ni128RoundKeys { rk: [__m128i; 11], diff --git a/src/auth/ecdsa.rs b/src/auth/ecdsa.rs index fd00b378..a93a1878 100644 --- a/src/auth/ecdsa.rs +++ b/src/auth/ecdsa.rs @@ -752,7 +752,7 @@ impl fmt::Display for EcdsaError { impl core::error::Error for EcdsaError {} /// Errors returned by fallible ECDSA key generation. -#[derive(Clone, Copy, Debug, PartialEq, Eq)] +#[derive(Clone, Copy, PartialEq, Eq)] pub enum EcdsaKeyGenerationError { /// The caller-provided random source failed. Random(E), @@ -760,6 +760,15 @@ pub enum EcdsaKeyGenerationError { InvalidSecretKey, } +impl fmt::Debug for EcdsaKeyGenerationError { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + match self { + Self::Random(_) => f.write_str("Random(..)"), + Self::InvalidSecretKey => f.write_str("InvalidSecretKey"), + } + } +} + impl EcdsaKeyGenerationError { /// Construct a random-source error. #[inline] @@ -785,17 +794,7 @@ impl fmt::Display for EcdsaKeyGenerationError { } } -impl core::error::Error for EcdsaKeyGenerationError -where - E: core::error::Error + 'static, -{ - fn source(&self) -> Option<&(dyn core::error::Error + 'static)> { - match self { - Self::Random(err) => Some(err), - Self::InvalidSecretKey => None, - } - } -} +impl core::error::Error for EcdsaKeyGenerationError where E: core::error::Error + 'static {} /// P-256 ECDSA secret scalar. pub struct EcdsaP256SecretKey([u8; Self::LENGTH]); diff --git a/src/auth/ed25519.rs b/src/auth/ed25519.rs index 6cbe4949..7d48359d 100644 --- a/src/auth/ed25519.rs +++ b/src/auth/ed25519.rs @@ -585,7 +585,8 @@ fn sign_with_expanded(expanded: &hash::ExpandedSecret, public: &Ed25519PublicKey let mut nonce_hasher = Sha512::new(); nonce_hasher.update(expanded.nonce_prefix()); nonce_hasher.update(message); - let mut nonce_digest = nonce_hasher.finalize(); + let mut nonce_digest = nonce_hasher.finalize_secret(); + nonce_hasher.zeroize_secret_state(); let mut nonce_scalar = scalar::reduce_64_bytes_mod_order_secret(&nonce_digest); let mut nonce_bytes = scalar::to_bytes(&nonce_scalar); let r_encoded = basepoint_mul_encoded_dispatch(&nonce_bytes); diff --git a/src/auth/ed25519/hash.rs b/src/auth/ed25519/hash.rs index 6e81df67..f4e1776c 100644 --- a/src/auth/ed25519/hash.rs +++ b/src/auth/ed25519/hash.rs @@ -26,7 +26,7 @@ impl ExpandedSecret { /// Expand a 32-byte Ed25519 secret key into clamped scalar bytes and nonce prefix. #[must_use] pub(crate) fn from_secret_key(secret: &Ed25519SecretKey) -> Self { - let mut digest = Sha512::digest(secret.as_bytes()); + let mut digest = Sha512::digest_secret(secret.as_bytes()); let (scalar_source, prefix_source) = digest.as_slice().split_at(SECRET_KEY_LENGTH); let mut scalar_bytes = [0u8; SECRET_KEY_LENGTH]; @@ -42,10 +42,14 @@ impl ExpandedSecret { ct::zeroize(&mut digest); clamp_secret_scalar(&mut scalar_bytes); - Self { + let expanded = Self { scalar_bytes, nonce_prefix, - } + }; + ct::zeroize_no_fence(&mut scalar_bytes); + ct::zeroize_no_fence(&mut nonce_prefix); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + expanded } /// Return the clamped scalar bytes. diff --git a/src/auth/hkdf.rs b/src/auth/hkdf.rs index 1b347a7a..ea764ce4 100644 --- a/src/auth/hkdf.rs +++ b/src/auth/hkdf.rs @@ -110,7 +110,6 @@ define_unit_error! { /// assert_eq!(okm, oneshot); /// # Ok::<(), rscrypto::auth::HkdfOutputLengthError>(()) /// ``` -#[derive(Clone)] pub struct HkdfSha256 { prk: [u8; SHA256_OUTPUT_SIZE], inner_init: [u32; 8], @@ -355,7 +354,6 @@ impl Drop for HkdfSha256 { /// assert_eq!(okm, oneshot); /// # Ok::<(), rscrypto::auth::HkdfOutputLengthError>(()) /// ``` -#[derive(Clone)] pub struct HkdfSha384 { prk: [u8; SHA384_OUTPUT_SIZE], inner_init: [u64; 8], @@ -569,7 +567,6 @@ impl Drop for HkdfSha384 { /// /// `new()` and `extract()` perform HKDF-Extract and store the pseudorandom key /// for later `expand()` calls. -#[derive(Clone)] pub struct HkdfSha512 { prk: [u8; SHA512_OUTPUT_SIZE], inner_init: [u64; 8], diff --git a/src/auth/hmac.rs b/src/auth/hmac.rs index 4ffb4ec8..c3b664d1 100644 --- a/src/auth/hmac.rs +++ b/src/auth/hmac.rs @@ -329,7 +329,6 @@ fn hmac_outer_pad_words( /// mac.update(b"message"); /// assert!(mac.verify(&tag).is_ok()); /// ``` -#[derive(Clone)] pub struct HmacSha256 { inner: Sha256, inner_init: Sha256Prefix, @@ -374,8 +373,9 @@ impl HmacSha256 { ) -> Self { let mut key_block = [0u8; SHA256_BLOCK_SIZE]; if key.len() > SHA256_BLOCK_SIZE { - let digest = Sha256::digest(key); + let mut digest = Sha256::digest_secret(key); key_block[..SHA256_TAG_SIZE].copy_from_slice(&digest); + ct::zeroize(&mut digest); } else if let Some(dst) = key_block.get_mut(..key.len()) { dst.copy_from_slice(key); } @@ -388,6 +388,7 @@ impl HmacSha256 { let mut outer = Sha256::new_with_compress_for_test(compress); outer.update(opad); let outer_init = outer.aligned_prefix(); + outer.zeroize_secret_state(); (inner, inner_init, outer_init) }); @@ -424,6 +425,17 @@ pub fn diag_hmac_sha256_verify_portable( ct::fixed_eq(&tag, expected) } +#[cfg(feature = "diag")] +#[doc(hidden)] +#[unsafe(no_mangle)] +#[inline(never)] +pub fn diag_zeroize_hmac_sha256_finalize(mut key: [u8; SHA256_TAG_SIZE]) -> u8 { + let mut state = HmacSha256::new(&key); + ct::zeroize(&mut key); + state.update(b"zeroize-finalize"); + core::hint::black_box(state.finalize().as_bytes()[0]) +} + impl Mac for HmacSha256 { const TAG_SIZE: usize = SHA256_TAG_SIZE; type Tag = HmacSha256Tag; @@ -431,10 +443,11 @@ impl Mac for HmacSha256 { fn new(key: &[u8]) -> Self { let mut key_block = [0u8; SHA256_BLOCK_SIZE]; if key.len() > SHA256_BLOCK_SIZE { - let digest = Sha256::digest(key); + let mut digest = Sha256::digest_secret(key); for (dst, src) in key_block.iter_mut().zip(digest.iter()) { *dst = *src; } + ct::zeroize(&mut digest); } else { for (dst, src) in key_block.iter_mut().zip(key.iter()) { *dst = *src; @@ -450,6 +463,7 @@ impl Mac for HmacSha256 { let inner_init_prefix = inner_init.aligned_prefix(); let outer_init_prefix = outer_init.aligned_prefix(); + outer_init.zeroize_secret_state(); (inner_init, inner_init_prefix, outer_init_prefix) }); @@ -468,14 +482,18 @@ impl Mac for HmacSha256 { #[inline] fn finalize(&self) -> Self::Tag { - let inner_hash = self.inner.finalize(); + let mut inner_hash = self.inner.finalize_secret(); let mut outer = Sha256::from_aligned_prefix(self.outer_init); outer.update(&inner_hash); - HmacSha256Tag::from_bytes(outer.finalize()) + ct::zeroize(&mut inner_hash); + let tag = outer.finalize_secret(); + outer.zeroize_secret_state(); + HmacSha256Tag::from_bytes(tag) } #[inline] fn reset(&mut self) { + self.inner.zeroize_secret_state(); self.inner.reset_to_aligned_prefix(self.inner_init); } @@ -499,10 +517,11 @@ impl Mac for HmacSha256 { let mut ipad = [0x36u8; SHA256_BLOCK_SIZE]; if key.len() > SHA256_BLOCK_SIZE { - let digest = Sha256::digest(key); + let mut digest = Sha256::digest_secret(key); for (ip, &kb) in ipad[..SHA256_TAG_SIZE].iter_mut().zip(digest.iter()) { *ip = kb ^ 0x36; } + ct::zeroize(&mut digest); } else { for (ip, &kb) in ipad[..key.len()].iter_mut().zip(key.iter()) { *ip = kb ^ 0x36; @@ -602,6 +621,7 @@ impl Mac for HmacSha256 { impl Drop for HmacSha256 { fn drop(&mut self) { + self.inner.zeroize_secret_state(); self.inner_init.zeroize(); self.outer_init.zeroize(); } @@ -625,7 +645,6 @@ impl Drop for HmacSha256 { /// assert!(mac.verify(&tag).is_ok()); /// assert!(HmacSha384::verify_tag(key, data, &tag).is_ok()); /// ``` -#[derive(Clone)] pub struct HmacSha384 { inner: Sha384, inner_init: Sha384Prefix, @@ -669,8 +688,9 @@ impl HmacSha384 { ) -> Self { let mut key_block = [0u8; SHA512_FAMILY_BLOCK_SIZE]; if key.len() > SHA512_FAMILY_BLOCK_SIZE { - let digest = Sha384::digest(key); + let mut digest = Sha384::digest_secret(key); key_block[..SHA384_TAG_SIZE].copy_from_slice(&digest); + ct::zeroize(&mut digest); } else if let Some(dst) = key_block.get_mut(..key.len()) { dst.copy_from_slice(key); } @@ -683,6 +703,7 @@ impl HmacSha384 { let mut outer = Sha384::new_with_compress_for_test(compress); outer.update(opad); let outer_init = outer.aligned_prefix(); + outer.zeroize_secret_state(); (inner, inner_init, outer_init) }); @@ -725,10 +746,11 @@ impl Mac for HmacSha384 { fn new(key: &[u8]) -> Self { let mut key_block = [0u8; SHA512_FAMILY_BLOCK_SIZE]; if key.len() > SHA512_FAMILY_BLOCK_SIZE { - let digest = Sha384::digest(key); + let mut digest = Sha384::digest_secret(key); for (dst, src) in key_block.iter_mut().zip(digest.iter()) { *dst = *src; } + ct::zeroize(&mut digest); } else { for (dst, src) in key_block.iter_mut().zip(key.iter()) { *dst = *src; @@ -744,6 +766,7 @@ impl Mac for HmacSha384 { let inner_init_prefix = inner_init.aligned_prefix(); let outer_init_prefix = outer_init.aligned_prefix(); + outer_init.zeroize_secret_state(); (inner_init, inner_init_prefix, outer_init_prefix) }); @@ -762,14 +785,18 @@ impl Mac for HmacSha384 { #[inline] fn finalize(&self) -> Self::Tag { - let inner_hash = self.inner.finalize(); + let mut inner_hash = self.inner.finalize_secret(); let mut outer = Sha384::from_aligned_prefix(self.outer_init); outer.update(&inner_hash); - HmacSha384Tag::from_bytes(outer.finalize()) + ct::zeroize(&mut inner_hash); + let tag = outer.finalize_secret(); + outer.zeroize_secret_state(); + HmacSha384Tag::from_bytes(tag) } #[inline] fn reset(&mut self) { + self.inner.zeroize_secret_state(); self.inner.reset_to_aligned_prefix(self.inner_init); } @@ -780,10 +807,11 @@ impl Mac for HmacSha384 { let mut ipad = [0x36u8; SHA512_FAMILY_BLOCK_SIZE]; if key.len() > SHA512_FAMILY_BLOCK_SIZE { - let digest = Sha384::digest(key); + let mut digest = Sha384::digest_secret(key); for (ip, &kb) in ipad[..SHA384_TAG_SIZE].iter_mut().zip(digest.iter()) { *ip = kb ^ 0x36; } + ct::zeroize(&mut digest); } else { for (ip, &kb) in ipad[..key.len()].iter_mut().zip(key.iter()) { *ip = kb ^ 0x36; @@ -879,6 +907,7 @@ impl Mac for HmacSha384 { impl Drop for HmacSha384 { fn drop(&mut self) { + self.inner.zeroize_secret_state(); self.inner_init.zeroize(); self.outer_init.zeroize(); } @@ -902,7 +931,6 @@ impl Drop for HmacSha384 { /// assert!(mac.verify(&tag).is_ok()); /// assert!(HmacSha512::verify_tag(key, data, &tag).is_ok()); /// ``` -#[derive(Clone)] pub struct HmacSha512 { inner: Sha512, inner_init: Sha512Prefix, @@ -946,8 +974,9 @@ impl HmacSha512 { ) -> Self { let mut key_block = [0u8; SHA512_FAMILY_BLOCK_SIZE]; if key.len() > SHA512_FAMILY_BLOCK_SIZE { - let digest = Sha512::digest(key); + let mut digest = Sha512::digest_secret(key); key_block[..SHA512_TAG_SIZE].copy_from_slice(&digest); + ct::zeroize(&mut digest); } else if let Some(dst) = key_block.get_mut(..key.len()) { dst.copy_from_slice(key); } @@ -960,6 +989,7 @@ impl HmacSha512 { let mut outer = Sha512::new_with_compress_for_test(compress); outer.update(opad); let outer_init = outer.aligned_prefix(); + outer.zeroize_secret_state(); (inner, inner_init, outer_init) }); @@ -1002,10 +1032,11 @@ impl Mac for HmacSha512 { fn new(key: &[u8]) -> Self { let mut key_block = [0u8; SHA512_FAMILY_BLOCK_SIZE]; if key.len() > SHA512_FAMILY_BLOCK_SIZE { - let digest = Sha512::digest(key); + let mut digest = Sha512::digest_secret(key); for (dst, src) in key_block.iter_mut().zip(digest.iter()) { *dst = *src; } + ct::zeroize(&mut digest); } else { for (dst, src) in key_block.iter_mut().zip(key.iter()) { *dst = *src; @@ -1021,6 +1052,7 @@ impl Mac for HmacSha512 { let inner_init_prefix = inner_init.aligned_prefix(); let outer_init_prefix = outer_init.aligned_prefix(); + outer_init.zeroize_secret_state(); (inner_init, inner_init_prefix, outer_init_prefix) }); @@ -1039,14 +1071,18 @@ impl Mac for HmacSha512 { #[inline] fn finalize(&self) -> Self::Tag { - let inner_hash = self.inner.finalize(); + let mut inner_hash = self.inner.finalize_secret(); let mut outer = Sha512::from_aligned_prefix(self.outer_init); outer.update(&inner_hash); - HmacSha512Tag::from_bytes(outer.finalize()) + ct::zeroize(&mut inner_hash); + let tag = outer.finalize_secret(); + outer.zeroize_secret_state(); + HmacSha512Tag::from_bytes(tag) } #[inline] fn reset(&mut self) { + self.inner.zeroize_secret_state(); self.inner.reset_to_aligned_prefix(self.inner_init); } @@ -1057,10 +1093,11 @@ impl Mac for HmacSha512 { let mut ipad = [0x36u8; SHA512_FAMILY_BLOCK_SIZE]; if key.len() > SHA512_FAMILY_BLOCK_SIZE { - let digest = Sha512::digest(key); + let mut digest = Sha512::digest_secret(key); for (ip, &kb) in ipad[..SHA512_TAG_SIZE].iter_mut().zip(digest.iter()) { *ip = kb ^ 0x36; } + ct::zeroize(&mut digest); } else { for (ip, &kb) in ipad[..key.len()].iter_mut().zip(key.iter()) { *ip = kb ^ 0x36; @@ -1156,6 +1193,7 @@ impl Mac for HmacSha512 { impl Drop for HmacSha512 { fn drop(&mut self) { + self.inner.zeroize_secret_state(); self.inner_init.zeroize(); self.outer_init.zeroize(); } diff --git a/src/auth/hmac_sha3.rs b/src/auth/hmac_sha3.rs index 08f00155..a88bd445 100644 --- a/src/auth/hmac_sha3.rs +++ b/src/auth/hmac_sha3.rs @@ -221,7 +221,6 @@ fn hmac_sha3_prefix_state( macro_rules! define_hmac_sha3 { ($name:ident, $tag:ident, $rate:expr, $tag_size:expr, $label:literal) => { #[doc = concat!("HMAC-", $label, " authentication state.")] - #[derive(Clone)] pub struct $name { inner: KeccakCore<$rate>, inner_init: KeccakCore<$rate>, @@ -266,8 +265,9 @@ macro_rules! define_hmac_sha3 { fn new(key: &[u8]) -> Self { let mut key_block = [0u8; $rate]; if key.len() > $rate { - let digest = sha3_digest::<$rate, $tag_size>(key); + let mut digest = sha3_digest::<$rate, $tag_size>(key); key_block[..$tag_size].copy_from_slice(&digest); + ct::zeroize(&mut digest); } else { key_block[..key.len()].copy_from_slice(key); } @@ -288,9 +288,10 @@ macro_rules! define_hmac_sha3 { #[inline] fn finalize(&self) -> Self::Tag { - let inner_hash = sha3_finalize::<$rate, $tag_size>(&self.inner); + let mut inner_hash = sha3_finalize::<$rate, $tag_size>(&self.inner); let mut outer = self.outer_init.clone(); outer.update(&inner_hash); + ct::zeroize(&mut inner_hash); $tag::from_bytes(sha3_finalize::<$rate, $tag_size>(&outer)) } @@ -318,6 +319,7 @@ define_hmac_sha3!( SHA3_224_TAG_SIZE, "SHA3-224" ); + define_hmac_sha3!( HmacSha3_256, HmacSha3_256Tag, @@ -339,3 +341,14 @@ define_hmac_sha3!( SHA3_512_TAG_SIZE, "SHA3-512" ); + +#[cfg(feature = "diag")] +#[doc(hidden)] +#[unsafe(no_mangle)] +#[inline(never)] +pub fn diag_zeroize_hmac_sha3_finalize(mut key: [u8; SHA3_256_TAG_SIZE]) -> u8 { + let mut state = HmacSha3_256::new(&key); + ct::zeroize(&mut key); + state.update(b"zeroize-finalize"); + core::hint::black_box(state.finalize().as_bytes()[0]) +} diff --git a/src/auth/kmac.rs b/src/auth/kmac.rs index 3d89126f..a386bf52 100644 --- a/src/auth/kmac.rs +++ b/src/auth/kmac.rs @@ -17,7 +17,6 @@ macro_rules! define_kmac { #[doc = concat!("KMAC", $bits, " keyed state.")] /// KMAC is a variable-output MAC/PRF. It intentionally does not implement /// [`crate::traits::Mac`], which assumes a fixed-size tag. - #[derive(Clone)] pub struct $name { state: $cshake, initial_state: $cshake, diff --git a/src/auth/pbkdf2.rs b/src/auth/pbkdf2.rs index d4a0cff4..839354e4 100644 --- a/src/auth/pbkdf2.rs +++ b/src/auth/pbkdf2.rs @@ -240,7 +240,6 @@ macro_rules! define_pbkdf2_sha2 { } ) => { $(#[$struct_meta])* - #[derive(Clone)] pub struct $name { inner_init: [$word_ty; 8], outer_init: [$word_ty; 8], @@ -302,8 +301,9 @@ macro_rules! define_pbkdf2_sha2 { let mut key_block = [0u8; $block_size_const]; if password.len() > $block_size_const { - let digest = <$digest_ty>::digest(password); + let mut digest = <$digest_ty>::digest_secret(password); key_block[..$output_size_const].copy_from_slice(&digest); + ct::zeroize(&mut digest); } else { key_block[..password.len()].copy_from_slice(password); } @@ -610,7 +610,9 @@ macro_rules! define_pbkdf2_sha2 { pub(crate) fn new_with_compress_for_test(password: &[u8], compress: $compress_ty) -> Self { let mut key_block = [0u8; $block_size_const]; if password.len() > $block_size_const { - key_block[..$output_size_const].copy_from_slice(&$test_oneshot(password, compress)); + let mut digest = $test_oneshot(password, compress); + key_block[..$output_size_const].copy_from_slice(&digest); + ct::zeroize(&mut digest); } else { key_block[..password.len()].copy_from_slice(password); } @@ -713,6 +715,9 @@ fn sha256_oneshot_with_compress(data: &[u8], compress: Sha256CompressBlocksFn) - for (chunk, &word) in out.chunks_exact_mut(4).zip(state.iter()) { chunk.copy_from_slice(&word.to_be_bytes()); } + ct::zeroize_words_no_fence(&mut state); + ct::zeroize_no_fence(&mut block); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); out } @@ -753,8 +758,9 @@ fn pbkdf2_sha256_new_fast_path(password: &[u8]) -> Option { let mut key_block = [0u8; SHA256_BLOCK_SIZE]; if password.len() > SHA256_BLOCK_SIZE { - let digest = Sha256::digest(password); + let mut digest = Sha256::digest_secret(password); key_block[..SHA256_OUTPUT_SIZE].copy_from_slice(&digest); + ct::zeroize(&mut digest); } else { key_block[..password.len()].copy_from_slice(password); } @@ -770,6 +776,8 @@ fn pbkdf2_sha256_new_fast_path(password: &[u8]) -> Option { let (inner_init, compress) = inner_prefix.pbkdf2_parts(); let (outer_init, _) = outer_prefix.pbkdf2_parts(); + inner.zeroize_secret_state(); + outer.zeroize_secret_state(); (inner_init, outer_init, compress) }); @@ -1143,6 +1151,9 @@ fn sha512_oneshot_with_compress(data: &[u8], compress: Sha512CompressBlocksFn) - for (chunk, &word) in out.chunks_exact_mut(8).zip(state.iter()) { chunk.copy_from_slice(&word.to_be_bytes()); } + ct::zeroize_words_no_fence(&mut state); + ct::zeroize_no_fence(&mut block); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); out } diff --git a/src/auth/rsa.rs b/src/auth/rsa.rs index 3b77a1a7..fe33c6ae 100644 --- a/src/auth/rsa.rs +++ b/src/auth/rsa.rs @@ -10795,8 +10795,8 @@ f70203010001a3533051301d0603551d0e04160414fd0e576ce3f05b08884ad67ef3e8b4d39039c6 #[test] fn private_key_components_debug_redacts_secret_material() { let der = test_pkcs1_private_key(); - let key = parse_pkcs1_private_key_der_with_policy(&der, &RsaPublicKeyPolicy::legacy_verification()).unwrap(); - let debug = format!("{key:?}"); + let components = parse_pkcs1_private_key_der_with_policy(&der, &RsaPublicKeyPolicy::legacy_verification()).unwrap(); + let debug = format!("{components:?}"); assert!(debug.contains("modulus_bits")); assert!(debug.contains("public_exponent")); @@ -10814,6 +10814,19 @@ f70203010001a3533051301d0603551d0e04160414fd0e576ce3f05b08884ad67ef3e8b4d39039c6 ] { assert!(!debug.contains(&format!("{secret:?}"))); } + + let key = RsaPrivateKey::from_pkcs1_der_with_policy(&der, &RsaPublicKeyPolicy::legacy_verification()).unwrap(); + assert_eq!( + format!("{key:?}"), + format!( + "RsaPrivateKey {{ public: {:?}, private_components: \"****\", .. }}", + key.public_key() + ) + ); + assert_eq!( + format!("{:?}", key.private_scratch()), + format!("RsaPrivateScratch {{ bytes: {}, .. }}", key.signature_len()) + ); } #[test] diff --git a/src/hashes/crypto/blake2b/mod.rs b/src/hashes/crypto/blake2b/mod.rs index a718857d..beccce2c 100644 --- a/src/hashes/crypto/blake2b/mod.rs +++ b/src/hashes/crypto/blake2b/mod.rs @@ -690,7 +690,6 @@ impl Drop for Core { /// .hash_256(b"message"); /// assert_ne!(tag, other); /// ``` -#[derive(Clone)] pub struct Blake2bParams { key_buf: [u8; MAX_KEY_LEN], key_len: u8, @@ -1160,7 +1159,6 @@ impl_std_io_write_for_digest!(Blake2b512); /// Constructors panic if `output_len` is `0` or greater than `64`. /// `finalize_into` panics if the supplied slice length does not match the /// hasher's configured output length. -#[derive(Clone)] pub struct Blake2b { core: Core, output_len: u8, diff --git a/src/hashes/crypto/blake2s/mod.rs b/src/hashes/crypto/blake2s/mod.rs index 77c08f4d..3c02ceab 100644 --- a/src/hashes/crypto/blake2s/mod.rs +++ b/src/hashes/crypto/blake2s/mod.rs @@ -616,7 +616,6 @@ impl Drop for Core { /// .hash_256(b"message"); /// assert_ne!(tag, other); /// ``` -#[derive(Clone)] pub struct Blake2sParams { key_buf: [u8; MAX_KEY_LEN], key_len: u8, diff --git a/src/hashes/crypto/blake3/mod.rs b/src/hashes/crypto/blake3/mod.rs index 696238d1..9eddae90 100644 --- a/src/hashes/crypto/blake3/mod.rs +++ b/src/hashes/crypto/blake3/mod.rs @@ -222,7 +222,11 @@ fn run_parallel_task(task: impl FnOnce()) -> bool { #[cfg(feature = "parallel")] #[inline] -fn with_subtree_scratch(subtree_chunks: usize, f: impl FnOnce(&mut [[u32; 8]], &mut [[u32; 8]]) -> R) -> R { +fn with_subtree_scratch( + subtree_chunks: usize, + flags: u32, + f: impl FnOnce(&mut [[u32; 8]], &mut [[u32; 8]]) -> R, +) -> R { BLAKE3_SUBTREE_SCRATCH0.with(|s0| { BLAKE3_SUBTREE_SCRATCH1.with(|s1| { let mut scratch0 = s0.borrow_mut(); @@ -233,11 +237,31 @@ fn with_subtree_scratch(subtree_chunks: usize, f: impl FnOnce(&mut [[u32; 8]] if scratch1.len() != subtree_chunks { scratch1.resize(subtree_chunks, [0u32; 8]); } - f(scratch0.as_mut_slice(), scratch1.as_mut_slice()) + let result = f(scratch0.as_mut_slice(), scratch1.as_mut_slice()); + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words_no_fence(scratch0.as_flattened_mut()); + ct::zeroize_words_no_fence(scratch1.as_flattened_mut()); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } + result }) }) } +#[cfg(all(feature = "parallel", feature = "diag"))] +#[doc(hidden)] +#[unsafe(no_mangle)] +#[inline(never)] +pub fn diag_zeroize_blake3_thread_scratch(mut input: [u32; 8]) -> u32 { + let output = with_subtree_scratch(1, KEYED_HASH, |scratch0, scratch1| { + scratch0[0] = input; + scratch1[0] = input; + core::hint::black_box(scratch0[0][0]) + }); + ct::zeroize_words(&mut input); + output +} + #[cfg(feature = "parallel")] #[inline] fn reduce_power_of_two_cvs_in_place( @@ -561,7 +585,7 @@ fn hash_power_of_two_subtree_roots_serial( debug_assert_ne!(subtree_chunks, 0); debug_assert_eq!(input.len(), out.len() * subtree_chunks * CHUNK_LEN); - with_subtree_scratch(subtree_chunks, |scratch0, scratch1| { + with_subtree_scratch(subtree_chunks, flags, |scratch0, scratch1| { for (i, slot) in out.iter_mut().enumerate() { let chunk_base = base_counter.wrapping_add((i * subtree_chunks) as u64); let bytes = &input[i * subtree_chunks * CHUNK_LEN..(i + 1) * subtree_chunks * CHUNK_LEN]; @@ -624,7 +648,7 @@ fn hash_power_of_two_subtree_roots_parallel_rayon(req: SubtreeRootsRequest<'_>) s.spawn(move |_| { #[cfg(test)] let ok = run_parallel_task(|| { - with_subtree_scratch(subtree_chunks, |scratch0, scratch1| { + with_subtree_scratch(subtree_chunks, flags, |scratch0, scratch1| { // SAFETY: `out_ptr` is valid for `out_len` outputs and this task's // range is disjoint from every other task. let out = unsafe { slice::from_raw_parts_mut(out_ptr.get().add(start), end - start) }; @@ -643,7 +667,7 @@ fn hash_power_of_two_subtree_roots_parallel_rayon(req: SubtreeRootsRequest<'_>) #[cfg(not(test))] { run_parallel_task(|| { - with_subtree_scratch(subtree_chunks, |scratch0, scratch1| { + with_subtree_scratch(subtree_chunks, flags, |scratch0, scratch1| { // SAFETY: `out_ptr` is valid for `out_len` outputs and this task's // range is disjoint from every other task. let out = unsafe { slice::from_raw_parts_mut(out_ptr.get().add(start), end - start) }; @@ -662,7 +686,7 @@ fn hash_power_of_two_subtree_roots_parallel_rayon(req: SubtreeRootsRequest<'_>) let (start, end) = thread_range(0, threads_total, out_len); #[cfg(test)] let ok = run_parallel_task(|| { - with_subtree_scratch(subtree_chunks, |scratch0, scratch1| { + with_subtree_scratch(subtree_chunks, flags, |scratch0, scratch1| { let counter = base_counter.wrapping_add((start * subtree_chunks) as u64); // SAFETY: disjoint partition for thread 0. let out = unsafe { slice::from_raw_parts_mut(out_ptr.get().add(start), end - start) }; @@ -681,7 +705,7 @@ fn hash_power_of_two_subtree_roots_parallel_rayon(req: SubtreeRootsRequest<'_>) #[cfg(not(test))] { run_parallel_task(|| { - with_subtree_scratch(subtree_chunks, |scratch0, scratch1| { + with_subtree_scratch(subtree_chunks, flags, |scratch0, scratch1| { let counter = base_counter.wrapping_add((start * subtree_chunks) as u64); // SAFETY: disjoint partition for thread 0. let out = unsafe { slice::from_raw_parts_mut(out_ptr.get().add(start), end - start) }; @@ -825,7 +849,13 @@ fn reduce_power_of_two_chunk_cvs(kernel: Kernel, key_words: [u32; 8], flags: u32 cur_len = pairs; } - cur[0] + let result = cur[0]; + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words_no_fence(cur.as_flattened_mut()); + ct::zeroize_words_no_fence(next.as_flattened_mut()); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } + result } #[cfg(not(target_endian = "little"))] @@ -851,7 +881,13 @@ fn reduce_power_of_two_chunk_cvs_bytes(kernel: Kernel, key_words: [u32; 8], flag cur_len = pairs; } - cur[0] + let result = cur[0]; + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_no_fence(cur.as_flattened_mut()); + ct::zeroize_no_fence(next.as_flattened_mut()); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } + result } /// Reduce a power-of-2 slice of CVs down to exactly 2 using SIMD parent @@ -892,7 +928,11 @@ fn reduce_subtree_to_pair( cur_len = pairs; } - if !in_next { (cvs[0], cvs[1]) } else { (next[0], next[1]) } + let result = if !in_next { (cvs[0], cvs[1]) } else { (next[0], next[1]) }; + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words(next.as_flattened_mut()); + } + result } #[cfg(feature = "parallel")] @@ -928,7 +968,7 @@ fn reduce_power_of_two_chunk_cvs_any( let mut cur = Cur::Input(cvs); let mut cur_len = cvs.len(); - loop { + let result = loop { let pairs = cur_len / 2; debug_assert!(pairs != 0); @@ -982,10 +1022,17 @@ fn reduce_power_of_two_chunk_cvs_any( }; if pairs == 1 { - return out0; + break out0; } cur_len = pairs; + }; + + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words_no_fence(buf0.as_flattened_mut()); + ct::zeroize_words_no_fence(buf1.as_flattened_mut()); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); } + result } #[cfg(target_endian = "little")] @@ -1252,7 +1299,7 @@ fn words8_to_le_bytes(words: &[u32; 8]) -> [u8; OUT_LEN] { out } -#[derive(Clone, Copy)] +#[derive(Clone)] struct OutputState { kernel_id: kernels::Blake3KernelId, input_chaining_value: [u32; 8], @@ -1262,6 +1309,17 @@ struct OutputState { flags: u32, } +impl Drop for OutputState { + fn drop(&mut self) { + if self.flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) == 0 { + return; + } + ct::zeroize_words_no_fence(&mut self.input_chaining_value); + ct::zeroize_words_no_fence(&mut self.block_words); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } +} + impl OutputState { #[inline] fn chaining_value(&self) -> [u32; 8] { @@ -1289,7 +1347,12 @@ impl OutputState { #[inline] fn root_hash_bytes(&self) -> [u8; OUT_LEN] { - words8_to_le_bytes(&self.root_hash_words()) + let mut words = self.root_hash_words(); + let digest = words8_to_le_bytes(&words); + if self.flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words(&mut words); + } + digest } #[inline] @@ -1307,7 +1370,7 @@ impl OutputState { } } -#[derive(Clone, Copy)] +#[derive(Clone)] struct RootEmitState { input_chaining_value: [u32; 8], block_bytes: [u8; BLOCK_LEN], @@ -1317,26 +1380,45 @@ struct RootEmitState { kernel_id: kernels::Blake3KernelId, } +impl Drop for RootEmitState { + fn drop(&mut self) { + if u32::from(self.flags) & (KEYED_HASH | DERIVE_KEY_MATERIAL) == 0 { + return; + } + ct::zeroize_words_no_fence(&mut self.input_chaining_value); + ct::zeroize_no_fence(&mut self.block_bytes); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } +} + impl RootEmitState { #[inline] fn from_parent( kernel_id: kernels::Blake3KernelId, - left_child_cv: [u32; 8], - right_child_cv: [u32; 8], - key_words: [u32; 8], + mut left_child_cv: [u32; 8], + mut right_child_cv: [u32; 8], + mut key_words: [u32; 8], flags: u32, ) -> Self { let mut block_bytes = [0u8; BLOCK_LEN]; block_bytes[..OUT_LEN].copy_from_slice(&words8_to_le_bytes(&left_child_cv)); block_bytes[OUT_LEN..].copy_from_slice(&words8_to_le_bytes(&right_child_cv)); - Self { + let state = Self { kernel_id, input_chaining_value: key_words, block_bytes, counter: 0, block_len: BLOCK_LEN as u8, flags: (PARENT | flags) as u8, + }; + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words_no_fence(&mut left_child_cv); + ct::zeroize_words_no_fence(&mut right_child_cv); + ct::zeroize_words_no_fence(&mut key_words); + ct::zeroize_no_fence(&mut block_bytes); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); } + state } #[inline] @@ -1364,6 +1446,9 @@ impl RootEmitState { let mut block = [0u8; OUTPUT_BLOCK_LEN]; self.emit_one_block(&mut block); out.copy_from_slice(&block[offset..offset + out.len()]); + if u32::from(self.flags) & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize(&mut block); + } } #[inline] @@ -1383,24 +1468,30 @@ impl RootEmitState { #[inline] fn compress_node_chaining_value( kernel_id: kernels::Blake3KernelId, - input_chaining_value: [u32; 8], + mut input_chaining_value: [u32; 8], block: &[u8; BLOCK_LEN], counter: u64, block_len: u32, flags: u32, ) -> [u32; 8] { - let block_words = words16_from_le_bytes_64(block); - first_8_words(kernels::compress_block_inline( + let mut block_words = words16_from_le_bytes_64(block); + let output = first_8_words(kernels::compress_block_inline( kernel_id, &input_chaining_value, &block_words, counter, block_len, flags, - )) + )); + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words_no_fence(&mut input_chaining_value); + ct::zeroize_words_no_fence(&mut block_words); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } + output } -#[derive(Clone, Copy)] +#[derive(Clone)] struct ChunkState { kernel_id: kernels::Blake3KernelId, chaining_value: [u32; 8], @@ -1411,10 +1502,21 @@ struct ChunkState { flags: u32, } +impl Drop for ChunkState { + fn drop(&mut self) { + if self.flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) == 0 { + return; + } + ct::zeroize_words_no_fence(&mut self.chaining_value); + ct::zeroize_no_fence(&mut self.block); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } +} + impl ChunkState { #[inline] - fn new(key_words: [u32; 8], chunk_counter: u64, flags: u32, kernel_id: kernels::Blake3KernelId) -> Self { - Self { + fn new(mut key_words: [u32; 8], chunk_counter: u64, flags: u32, kernel_id: kernels::Blake3KernelId) -> Self { + let state = Self { kernel_id, chaining_value: key_words, chunk_counter, @@ -1422,7 +1524,11 @@ impl ChunkState { block_len: 0, blocks_compressed: 0, flags, + }; + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words(&mut key_words); } + state } #[inline] @@ -1481,7 +1587,8 @@ impl ChunkState { // Exact full-chunk fast path (only pristine state). if self.blocks_compressed == 0 && input.len() == CHUNK_LEN { - if should_use_exact_one_chunk_fast_path(self.kernel_id) { + if self.flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) == 0 && should_use_exact_one_chunk_fast_path(self.kernel_id) + { self.absorb_exact_one_chunk(input); return; } @@ -1775,28 +1882,36 @@ fn should_use_exact_one_chunk_fast_path(_kernel_id: kernels::Blake3KernelId) -> #[inline] fn parent_output( kernel_id: kernels::Blake3KernelId, - left_child_cv: [u32; 8], - right_child_cv: [u32; 8], - key_words: [u32; 8], + mut left_child_cv: [u32; 8], + mut right_child_cv: [u32; 8], + mut key_words: [u32; 8], flags: u32, ) -> OutputState { let mut block_words = [0u32; 16]; block_words[..8].copy_from_slice(&left_child_cv); block_words[8..].copy_from_slice(&right_child_cv); - OutputState { + let output = OutputState { kernel_id, input_chaining_value: key_words, block_words, counter: 0, block_len: BLOCK_LEN as u32, flags: PARENT | flags, + }; + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words_no_fence(&mut left_child_cv); + ct::zeroize_words_no_fence(&mut right_child_cv); + ct::zeroize_words_no_fence(&mut key_words); + ct::zeroize_words_no_fence(&mut block_words); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); } + output } #[inline] fn single_chunk_output( kernel: Kernel, - key_words: [u32; 8], + mut key_words: [u32; 8], chunk_counter: u64, flags: u32, input: &[u8], @@ -1826,8 +1941,8 @@ fn single_chunk_output( ); } - let block_words = words16_from_le_bytes_64(&last_block); - return OutputState { + let mut block_words = words16_from_le_bytes_64(&last_block); + let output = OutputState { kernel_id: kernel.id, input_chaining_value: cv_words, block_words, @@ -1835,6 +1950,14 @@ fn single_chunk_output( block_len: BLOCK_LEN as u32, flags: flags | CHUNK_END, }; + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words_no_fence(&mut cv_words); + ct::zeroize_no_fence(&mut last_block); + ct::zeroize_words_no_fence(&mut block_words); + ct::zeroize_words_no_fence(&mut key_words); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } + return output; } } @@ -1860,7 +1983,7 @@ fn single_chunk_output( &input[..full_bytes], ); - let block_words = if cfg!(target_endian = "little") { + let mut block_words = if cfg!(target_endian = "little") { let mut out = [0u32; 16]; if !input.is_empty() { let offset = full_blocks * BLOCK_LEN; @@ -1876,31 +1999,46 @@ fn single_chunk_output( let offset = full_blocks * BLOCK_LEN; last_block[..last_len].copy_from_slice(&input[offset..offset + last_len]); } - words16_from_le_bytes_64(&last_block) + let block_words = words16_from_le_bytes_64(&last_block); + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize(&mut last_block); + } + block_words }; let start = if blocks_compressed == 0 { CHUNK_START } else { 0 }; - OutputState { + let output = OutputState { kernel_id: kernel.id, input_chaining_value: chaining_value, block_words, counter: chunk_counter, block_len: last_len as u32, flags: flags | start | CHUNK_END, + }; + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words_no_fence(&mut chaining_value); + ct::zeroize_words_no_fence(&mut block_words); + ct::zeroize_words_no_fence(&mut key_words); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); } + output } #[inline] fn root_output_oneshot( kernel: Kernel, - key_words: [u32; 8], + mut key_words: [u32; 8], flags: u32, mode: ParallelPolicyKind, input: &[u8], ) -> OutputState { // Fast path for <= 1 chunk (root is the chunk itself). if input.len() <= CHUNK_LEN { - return single_chunk_output(kernel, key_words, 0, flags, input); + let output = single_chunk_output(kernel, key_words, 0, flags, input); + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words(&mut key_words); + } + return output; } let full_chunks = input.len() / CHUNK_LEN; @@ -1921,7 +2059,11 @@ fn root_output_oneshot( if let Some(threads) = control::parallel_policy_threads_with_admission(mode, input.len(), full_chunks, commit_full_chunks) { - return parallel::root_output_oneshot_join_parallel(kernel, key_words, flags, input, threads); + let output = parallel::root_output_oneshot_join_parallel(kernel, key_words, flags, input, threads); + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words(&mut key_words); + } + return output; } } @@ -1959,10 +2101,18 @@ fn root_output_oneshot( cur_is_cur = !cur_is_cur; cur_len = pairs; } - if cur_is_cur { - return parent_output(kernel.id, cur[0], cur[1], key_words, flags); + let output = if cur_is_cur { + parent_output(kernel.id, cur[0], cur[1], key_words, flags) + } else { + parent_output(kernel.id, next[0], next[1], key_words, flags) + }; + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words_no_fence(cur.as_flattened_mut()); + ct::zeroize_words_no_fence(next.as_flattened_mut()); + ct::zeroize_words_no_fence(&mut key_words); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); } - return parent_output(kernel.id, next[0], next[1], key_words, flags); + return output; } #[cfg(not(target_endian = "little"))] @@ -2008,9 +2158,20 @@ fn root_output_oneshot( cur_len = pairs; } let final_cvs: &[[u8; OUT_LEN]] = if cur_is_cur { &cur[..] } else { &next[..] }; - let left = words8_from_le_bytes_32(&final_cvs[0]); - let right = words8_from_le_bytes_32(&final_cvs[1]); - return parent_output(kernel.id, left, right, key_words, flags); + let output = parent_output( + kernel.id, + words8_from_le_bytes_32(&final_cvs[0]), + words8_from_le_bytes_32(&final_cvs[1]), + key_words, + flags, + ); + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_no_fence(cur.as_flattened_mut()); + ct::zeroize_no_fence(next.as_flattened_mut()); + ct::zeroize_words_no_fence(&mut key_words); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } + return output; } } @@ -2045,10 +2206,18 @@ fn root_output_oneshot( cur_is_cur = !cur_is_cur; cur_len = pairs; } - if cur_is_cur { - return parent_output(kernel.id, cur[0], cur[1], key_words, flags); + let output = if cur_is_cur { + parent_output(kernel.id, cur[0], cur[1], key_words, flags) + } else { + parent_output(kernel.id, next[0], next[1], key_words, flags) + }; + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words_no_fence(cur.as_flattened_mut()); + ct::zeroize_words_no_fence(next.as_flattened_mut()); + ct::zeroize_words_no_fence(&mut key_words); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); } - return parent_output(kernel.id, next[0], next[1], key_words, flags); + return output; } #[cfg(not(target_endian = "little"))] @@ -2094,9 +2263,20 @@ fn root_output_oneshot( cur_len = pairs; } let final_cvs: &[[u8; OUT_LEN]] = if cur_is_cur { &cur[..] } else { &next[..] }; - let left = words8_from_le_bytes_32(&final_cvs[0]); - let right = words8_from_le_bytes_32(&final_cvs[1]); - return parent_output(kernel.id, left, right, key_words, flags); + let output = parent_output( + kernel.id, + words8_from_le_bytes_32(&final_cvs[0]), + words8_from_le_bytes_32(&final_cvs[1]), + key_words, + flags, + ); + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_no_fence(cur.as_flattened_mut()); + ct::zeroize_no_fence(next.as_flattened_mut()); + ct::zeroize_words_no_fence(&mut key_words); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } + return output; } } } @@ -2104,7 +2284,7 @@ fn root_output_oneshot( // Local CV stack to avoid constructing a full streaming hasher. let mut cv_stack: [MaybeUninit<[u32; 8]>; CV_STACK_LEN] = uninit_cv_stack(); let mut cv_stack_len = 0usize; - let right_cv = { + let mut right_cv = { #[cfg(target_endian = "little")] { let mut cvs = [[0u32; 8]; MAX_SIMD_DEGREE]; @@ -2152,12 +2332,23 @@ fn root_output_oneshot( offset = offset.strict_add(batch.strict_mul(CHUNK_LEN)); } - if remainder != 0 { + let right_cv = if remainder != 0 { let chunk_bytes = &input[full_chunks * CHUNK_LEN..]; single_chunk_output(kernel, key_words, full_chunks as u64, flags, chunk_bytes).chaining_value() } else { - last_full_chunk_cv.unwrap_or_else(|| recompute_last_full_chunk_cv(kernel, key_words, flags, full_chunks, input)) + last_full_chunk_cv + .as_ref() + .copied() + .unwrap_or_else(|| recompute_last_full_chunk_cv(kernel, key_words, flags, full_chunks, input)) + }; + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + if let Some(cv) = last_full_chunk_cv.as_mut() { + ct::zeroize_words_no_fence(cv); + } + ct::zeroize_words_no_fence(cvs.as_flattened_mut()); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); } + right_cv } #[cfg(not(target_endian = "little"))] @@ -2206,21 +2397,22 @@ fn root_output_oneshot( offset = offset.strict_add(batch.strict_mul(CHUNK_LEN)); } - if remainder != 0 { + let right_cv = if remainder != 0 { let chunk_bytes = &input[full_chunks * CHUNK_LEN..]; single_chunk_output(kernel, key_words, full_chunks as u64, flags, chunk_bytes).chaining_value() + } else if let Some(cv) = last_full_chunk_cv.as_ref() { + words8_from_le_bytes_32(cv) } else { - let last_full_chunk_cv = last_full_chunk_cv.unwrap_or_else(|| { - words8_to_le_bytes(&recompute_last_full_chunk_cv( - kernel, - key_words, - flags, - full_chunks, - input, - )) - }); - words8_from_le_bytes_32(&last_full_chunk_cv) + recompute_last_full_chunk_cv(kernel, key_words, flags, full_chunks, input) + }; + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + if let Some(cv) = last_full_chunk_cv.as_mut() { + ct::zeroize_no_fence(cv); + } + ct::zeroize_no_fence(cvs.as_flattened_mut()); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); } + right_cv } }; @@ -2228,13 +2420,30 @@ fn root_output_oneshot( debug_assert!(parent_nodes_remaining > 0); parent_nodes_remaining -= 1; // SAFETY: `cv_stack_len` tracks the number of initialized entries. - let left = unsafe { cv_stack[parent_nodes_remaining].assume_init_read() }; + let mut left = unsafe { cv_stack[parent_nodes_remaining].assume_init_read() }; let mut output = parent_output(kernel.id, left, right_cv, key_words, flags); + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words_no_fence(&mut left); + ct::zeroize_words_no_fence(&mut right_cv); + } while parent_nodes_remaining > 0 { parent_nodes_remaining -= 1; // SAFETY: `cv_stack_len` tracks the number of initialized entries. - let left = unsafe { cv_stack[parent_nodes_remaining].assume_init_read() }; - output = parent_output(kernel.id, left, output.chaining_value(), key_words, flags); + let mut left = unsafe { cv_stack[parent_nodes_remaining].assume_init_read() }; + let mut right = output.chaining_value(); + output = parent_output(kernel.id, left, right, key_words, flags); + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words_no_fence(&mut left); + ct::zeroize_words_no_fence(&mut right); + } + } + + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + for slot in &mut cv_stack { + ct::zeroize_words_no_fence(slot.write([0u32; 8])); + } + ct::zeroize_words_no_fence(&mut key_words); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); } output @@ -2243,7 +2452,7 @@ fn root_output_oneshot( #[inline] fn recompute_last_full_chunk_cv( kernel: Kernel, - key_words: [u32; 8], + mut key_words: [u32; 8], flags: u32, full_chunks: usize, input: &[u8], @@ -2253,14 +2462,18 @@ fn recompute_last_full_chunk_cv( let last_chunk_index = full_chunks.saturating_sub(1); let offset = input.len().saturating_sub(CHUNK_LEN); - single_chunk_output(kernel, key_words, last_chunk_index as u64, flags, &input[offset..]).chaining_value() + let cv = single_chunk_output(kernel, key_words, last_chunk_index as u64, flags, &input[offset..]).chaining_value(); + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words(&mut key_words); + } + cv } #[cfg(feature = "parallel")] #[inline] fn hash_one_full_chunk_cv( kernel: Kernel, - key_words: [u32; 8], + mut key_words: [u32; 8], flags: u32, chunk_counter: u64, chunk: &[u8], @@ -2270,7 +2483,13 @@ fn hash_one_full_chunk_cv( let mut out = [0u8; OUT_LEN]; // SAFETY: `chunk` is exactly one full chunk, and `out` is OUT_LEN bytes. unsafe { (kernel.hash_many_contiguous)(chunk.as_ptr(), 1, &key_words, chunk_counter, flags, out.as_mut_ptr()) }; - words8_from_le_bytes_32(&out) + let cv = words8_from_le_bytes_32(&out); + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_no_fence(&mut out); + ct::zeroize_words_no_fence(&mut key_words); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } + cv } #[cfg(feature = "parallel")] @@ -2306,7 +2525,7 @@ fn hash_full_chunks_cvs_scoped( } #[inline] -fn digest_oneshot_words(kernel: Kernel, key_words: [u32; 8], flags: u32, input: &[u8]) -> [u32; 8] { +fn digest_oneshot_words(kernel: Kernel, mut key_words: [u32; 8], flags: u32, input: &[u8]) -> [u32; 8] { // Fast path for single-chunk inputs (≤1024B): use platform-specific helpers. // On x86, this also handles tiny inputs (≤64B) so they benefit from assembly // compress instead of the generic intrinsics/function-pointer path. @@ -2316,7 +2535,11 @@ fn digest_oneshot_words(kernel: Kernel, key_words: [u32; 8], flags: u32, input: match kernel.id { kernels::Blake3KernelId::X86Sse41 | kernels::Blake3KernelId::X86Avx2 | kernels::Blake3KernelId::X86Avx512 => { // SAFETY: x86 SIMD availability is validated by dispatch before selecting these kernels. - return unsafe { digest_one_chunk_root_hash_words_x86(kernel, key_words, flags, input) }; + let output = unsafe { digest_one_chunk_root_hash_words_x86(kernel, key_words, flags, input) }; + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words(&mut key_words); + } + return output; } _ => {} } @@ -2325,59 +2548,139 @@ fn digest_oneshot_words(kernel: Kernel, key_words: [u32; 8], flags: u32, input: // Tiny inputs (≤64B) on non-x86 or portable kernel: unified helper. if input.len() <= BLOCK_LEN { - return hash_tiny_to_root_words(kernel, key_words, flags, input); + let output = hash_tiny_to_root_words(kernel, key_words, flags, input); + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words(&mut key_words); + } + return output; } #[cfg(target_arch = "aarch64")] { if input.len() <= CHUNK_LEN && kernel.id == kernels::Blake3KernelId::Aarch64Neon { // SAFETY: aarch64 NEON availability is validated by dispatch before selecting this kernel. - return unsafe { digest_one_chunk_root_hash_words_aarch64(kernel, key_words, flags, input) }; + let output = unsafe { digest_one_chunk_root_hash_words_aarch64(kernel, key_words, flags, input) }; + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words(&mut key_words); + } + return output; } } if input.len() <= CHUNK_LEN { - return digest_one_chunk_root_hash_words_generic(kernel, key_words, flags, input); + let output = digest_one_chunk_root_hash_words_generic(kernel, key_words, flags, input); + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words(&mut key_words); + } + return output; } // Fallback: keep the large-input path in a cold function to avoid // inflating short-input codegen in this hot entry point. - digest_oneshot_words_fallback(kernel, key_words, flags, input) + let output = digest_oneshot_words_fallback(kernel, key_words, flags, input); + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words(&mut key_words); + } + output } #[cold] #[inline(never)] -fn digest_oneshot_words_fallback(kernel: Kernel, key_words: [u32; 8], flags: u32, input: &[u8]) -> [u32; 8] { +fn digest_oneshot_words_fallback(kernel: Kernel, mut key_words: [u32; 8], flags: u32, input: &[u8]) -> [u32; 8] { let mode = control::policy_kind_from_flags(flags, false); let output = root_output_oneshot(kernel, key_words, flags, mode, input); - output.root_hash_words() + let digest = output.root_hash_words(); + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words(&mut key_words); + } + digest } #[inline] -fn digest_oneshot(kernel: Kernel, key_words: [u32; 8], flags: u32, input: &[u8]) -> [u8; OUT_LEN] { +fn digest_oneshot(kernel: Kernel, mut key_words: [u32; 8], flags: u32, input: &[u8]) -> [u8; OUT_LEN] { #[cfg(target_arch = "aarch64")] { - if kernel.id == kernels::Blake3KernelId::Aarch64Neon && input.len() == CHUNK_LEN { + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) == 0 + && kernel.id == kernels::Blake3KernelId::Aarch64Neon + && input.len() == CHUNK_LEN + { // SAFETY: aarch64 NEON is validated by dispatch before selecting this kernel. return unsafe { aarch64::root_hash_one_chunk_root_aarch64(input.as_ptr(), &key_words, flags) }; } } - words8_to_le_bytes(&digest_oneshot_words(kernel, key_words, flags, input)) + let mut words = digest_oneshot_words(kernel, key_words, flags, input); + let digest = words8_to_le_bytes(&words); + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words_no_fence(&mut words); + ct::zeroize_words_no_fence(&mut key_words); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } + digest } #[inline] -fn digest_public_oneshot(key_words: [u32; 8], flags: u32, input: &[u8]) -> [u8; OUT_LEN] { +fn digest_public_oneshot(mut key_words: [u32; 8], flags: u32, input: &[u8]) -> [u8; OUT_LEN] { let kernel = dispatch::hasher_dispatch().size_class_kernel(input.len()); - digest_oneshot(kernel, key_words, flags, input) + let digest = digest_oneshot(kernel, key_words, flags, input); + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words(&mut key_words); + } + digest } #[cfg(feature = "diag")] #[must_use] pub fn diag_blake3_keyed_digest_portable(key: &[u8; KEY_LEN]) -> Blake3KeyedHash { - let key_words = words8_from_le_bytes_32(key); + let mut key_words = words8_from_le_bytes_32(key); let kernel = kernels::kernel(kernels::Blake3KernelId::Portable); - Blake3KeyedHash::from_bytes(digest_oneshot(kernel, key_words, KEYED_HASH, b"binsec")) + let digest = Blake3KeyedHash::from_bytes(digest_oneshot(kernel, key_words, KEYED_HASH, b"binsec")); + ct::zeroize_words(&mut key_words); + digest +} + +#[cfg(feature = "diag")] +#[doc(hidden)] +#[unsafe(no_mangle)] +#[inline(never)] +pub fn diag_zeroize_blake3_drop(mut key: [u8; KEY_LEN]) -> u8 { + let mut state = Blake3::new_keyed(&key); + ct::zeroize(&mut key); + state.update(b"zeroize-drop"); + core::hint::black_box(state.finalize()[0]) +} + +#[cfg(feature = "diag")] +#[doc(hidden)] +#[unsafe(no_mangle)] +#[inline(never)] +pub fn diag_zeroize_blake3_reuse(mut key: [u8; KEY_LEN]) -> u8 { + let mut state = Blake3::new_keyed(&key); + ct::zeroize(&mut key); + state.update(b"discarded keyed prefix"); + state.reset(); + state.update(b"reused keyed state"); + core::hint::black_box(state.finalize()[0]) +} + +#[cfg(feature = "diag")] +#[doc(hidden)] +#[unsafe(no_mangle)] +#[inline(never)] +pub fn diag_zeroize_blake3_xof_move(mut key: [u8; KEY_LEN]) -> u8 { + let reader = Blake3::keyed_xof(&key, b"moved keyed XOF"); + ct::zeroize(&mut key); + diag_zeroize_blake3_xof_consume(reader) +} + +#[cfg(feature = "diag")] +#[doc(hidden)] +#[unsafe(no_mangle)] +#[inline(never)] +pub fn diag_zeroize_blake3_xof_consume(mut reader: Blake3XofReader) -> u8 { + let mut output = [0u8; 1]; + reader.squeeze(&mut output); + core::hint::black_box(output[0]) } #[cfg(feature = "diag")] @@ -2583,10 +2886,10 @@ pub fn diag_blake3_keyed_digest_with_kernel( data: &[u8], ) -> Option { let kernel = diag_blake3_kernel(kernel)?; - let key_words = words8_from_le_bytes_32(key); - Some(Blake3KeyedHash::from_bytes(digest_oneshot( - kernel, key_words, KEYED_HASH, data, - ))) + let mut key_words = words8_from_le_bytes_32(key); + let digest = Blake3KeyedHash::from_bytes(digest_oneshot(kernel, key_words, KEYED_HASH, data)); + ct::zeroize_words(&mut key_words); + Some(digest) } #[cfg(feature = "diag")] @@ -2766,8 +3069,10 @@ impl Blake3 { #[inline] #[must_use] pub fn keyed_digest(key: &[u8; KEY_LEN], data: &[u8]) -> Blake3KeyedHash { - let key_words = words8_from_le_bytes_32(key); - Blake3KeyedHash::from_bytes(digest_public_oneshot(key_words, KEYED_HASH, data)) + let mut key_words = words8_from_le_bytes_32(key); + let digest = Blake3KeyedHash::from_bytes(digest_public_oneshot(key_words, KEYED_HASH, data)); + ct::zeroize_words(&mut key_words); + digest } /// Compute and verify the keyed hash through its sealed comparison decision. @@ -2789,19 +3094,22 @@ impl Blake3 { #[inline] #[must_use] pub fn keyed_xof(key: &[u8; KEY_LEN], data: &[u8]) -> Blake3XofReader { - let key_words = words8_from_le_bytes_32(key); + let mut key_words = words8_from_le_bytes_32(key); let plan = dispatch::hasher_dispatch(); let kernel = plan.size_class_kernel(data.len()); - if data.len() <= CHUNK_LEN { - return xof_oneshot_single_chunk(kernel, key_words, KEYED_HASH, data); - } - Blake3XofReader::from_output(root_output_oneshot( - kernel, - key_words, - KEYED_HASH, - control::policy_kind_from_flags(KEYED_HASH, true), - data, - )) + let reader = if data.len() <= CHUNK_LEN { + xof_oneshot_single_chunk(kernel, key_words, KEYED_HASH, data) + } else { + Blake3XofReader::from_output(root_output_oneshot( + kernel, + key_words, + KEYED_HASH, + control::policy_kind_from_flags(KEYED_HASH, true), + data, + )) + }; + ct::zeroize_words(&mut key_words); + reader } /// Compute the derived key for `key_material` under `context`, in one shot. @@ -2823,23 +3131,30 @@ impl Blake3 { } }; - digest_public_oneshot(context_key_words, DERIVE_KEY_MATERIAL, key_material) + let mut context_key_words = context_key_words; + let derived = digest_public_oneshot(context_key_words, DERIVE_KEY_MATERIAL, key_material); + ct::zeroize_words(&mut context_key_words); + derived } #[inline] - fn new_internal(key_words: [u32; 8], flags: u32) -> Self { + fn new_internal(mut key_words: [u32; 8], flags: u32) -> Self { let dispatch = dispatch::hasher_dispatch(); - Self::new_internal_with_dispatch(key_words, flags, dispatch.stream_kernel().id, dispatch) + let state = Self::new_internal_with_dispatch(key_words, flags, dispatch.stream_kernel().id, dispatch); + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words(&mut key_words); + } + state } #[inline] fn new_internal_with_dispatch( - key_words: [u32; 8], + mut key_words: [u32; 8], flags: u32, kernel_id: kernels::Blake3KernelId, dispatch: dispatch::HasherDispatch, ) -> Self { - Self { + let state = Self { dispatch, bulk_kernel_id: kernel_id, #[cfg(feature = "parallel")] @@ -2850,7 +3165,11 @@ impl Blake3 { key_words, cv_stack: uninit_cv_stack(), cv_stack_len: 0, + }; + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words(&mut key_words); } + state } /// Process the largest aligned power-of-2 subtree from the input using SIMD @@ -2927,7 +3246,7 @@ impl Blake3 { // Reduce to 2 CVs so the final parent compression can be deferred to // finalize() for ROOT flag application. let half = (subtree_chunks / 2) as u64; - let (left_cv, right_cv) = reduce_subtree_to_pair(kernel, self.key_words, flags, cvs); + let (mut left_cv, mut right_cv) = reduce_subtree_to_pair(kernel, self.key_words, flags, cvs); // Push the left subtree onto the stack. self.add_subtree_cv(left_cv, half); @@ -2937,11 +3256,26 @@ impl Blake3 { self.chunk_state = ChunkState::new(self.key_words, full_counter, flags, kernel_id); self.pending_chunk_cv = Some(right_cv); self.pending_cv_chunks = half; + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words_no_fence(&mut left_cv); + ct::zeroize_words_no_fence(&mut right_cv); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } } else { // Non-terminal: more input follows. Reduce to 1 CV and push. - let (left_cv, right_cv) = reduce_subtree_to_pair(kernel, self.key_words, flags, cvs); - let subtree_cv = kernels::parent_cv_inline(kernel.id, left_cv, right_cv, self.key_words, flags); + let (mut left_cv, mut right_cv) = reduce_subtree_to_pair(kernel, self.key_words, flags, cvs); + let mut subtree_cv = kernels::parent_cv_inline(kernel.id, left_cv, right_cv, self.key_words, flags); self.add_subtree_cv(subtree_cv, subtree_chunks as u64); + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words_no_fence(&mut left_cv); + ct::zeroize_words_no_fence(&mut right_cv); + ct::zeroize_words_no_fence(&mut subtree_cv); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } + } + + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words(cvs.as_flattened_mut()); } Some(subtree_len) @@ -3012,9 +3346,16 @@ impl Blake3 { let offset = batch.strict_sub(1).strict_mul(OUT_LEN); // SAFETY: `out_buf` is `OUT_LEN * MAX_SIMD_DEGREE`, and `offset` // is `(batch - 1) * OUT_LEN` with `batch <= MAX_SIMD_DEGREE`. - let cv = unsafe { words8_from_le_bytes_32(&*(out_buf.as_ptr().add(offset) as *const [u8; OUT_LEN])) }; + let mut cv = unsafe { words8_from_le_bytes_32(&*(out_buf.as_ptr().add(offset) as *const [u8; OUT_LEN])) }; self.pending_chunk_cv = Some(cv); self.pending_cv_chunks = 1; + if self.chunk_state.flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words(&mut cv); + } + } + + if self.chunk_state.flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize(&mut out_buf); } Some(batch.strict_mul(CHUNK_LEN)) @@ -3084,8 +3425,10 @@ impl Blake3 { #[must_use] #[inline] pub fn new_keyed(key: &[u8; KEY_LEN]) -> Self { - let key_words = words8_from_le_bytes_32(key); - Self::new_internal(key_words, KEYED_HASH) + let mut key_words = words8_from_le_bytes_32(key); + let state = Self::new_internal(key_words, KEYED_HASH); + ct::zeroize_words(&mut key_words); + state } /// Construct a new hasher for the key derivation function. @@ -3096,34 +3439,59 @@ impl Blake3 { let key_words = control::derive_context_key_words_cached(context); #[cfg(not(feature = "std"))] let key_words = control::derive_context_key_words(context); - Self::new_internal(key_words, DERIVE_KEY_MATERIAL) + let mut key_words = key_words; + let state = Self::new_internal(key_words, DERIVE_KEY_MATERIAL); + ct::zeroize_words(&mut key_words); + state } #[inline] - fn push_stack(&mut self, cv: [u32; 8]) { + fn push_stack(&mut self, mut cv: [u32; 8]) { self.cv_stack[self.cv_stack_len as usize].write(cv); self.cv_stack_len = self.cv_stack_len.wrapping_add(1); + if self.chunk_state.flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words(&mut cv); + } } #[inline] fn pop_stack(&mut self) -> [u32; 8] { self.cv_stack_len = self.cv_stack_len.wrapping_sub(1); // SAFETY: `cv_stack_len` tracks the number of initialized entries. - unsafe { self.cv_stack[self.cv_stack_len as usize].assume_init_read() } + let cv = unsafe { self.cv_stack[self.cv_stack_len as usize].assume_init_read() }; + if self.chunk_state.flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + // SAFETY: the slot held the `Copy` CV read above and remains initialized + // until the next push overwrites it. + ct::zeroize_words(unsafe { self.cv_stack[self.cv_stack_len as usize].assume_init_mut() }); + } + cv } fn add_chunk_chaining_value(&mut self, mut new_cv: [u32; 8], mut total_chunks: u64) { while total_chunks & 1 == 0 { - new_cv = kernels::parent_cv_inline( + let mut left = self.pop_stack(); + let mut next_cv = kernels::parent_cv_inline( self.chunk_state.kernel_id, - self.pop_stack(), + left, new_cv, self.key_words, self.chunk_state.flags, ); + if self.chunk_state.flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words_no_fence(&mut left); + ct::zeroize_words_no_fence(&mut new_cv); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } + core::mem::swap(&mut new_cv, &mut next_cv); + if self.chunk_state.flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words(&mut next_cv); + } total_chunks >>= 1; } self.push_stack(new_cv); + if self.chunk_state.flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words(&mut new_cv); + } } /// Merge a CV that represents `num_chunks` chunks into the CV stack. @@ -3140,16 +3508,24 @@ impl Blake3 { let level = num_chunks.trailing_zeros(); let mut total = self.chunk_state.chunk_counter >> level; while total & 1 == 0 { - cv = kernels::parent_cv_inline( - self.bulk_kernel_id, - self.pop_stack(), - cv, - self.key_words, - self.chunk_state.flags, - ); + let mut left = self.pop_stack(); + let mut next_cv = + kernels::parent_cv_inline(self.bulk_kernel_id, left, cv, self.key_words, self.chunk_state.flags); + if self.chunk_state.flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words_no_fence(&mut left); + ct::zeroize_words_no_fence(&mut cv); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } + core::mem::swap(&mut cv, &mut next_cv); + if self.chunk_state.flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words(&mut next_cv); + } total >>= 1; } self.push_stack(cv); + if self.chunk_state.flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words(&mut cv); + } } /// Push a reduced subtree CV onto the stack at the correct tree level, @@ -3158,7 +3534,7 @@ impl Blake3 { /// `subtree_chunks` must be a power of 2 and must be aligned with the /// current chunk counter (i.e. `chunk_counter % subtree_chunks == 0`). #[cfg(target_endian = "little")] - fn add_subtree_cv(&mut self, cv: [u32; 8], subtree_chunks: u64) { + fn add_subtree_cv(&mut self, mut cv: [u32; 8], subtree_chunks: u64) { debug_assert!(subtree_chunks.is_power_of_two()); let new_counter = self.chunk_state.chunk_counter.strict_add(subtree_chunks); self.chunk_state = ChunkState::new( @@ -3168,23 +3544,41 @@ impl Blake3 { self.chunk_state.kernel_id, ); self.merge_cv_into_stack(cv, subtree_chunks); + if self.chunk_state.flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words(&mut cv); + } } #[inline] fn commit_pending_chunk_cv(&mut self) { - if let Some(cv) = self.pending_chunk_cv.take() { - let chunks = self.pending_cv_chunks; - self.pending_cv_chunks = 0; - self.merge_cv_into_stack(cv, chunks); + let Some(mut cv) = self.pending_chunk_cv else { + return; + }; + if self.chunk_state.flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + // Clear the source payload before changing the discriminant. Assigning + // `None` alone is not required to overwrite an `Option`'s old payload. + if let Some(source) = self.pending_chunk_cv.as_mut() { + ct::zeroize_words(source); + } + } + self.pending_chunk_cv = None; + let chunks = self.pending_cv_chunks; + self.pending_cv_chunks = 0; + self.merge_cv_into_stack(cv, chunks); + if self.chunk_state.flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words(&mut cv); } } #[inline] fn advance_full_chunk(&mut self) { debug_assert_eq!(self.chunk_state.len(), CHUNK_LEN); - let chunk_cv = self.chunk_state.output().chaining_value(); + let mut chunk_cv = self.chunk_state.output().chaining_value(); let total_chunks = self.chunk_state.chunk_counter.strict_add(1); self.add_chunk_chaining_value(chunk_cv, total_chunks); + if self.chunk_state.flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words(&mut chunk_cv); + } self.chunk_state = ChunkState::new( self.key_words, total_chunks, @@ -3213,20 +3607,26 @@ impl Blake3 { fn root_output(&self) -> OutputState { let mut parent_nodes_remaining = self.cv_stack_len as usize; - let mut output = if let Some(right_cv) = self.pending_chunk_cv { + let mut output = if let Some(mut right_cv) = self.pending_chunk_cv { let Some(left_index) = parent_nodes_remaining.checked_sub(1) else { return self.chunk_state.output(); }; parent_nodes_remaining = left_index; // SAFETY: `cv_stack_len` tracks the number of initialized entries. - let left = unsafe { *self.cv_stack[left_index].assume_init_ref() }; - parent_output( + let mut left = unsafe { *self.cv_stack[left_index].assume_init_ref() }; + let output = parent_output( self.chunk_state.kernel_id, left, right_cv, self.key_words, self.chunk_state.flags, - ) + ); + if self.chunk_state.flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words_no_fence(&mut left); + ct::zeroize_words_no_fence(&mut right_cv); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } + output } else { self.chunk_state.output() }; @@ -3234,14 +3634,20 @@ impl Blake3 { while parent_nodes_remaining > 0 { parent_nodes_remaining -= 1; // SAFETY: `cv_stack_len` tracks the number of initialized entries. - let left = unsafe { *self.cv_stack[parent_nodes_remaining].assume_init_ref() }; + let mut left = unsafe { *self.cv_stack[parent_nodes_remaining].assume_init_ref() }; + let mut right = output.chaining_value(); output = parent_output( self.chunk_state.kernel_id, left, - output.chaining_value(), + right, self.key_words, self.chunk_state.flags, ); + if self.chunk_state.flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words_no_fence(&mut left); + ct::zeroize_words_no_fence(&mut right); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } } output } @@ -3258,29 +3664,41 @@ impl Blake3 { #[inline(never)] fn root_emit_state_slow(&self) -> RootEmitState { let mut parent_nodes_remaining = self.cv_stack_len as usize; - let mut current_cv = if let Some(right_cv) = self.pending_chunk_cv { + let mut current_cv = if let Some(mut right_cv) = self.pending_chunk_cv { let Some(left_index) = parent_nodes_remaining.checked_sub(1) else { return self.chunk_state.root_emit_state(); }; parent_nodes_remaining = left_index; // SAFETY: `cv_stack_len` tracks the number of initialized entries. - let left = unsafe { *self.cv_stack[left_index].assume_init_ref() }; + let mut left = unsafe { *self.cv_stack[left_index].assume_init_ref() }; if parent_nodes_remaining == 0 { - return RootEmitState::from_parent( + let state = RootEmitState::from_parent( self.chunk_state.kernel_id, left, right_cv, self.key_words, self.chunk_state.flags, ); + if self.chunk_state.flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words_no_fence(&mut left); + ct::zeroize_words_no_fence(&mut right_cv); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } + return state; } - kernels::parent_cv_inline( + let cv = kernels::parent_cv_inline( self.chunk_state.kernel_id, left, right_cv, self.key_words, self.chunk_state.flags, - ) + ); + if self.chunk_state.flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words_no_fence(&mut left); + ct::zeroize_words_no_fence(&mut right_cv); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } + cv } else { self.chunk_state.output_chaining_value() }; @@ -3288,25 +3706,43 @@ impl Blake3 { while parent_nodes_remaining > 0 { parent_nodes_remaining -= 1; // SAFETY: `cv_stack_len` tracks the number of initialized entries. - let left = unsafe { *self.cv_stack[parent_nodes_remaining].assume_init_ref() }; + let mut left = unsafe { *self.cv_stack[parent_nodes_remaining].assume_init_ref() }; if parent_nodes_remaining == 0 { - return RootEmitState::from_parent( + let state = RootEmitState::from_parent( self.chunk_state.kernel_id, left, current_cv, self.key_words, self.chunk_state.flags, ); + if self.chunk_state.flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words_no_fence(&mut left); + ct::zeroize_words_no_fence(&mut current_cv); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } + return state; } - current_cv = kernels::parent_cv_inline( + let mut next_cv = kernels::parent_cv_inline( self.chunk_state.kernel_id, left, current_cv, self.key_words, self.chunk_state.flags, ); + if self.chunk_state.flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words_no_fence(&mut left); + ct::zeroize_words_no_fence(&mut current_cv); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } + core::mem::swap(&mut current_cv, &mut next_cv); + if self.chunk_state.flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words(&mut next_cv); + } } + if self.chunk_state.flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words(&mut current_cv); + } self.chunk_state.root_emit_state() } @@ -3328,11 +3764,6 @@ impl Drop for Blake3 { // SAFETY: word is a valid, aligned, dereferenceable pointer to initialized memory. unsafe { core::ptr::write_volatile(word, 0) }; } - for word in self.chunk_state.chaining_value.iter_mut() { - // SAFETY: word is a valid, aligned, dereferenceable pointer to initialized memory. - unsafe { core::ptr::write_volatile(word, 0) }; - } - crate::traits::ct::zeroize(&mut self.chunk_state.block); for slot in self.cv_stack[..self.cv_stack_len as usize].iter_mut() { // SAFETY: cv_stack[..cv_stack_len] entries are initialized. let words = unsafe { slot.assume_init_mut() }; @@ -3402,11 +3833,11 @@ impl Digest for Blake3 { let block_len = self.chunk_state.block_len as usize; let add_chunk_start = self.chunk_state.blocks_compressed == 0; - let cv = self.chunk_state.chaining_value; + let mut cv = self.chunk_state.chaining_value; let kernel = kernels::kernel(self.chunk_state.kernel_id); if block_len == BLOCK_LEN { - let out_words = compress_chunk_tail_to_root_words( + let mut out_words = compress_chunk_tail_to_root_words( kernel, cv, &self.chunk_state.block, @@ -3414,14 +3845,27 @@ impl Digest for Blake3 { self.chunk_state.flags, add_chunk_start, ); - return words8_to_le_bytes(&out_words); + let digest = words8_to_le_bytes(&out_words); + if self.chunk_state.flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words_no_fence(&mut out_words); + ct::zeroize_words_no_fence(&mut cv); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } + return digest; } let mut block = self.chunk_state.block; block[block_len..].fill(0); - let out_words = + let mut out_words = compress_chunk_tail_to_root_words(kernel, cv, &block, block_len, self.chunk_state.flags, add_chunk_start); - return words8_to_le_bytes(&out_words); + let digest = words8_to_le_bytes(&out_words); + if self.chunk_state.flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_no_fence(&mut block); + ct::zeroize_words_no_fence(&mut out_words); + ct::zeroize_words_no_fence(&mut cv); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } + return digest; } // If the caller never provided any input (or only provided empty updates), @@ -3460,12 +3904,12 @@ impl_std_io_write_for_digest!(Blake3); /// For 65–1024B, full blocks are compressed directly via the kernel's compress /// function pointer, avoiding the indirect `chunk_compress_blocks` call. #[inline] -fn xof_oneshot_single_chunk(kernel: Kernel, key_words: [u32; 8], flags: u32, input: &[u8]) -> Blake3XofReader { +fn xof_oneshot_single_chunk(kernel: Kernel, mut key_words: [u32; 8], flags: u32, input: &[u8]) -> Blake3XofReader { debug_assert!(input.len() <= CHUNK_LEN); // Tiny input (≤ 64B): zero-pad into the final block bytes, no compression needed. if input.len() <= BLOCK_LEN { - let block_bytes = { + let mut block_bytes = { let mut out = [0u8; BLOCK_LEN]; if !input.is_empty() { out[..input.len()].copy_from_slice(input); @@ -3473,7 +3917,7 @@ fn xof_oneshot_single_chunk(kernel: Kernel, key_words: [u32; 8], flags: u32, inp out }; - return Blake3XofReader::new(RootEmitState { + let reader = Blake3XofReader::new(RootEmitState { kernel_id: kernel.id, input_chaining_value: key_words, block_bytes, @@ -3481,6 +3925,12 @@ fn xof_oneshot_single_chunk(kernel: Kernel, key_words: [u32; 8], flags: u32, inp block_len: input.len() as u8, flags: (flags | CHUNK_START | CHUNK_END) as u8, }); + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_no_fence(&mut block_bytes); + ct::zeroize_words_no_fence(&mut key_words); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } + return reader; } #[cfg(target_arch = "x86_64")] @@ -3490,6 +3940,9 @@ fn xof_oneshot_single_chunk(kernel: Kernel, key_words: [u32; 8], flags: u32, inp // SAFETY: the helper revalidates the x86 kernel allowlist, exact-block // preconditions, and OS support before touching any target-specific path. if let Some(reader) = unsafe { xof_oneshot_single_chunk_x86_exact_blocks(kernel, key_words, flags, input) } { + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words(&mut key_words); + } return reader; } } @@ -3511,8 +3964,11 @@ fn xof_oneshot_single_chunk(kernel: Kernel, key_words: [u32; 8], flags: u32, inp let offset = i * BLOCK_LEN; let block_flags = flags | if i == 0 { CHUNK_START } else { 0 }; // SAFETY: `offset + BLOCK_LEN <= input.len()` by construction. - let block_words = unsafe { words16_from_le_bytes_64(&*input.as_ptr().add(offset).cast::<[u8; BLOCK_LEN]>()) }; + let mut block_words = unsafe { words16_from_le_bytes_64(&*input.as_ptr().add(offset).cast::<[u8; BLOCK_LEN]>()) }; cv = first_8_words((kernel.compress)(&cv, &block_words, 0, BLOCK_LEN as u32, block_flags)); + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words(&mut block_words); + } } // Stage the final block bytes for the root-output reader. @@ -3521,14 +3977,21 @@ fn xof_oneshot_single_chunk(kernel: Kernel, key_words: [u32; 8], flags: u32, inp let offset = full_blocks * BLOCK_LEN; block_bytes[..last_len].copy_from_slice(&input[offset..offset + last_len]); - Blake3XofReader::new(RootEmitState { + let reader = Blake3XofReader::new(RootEmitState { kernel_id: kernel.id, input_chaining_value: cv, block_bytes, counter: 0, block_len: last_len as u8, flags: (flags | start | CHUNK_END) as u8, - }) + }); + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words_no_fence(&mut cv); + ct::zeroize_no_fence(&mut block_bytes); + ct::zeroize_words_no_fence(&mut key_words); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } + reader } /// BLAKE3 extendable-output reader. @@ -3602,6 +4065,9 @@ impl Blake3XofReader { self.root.counter = self.root.counter.wrapping_add(1); self.position_within_block = 0; } + if u32::from(self.root.flags) & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize(&mut block); + } *out = &mut core::mem::take(out)[take..]; } @@ -3688,7 +4154,7 @@ impl_xof_read!(Blake3XofReader); #[must_use] fn compress_chunk_tail_to_root_words( kernel: Kernel, - cv: [u32; 8], + mut cv: [u32; 8], block: &[u8; BLOCK_LEN], block_len: usize, flags: u32, @@ -3705,7 +4171,11 @@ fn compress_chunk_tail_to_root_words( kernels::Blake3KernelId::X86Sse41 | kernels::Blake3KernelId::X86Avx2 | kernels::Blake3KernelId::X86Avx512 => { // SAFETY: dispatch validates required CPU features before selecting // each x86 kernel; `block` is a readable 64-byte buffer. - return unsafe { (kernel.x86_compress_cv_bytes)(&cv, block.as_ptr(), 0, block_len as u32, final_flags) }; + let output = unsafe { (kernel.x86_compress_cv_bytes)(&cv, block.as_ptr(), 0, block_len as u32, final_flags) }; + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words(&mut cv); + } + return output; } _ => {} } @@ -3715,13 +4185,23 @@ fn compress_chunk_tail_to_root_words( { if kernel.id == kernels::Blake3KernelId::Aarch64Neon { // SAFETY: NEON availability validated by dispatch - return unsafe { aarch64::compress_cv_neon_bytes(&cv, block.as_ptr(), 0, block_len as u32, final_flags) }; + let output = unsafe { aarch64::compress_cv_neon_bytes(&cv, block.as_ptr(), 0, block_len as u32, final_flags) }; + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words(&mut cv); + } + return output; } } // Portable fallback - let block_words = words16_from_le_bytes_64(block); - first_8_words((kernel.compress)(&cv, &block_words, 0, block_len as u32, final_flags)) + let mut block_words = words16_from_le_bytes_64(block); + let output = first_8_words((kernel.compress)(&cv, &block_words, 0, block_len as u32, final_flags)); + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words_no_fence(&mut cv); + ct::zeroize_words_no_fence(&mut block_words); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } + output } /// Hash tiny input (≤64B) directly to the root hash bytes. @@ -3730,13 +4210,19 @@ fn compress_chunk_tail_to_root_words( /// It handles input padding and dispatches to the appropriate kernel. #[inline] #[must_use] -fn hash_tiny_to_root_words(kernel: Kernel, key_words: [u32; 8], flags: u32, input: &[u8]) -> [u32; 8] { +fn hash_tiny_to_root_words(kernel: Kernel, mut key_words: [u32; 8], flags: u32, input: &[u8]) -> [u32; 8] { debug_assert!(input.len() <= BLOCK_LEN); let mut block = [0u8; BLOCK_LEN]; block[..input.len()].copy_from_slice(input); - compress_chunk_tail_to_root_words(kernel, key_words, &block, input.len(), flags, true) + let output = compress_chunk_tail_to_root_words(kernel, key_words, &block, input.len(), flags, true); + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_no_fence(&mut block); + ct::zeroize_words_no_fence(&mut key_words); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } + output } /// Hash one chunk-or-less input directly to root words. @@ -3745,7 +4231,12 @@ fn hash_tiny_to_root_words(kernel: Kernel, key_words: [u32; 8], flags: u32, inpu /// assembly/intrinsics helpers. #[inline] #[must_use] -fn digest_one_chunk_root_hash_words_generic(kernel: Kernel, key_words: [u32; 8], flags: u32, input: &[u8]) -> [u32; 8] { +fn digest_one_chunk_root_hash_words_generic( + kernel: Kernel, + mut key_words: [u32; 8], + flags: u32, + input: &[u8], +) -> [u32; 8] { debug_assert!(input.len() <= CHUNK_LEN); let (full_blocks, last_len) = if input.is_empty() { @@ -3769,8 +4260,11 @@ fn digest_one_chunk_root_hash_words_generic(kernel: Kernel, key_words: [u32; 8], let offset = i * BLOCK_LEN; let block_flags = flags | if i == 0 { CHUNK_START } else { 0 }; // SAFETY: `offset + BLOCK_LEN <= input.len()` by construction. - let block_words = unsafe { words16_from_le_bytes_64(&*input.as_ptr().add(offset).cast::<[u8; BLOCK_LEN]>()) }; + let mut block_words = unsafe { words16_from_le_bytes_64(&*input.as_ptr().add(offset).cast::<[u8; BLOCK_LEN]>()) }; cv = first_8_words((kernel.compress)(&cv, &block_words, 0, BLOCK_LEN as u32, block_flags)); + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words(&mut block_words); + } } let start = if full_blocks == 0 { CHUNK_START } else { 0 }; @@ -3779,8 +4273,15 @@ fn digest_one_chunk_root_hash_words_generic(kernel: Kernel, key_words: [u32; 8], if last_len == BLOCK_LEN && !input.is_empty() { let offset = full_blocks * BLOCK_LEN; // SAFETY: `offset + BLOCK_LEN <= input.len()` by construction. - let block_words = unsafe { words16_from_le_bytes_64(&*input.as_ptr().add(offset).cast::<[u8; BLOCK_LEN]>()) }; - return first_8_words((kernel.compress)(&cv, &block_words, 0, BLOCK_LEN as u32, final_flags)); + let mut block_words = unsafe { words16_from_le_bytes_64(&*input.as_ptr().add(offset).cast::<[u8; BLOCK_LEN]>()) }; + let output = first_8_words((kernel.compress)(&cv, &block_words, 0, BLOCK_LEN as u32, final_flags)); + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words_no_fence(&mut block_words); + ct::zeroize_words_no_fence(&mut cv); + ct::zeroize_words_no_fence(&mut key_words); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } + return output; } let mut final_block = [0u8; BLOCK_LEN]; @@ -3790,8 +4291,16 @@ fn digest_one_chunk_root_hash_words_generic(kernel: Kernel, key_words: [u32; 8], unsafe { ptr::copy_nonoverlapping(input.as_ptr().add(offset), final_block.as_mut_ptr(), last_len) }; } - let final_words = words16_from_le_bytes_64(&final_block); - first_8_words((kernel.compress)(&cv, &final_words, 0, last_len as u32, final_flags)) + let mut final_words = words16_from_le_bytes_64(&final_block); + let output = first_8_words((kernel.compress)(&cv, &final_words, 0, last_len as u32, final_flags)); + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_no_fence(&mut final_block); + ct::zeroize_words_no_fence(&mut final_words); + ct::zeroize_words_no_fence(&mut cv); + ct::zeroize_words_no_fence(&mut key_words); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } + output } // x86_64 tiny one-shot helpers (keyed/derive sensitive) @@ -3837,7 +4346,7 @@ fn use_x86_hash_many_exact_block_one_chunk_fast_path(kernel: Kernel, input_len: #[cfg(target_arch = "x86_64")] #[inline] unsafe fn avx2_owned_exact_block_chain( - key_words: [u32; 8], + mut key_words: [u32; 8], flags: u32, input: &[u8], final_extra_flags: u32, @@ -3872,13 +4381,19 @@ unsafe fn avx2_owned_exact_block_chain( ) }; } - cv + let output = cv; + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words_no_fence(&mut cv); + ct::zeroize_words_no_fence(&mut key_words); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } + output } #[cfg(all(feature = "diag", target_arch = "x86_64"))] #[inline] unsafe fn avx512_owned_exact_block_hash_many( - key_words: [u32; 8], + mut key_words: [u32; 8], flags: u32, input: &[u8], final_extra_flags: u32, @@ -3912,14 +4427,21 @@ unsafe fn avx512_owned_exact_block_hash_many( let mut lane0 = [0u8; OUT_LEN]; lane0.copy_from_slice(&out[..OUT_LEN]); - words8_from_le_bytes_32(&lane0) + let output = words8_from_le_bytes_32(&lane0); + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_no_fence(&mut out); + ct::zeroize_no_fence(&mut lane0); + ct::zeroize_words_no_fence(&mut key_words); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } + output } #[cfg(target_arch = "x86_64")] #[inline] unsafe fn xof_oneshot_single_chunk_x86_exact_blocks( kernel: Kernel, - key_words: [u32; 8], + mut key_words: [u32; 8], flags: u32, input: &[u8], ) -> Option { @@ -3928,6 +4450,9 @@ unsafe fn xof_oneshot_single_chunk_x86_exact_blocks( debug_assert!(input.len().is_multiple_of(BLOCK_LEN)); if !use_x86_hash_many_exact_block_one_chunk_fast_path(kernel, input.len()) { + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words(&mut key_words); + } return None; } @@ -3942,8 +4467,11 @@ unsafe fn xof_oneshot_single_chunk_x86_exact_blocks( let prefix_len = prefix_blocks * BLOCK_LEN; // SAFETY: AVX2 dispatch selected this kernel. `prefix_len` is non-zero, // exact-block aligned, and within `input`. - let cv = unsafe { avx2_owned_exact_block_chain(key_words, flags, &input[..prefix_len], 0) }; + let mut cv = unsafe { avx2_owned_exact_block_chain(key_words, flags, &input[..prefix_len], 0) }; cv_bytes = words8_to_le_bytes(&cv); + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words(&mut cv); + } } kernels::Blake3KernelId::X86Avx512 => { #[cfg(feature = "diag")] @@ -3951,8 +4479,11 @@ unsafe fn xof_oneshot_single_chunk_x86_exact_blocks( let prefix_len = prefix_blocks * BLOCK_LEN; // SAFETY: Diagnostic availability checked the owned AVX-512 hash-many feature set, and the prefix // is exact-block aligned. - let cv = unsafe { avx512_owned_exact_block_hash_many(key_words, flags, &input[..prefix_len], 0) }; + let mut cv = unsafe { avx512_owned_exact_block_hash_many(key_words, flags, &input[..prefix_len], 0) }; cv_bytes = words8_to_le_bytes(&cv); + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words(&mut cv); + } } else { let input_ptrs = [input.as_ptr()]; #[cfg(any(target_os = "linux", target_os = "macos", target_os = "windows"))] @@ -3973,7 +4504,12 @@ unsafe fn xof_oneshot_single_chunk_x86_exact_blocks( ); } #[cfg(not(any(target_os = "linux", target_os = "macos", target_os = "windows")))] - return None; + { + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words(&mut key_words); + } + return None; + } } #[cfg(not(feature = "diag"))] { @@ -3996,10 +4532,20 @@ unsafe fn xof_oneshot_single_chunk_x86_exact_blocks( ); } #[cfg(not(any(target_os = "linux", target_os = "macos", target_os = "windows")))] - return None; + { + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words(&mut key_words); + } + return None; + } + } + } + _ => { + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words(&mut key_words); } + return None; } - _ => return None, } let mut block_bytes = [0u8; BLOCK_LEN]; @@ -4009,21 +4555,28 @@ unsafe fn xof_oneshot_single_chunk_x86_exact_blocks( ptr::copy_nonoverlapping(input.as_ptr().add(last_offset), block_bytes.as_mut_ptr(), BLOCK_LEN); } - Some(Blake3XofReader::new(RootEmitState { + let reader = Blake3XofReader::new(RootEmitState { kernel_id: kernel.id, input_chaining_value: words8_from_le_bytes_32(&cv_bytes), block_bytes, counter: 0, block_len: BLOCK_LEN as u8, flags: (flags | CHUNK_END) as u8, - })) + }); + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_no_fence(&mut cv_bytes); + ct::zeroize_no_fence(&mut block_bytes); + ct::zeroize_words_no_fence(&mut key_words); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } + Some(reader) } #[cfg(target_arch = "x86_64")] #[inline] unsafe fn digest_one_chunk_root_hash_words_x86( kernel: Kernel, - key_words: [u32; 8], + mut key_words: [u32; 8], flags: u32, input: &[u8], ) -> [u32; 8] { @@ -4041,7 +4594,11 @@ unsafe fn digest_one_chunk_root_hash_words_x86( debug_assert!((flags | CHUNK_END | ROOT) <= u8::MAX as u32); // SAFETY: AVX2 dispatch selected this kernel, and this branch is restricted // to non-empty exact-block one-chunk input. - return unsafe { avx2_owned_exact_block_chain(key_words, flags, input, CHUNK_END | ROOT) }; + let output = unsafe { avx2_owned_exact_block_chain(key_words, flags, input, CHUNK_END | ROOT) }; + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words(&mut key_words); + } + return output; } #[cfg(any(target_os = "linux", target_os = "macos", target_os = "windows"))] @@ -4059,7 +4616,11 @@ unsafe fn digest_one_chunk_root_hash_words_x86( if kernel.owned_x86_hash_many { // SAFETY: Diagnostic availability checked the owned AVX-512 hash-many feature set, and `input` is // one exact-block chunk prefix no longer than CHUNK_LEN. - return unsafe { avx512_owned_exact_block_hash_many(key_words, flags, input, CHUNK_END | ROOT) }; + let output = unsafe { avx512_owned_exact_block_hash_many(key_words, flags, input, CHUNK_END | ROOT) }; + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words(&mut key_words); + } + return output; } #[cfg(feature = "diag")] let force_avx512_exact_block_asm = kernel.force_x86_avx512_exact_block_asm; @@ -4071,7 +4632,11 @@ unsafe fn digest_one_chunk_root_hash_words_x86( // narrow so 64..1024B stay on the proven AVX-512 exact-block path. // SAFETY: this branch is only enabled when AVX2 is available per dispatch, // and input is one contiguous exact-block chunk. - return unsafe { avx2_owned_exact_block_chain(key_words, flags, input, CHUNK_END | ROOT) }; + let output = unsafe { avx2_owned_exact_block_chain(key_words, flags, input, CHUNK_END | ROOT) }; + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words(&mut key_words); + } + return output; } let input_ptrs = [input.as_ptr()]; let mut out = [0u8; OUT_LEN]; @@ -4091,7 +4656,13 @@ unsafe fn digest_one_chunk_root_hash_words_x86( out.as_mut_ptr(), ); } - return words8_from_le_bytes_32(&out); + let output = words8_from_le_bytes_32(&out); + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_no_fence(&mut out); + ct::zeroize_words_no_fence(&mut key_words); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } + return output; } // Keep one final block for CHUNK_END|ROOT. For aligned inputs we process @@ -4127,7 +4698,15 @@ unsafe fn digest_one_chunk_root_hash_words_x86( kernels::Blake3KernelId::X86Avx512 => { cv = (kernel.x86_compress_cv_bytes)(&cv, first_block_ptr, 0, BLOCK_LEN as u32, first_flags); } - _ => return digest_one_chunk_root_hash_words_generic(kernel, key_words, flags, input), + _ => { + let output = digest_one_chunk_root_hash_words_generic(kernel, key_words, flags, input); + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words_no_fence(&mut cv); + ct::zeroize_words_no_fence(&mut key_words); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } + return output; + } } } @@ -4155,10 +4734,24 @@ unsafe fn digest_one_chunk_root_hash_words_x86( kernels::Blake3KernelId::X86Avx512 => { cv = (kernel.x86_compress_cv_bytes)(&cv, block_ptr, 0, BLOCK_LEN as u32, final_flags); } - _ => return digest_one_chunk_root_hash_words_generic(kernel, key_words, flags, input), + _ => { + let output = digest_one_chunk_root_hash_words_generic(kernel, key_words, flags, input); + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words_no_fence(&mut cv); + ct::zeroize_words_no_fence(&mut key_words); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } + return output; + } } } - return cv; + let output = cv; + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words_no_fence(&mut cv); + ct::zeroize_words_no_fence(&mut key_words); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } + return output; } // Partial final block (including empty): pad to 64 bytes. @@ -4182,10 +4775,26 @@ unsafe fn digest_one_chunk_root_hash_words_x86( kernels::Blake3KernelId::X86Avx512 => { cv = (kernel.x86_compress_cv_bytes)(&cv, block_ptr, 0, last_len as u32, final_flags); } - _ => return digest_one_chunk_root_hash_words_generic(kernel, key_words, flags, input), + _ => { + let output = digest_one_chunk_root_hash_words_generic(kernel, key_words, flags, input); + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_no_fence(&mut padded); + ct::zeroize_words_no_fence(&mut cv); + ct::zeroize_words_no_fence(&mut key_words); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } + return output; + } } } - cv + let output = cv; + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_no_fence(&mut padded); + ct::zeroize_words_no_fence(&mut cv); + ct::zeroize_words_no_fence(&mut key_words); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } + output } // aarch64 tiny one-shot helpers (keyed/derive sensitive) @@ -4194,7 +4803,7 @@ unsafe fn digest_one_chunk_root_hash_words_x86( #[inline] unsafe fn digest_one_chunk_root_hash_words_aarch64( kernel: Kernel, - key_words: [u32; 8], + mut key_words: [u32; 8], flags: u32, input: &[u8], ) -> [u32; 8] { @@ -4254,7 +4863,14 @@ unsafe fn digest_one_chunk_root_hash_words_aarch64( // SAFETY: `block_ptr` points to 64 bytes (either into `input` or `padded`), // and dispatch only selects NEON when the required CPU features are present. - unsafe { aarch64::compress_cv_neon_bytes(&cv, block_ptr, 0, last_len as u32, final_flags) } + let output = unsafe { aarch64::compress_cv_neon_bytes(&cv, block_ptr, 0, last_len as u32, final_flags) }; + if flags & (KEYED_HASH | DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_no_fence(&mut padded); + ct::zeroize_words_no_fence(&mut cv); + ct::zeroize_words_no_fence(&mut key_words); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } + output } #[cfg(test)] diff --git a/src/hashes/crypto/blake3/parallel.rs b/src/hashes/crypto/blake3/parallel.rs index 1b9c1e8f..1324a24d 100644 --- a/src/hashes/crypto/blake3/parallel.rs +++ b/src/hashes/crypto/blake3/parallel.rs @@ -5,7 +5,7 @@ use super::{ hash_full_chunks_cvs_scoped, hash_one_full_chunk_cv, hash_power_of_two_subtree_roots_parallel_rayon, join, kernels, pow2_floor, reduce_power_of_two_chunk_cvs_any, single_chunk_output, words8_to_le_bytes, words16_from_le_bytes_64, }; -use crate::hashes::crypto::blake3::kernels::Kernel; +use crate::{hashes::crypto::blake3::kernels::Kernel, traits::ct}; struct ParallelBatchSpec<'a> { input: &'a [u8], @@ -20,16 +20,40 @@ struct ParallelBatchSpec<'a> { pub(super) struct ParallelBatchScratch { roots: alloc::vec::Vec<[u32; 8]>, leaf_cvs: alloc::vec::Vec<[u32; 8]>, + zeroize: bool, } impl ParallelBatchScratch { #[inline] fn clear(&mut self) { + if self.zeroize { + ct::zeroize_words_no_fence(self.roots.as_flattened_mut()); + ct::zeroize_words_no_fence(self.leaf_cvs.as_flattened_mut()); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } self.roots.clear(); self.leaf_cvs.clear(); + self.zeroize = false; + } +} + +impl Drop for ParallelBatchScratch { + fn drop(&mut self) { + self.clear(); } } +#[cfg(feature = "diag")] +#[doc(hidden)] +#[unsafe(no_mangle)] +#[inline(never)] +pub fn diag_zeroize_blake3_parallel_scratch(input: [u32; 8]) -> u32 { + let mut scratch = ParallelBatchScratch::default(); + scratch.zeroize = true; + scratch.roots.push(input); + core::hint::black_box(scratch.roots[0][0]) +} + impl Blake3 { #[inline] fn streaming_parallel_threads( @@ -50,22 +74,32 @@ impl Blake3 { #[inline(never)] fn commit_parallel_batch(&mut self, batch: ParallelBatchSpec<'_>, scratch: &mut ParallelBatchScratch) { #[inline] - fn push_stack(stack: &mut [MaybeUninit<[u32; 8]>; CV_STACK_LEN], len: &mut usize, cv: [u32; 8]) { + fn push_stack(stack: &mut [MaybeUninit<[u32; 8]>; CV_STACK_LEN], len: &mut usize, mut cv: [u32; 8], zeroize: bool) { stack[*len].write(cv); *len += 1; + if zeroize { + ct::zeroize_words(&mut cv); + } } #[inline] - fn pop_stack(stack: &mut [MaybeUninit<[u32; 8]>; CV_STACK_LEN], len: &mut usize) -> [u32; 8] { + fn pop_stack(stack: &mut [MaybeUninit<[u32; 8]>; CV_STACK_LEN], len: &mut usize, zeroize: bool) -> [u32; 8] { *len -= 1; // SAFETY: `len` tracks the number of initialized entries. - unsafe { stack[*len].assume_init_read() } + let cv = unsafe { stack[*len].assume_init_read() }; + if zeroize { + // SAFETY: the slot held the `Copy` CV read above and remains + // initialized until the next push overwrites it. + ct::zeroize_words(unsafe { stack[*len].assume_init_mut() }); + } + cv } let batch_input = batch.input; let base_counter = batch.base_counter; let commit = batch.commit_chunks; let threads = batch.threads; + let zeroize = self.chunk_state.flags & (super::KEYED_HASH | super::DERIVE_KEY_MATERIAL) != 0; let mut stack_len = self.cv_stack_len as usize; let mut counter = base_counter; @@ -73,6 +107,7 @@ impl Blake3 { let mut remaining_commit = commit; scratch.clear(); + scratch.zeroize = zeroize; while remaining_commit != 0 { let mut size = pow2_floor(remaining_commit); @@ -95,7 +130,7 @@ impl Blake3 { let subtree_input = &batch_input[bytes_base..bytes_base + subtree_bytes]; let bulk_kernel = kernels::kernel(self.bulk_kernel_id); - let subtree_cv = if size >= threads && (counter == 0 || (counter & (size as u64 - 1)) == 0) { + let mut subtree_cv = if size >= threads && (counter == 0 || (counter & (size as u64 - 1)) == 0) { const MAX_SUBTREE_CHUNKS: usize = 1 << 12; let mut subtree_chunks = size / threads; @@ -151,16 +186,26 @@ impl Blake3 { let mut total = counter >> level; let mut cv = subtree_cv; while total & 1 == 0 { - cv = kernels::parent_cv_inline( - self.bulk_kernel_id, - pop_stack(&mut self.cv_stack, &mut stack_len), - cv, - self.key_words, - self.chunk_state.flags, - ); + let mut left = pop_stack(&mut self.cv_stack, &mut stack_len, zeroize); + let mut next_cv = + kernels::parent_cv_inline(self.bulk_kernel_id, left, cv, self.key_words, self.chunk_state.flags); + if zeroize { + ct::zeroize_words_no_fence(&mut left); + ct::zeroize_words_no_fence(&mut cv); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } + core::mem::swap(&mut cv, &mut next_cv); + if zeroize { + ct::zeroize_words(&mut next_cv); + } total >>= 1; } - push_stack(&mut self.cv_stack, &mut stack_len, cv); + push_stack(&mut self.cv_stack, &mut stack_len, cv, zeroize); + if zeroize { + ct::zeroize_words_no_fence(&mut cv); + ct::zeroize_words_no_fence(&mut subtree_cv); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } offset_chunks = offset_chunks.strict_add(size); remaining_commit -= size; @@ -312,7 +357,7 @@ fn compress_subtree_wide_bytes( let mut out_len = full_chunks; let rem = chunks_exact.remainder(); if !rem.is_empty() { - let cv_words = single_chunk_output( + let mut cv_words = single_chunk_output( kernel, key_words, chunk_counter.strict_add(full_chunks as u64), @@ -322,6 +367,9 @@ fn compress_subtree_wide_bytes( .chaining_value(); out[out_len] = words8_to_le_bytes(&cv_words); out_len = out_len.strict_add(1); + if flags & (super::KEYED_HASH | super::DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_words(&mut cv_words); + } } return out_len; @@ -377,14 +425,18 @@ fn compress_subtree_wide_bytes( debug_assert_eq!(left_n, degree); debug_assert!(right_n >= 1 && right_n <= left_n); - if left_n == 1 { + let out_len = if left_n == 1 { out[0] = left_out[0]; out[1] = right_out[0]; - return 2; + 2 + } else { + let num_children = left_n.strict_add(right_n); + compress_parents_parallel_bytes(kernel, &cv_array[..num_children], key_words, flags, out) + }; + if flags & (super::KEYED_HASH | super::DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize(cv_array.as_flattened_mut()); } - - let num_children = left_n.strict_add(right_n); - compress_parents_parallel_bytes(kernel, &cv_array[..num_children], key_words, flags, out) + out_len } #[inline] @@ -422,13 +474,18 @@ fn compress_subtree_to_parent_node_bytes( let mut out = [0u8; BLOCK_LEN]; out[..OUT_LEN].copy_from_slice(&cv_array[0]); out[OUT_LEN..].copy_from_slice(&cv_array[1]); + if flags & (super::KEYED_HASH | super::DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_no_fence(cv_array.as_flattened_mut()); + ct::zeroize_no_fence(out_array.as_flattened_mut()); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } out } #[inline] pub(super) fn root_output_oneshot_join_parallel( kernel: Kernel, - key_words: [u32; 8], + mut key_words: [u32; 8], flags: u32, input: &[u8], threads: usize, @@ -440,15 +497,22 @@ pub(super) fn root_output_oneshot_join_parallel( let depth = (usize::BITS - 1 - threads.leading_zeros()) as usize; let budget = depth.max(1); - let parent_block = + let mut parent_block = compress_subtree_to_parent_node_bytes::(kernel, key_words, 0, flags, input, budget); - let block_words = words16_from_le_bytes_64(&parent_block); - OutputState { + let mut block_words = words16_from_le_bytes_64(&parent_block); + let output = OutputState { kernel_id: kernel.id, input_chaining_value: key_words, block_words, counter: 0, block_len: BLOCK_LEN as u32, flags: super::PARENT | flags, + }; + if flags & (super::KEYED_HASH | super::DERIVE_KEY_MATERIAL) != 0 { + ct::zeroize_no_fence(&mut parent_block); + ct::zeroize_words_no_fence(&mut block_words); + ct::zeroize_words_no_fence(&mut key_words); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); } + output } diff --git a/src/hashes/crypto/keccak.rs b/src/hashes/crypto/keccak.rs index 1be772b1..3556dac6 100644 --- a/src/hashes/crypto/keccak.rs +++ b/src/hashes/crypto/keccak.rs @@ -1062,7 +1062,7 @@ impl KeccakCoreImpl(&self, ds: u8, out: &mut [u8; OUT]) { debug_assert!(OUT <= RATE); - let state = self.finalize_state(ds); + let mut state = self.finalize_state(ds); let (chunks, rem) = out.as_chunks_mut::<8>(); for (chunk, &word) in chunks.iter_mut().zip(state.iter()) { @@ -1072,6 +1072,9 @@ impl KeccakCoreImpl KeccakXofImpl { diff --git a/src/hashes/crypto/sha256/mod.rs b/src/hashes/crypto/sha256/mod.rs index c7d12ec3..c33c1aa5 100644 --- a/src/hashes/crypto/sha256/mod.rs +++ b/src/hashes/crypto/sha256/mod.rs @@ -581,9 +581,19 @@ impl Sha256 { #[inline] fn finalize_impl(&self) -> [u8; 32] { if kernels::COMPILE_TIME_HW { - return self.finalize_inner_with(kernels::compile_time_best()); + return self.finalize_inner_with::(kernels::compile_time_best()); } - self.finalize_inner_with(self.compress_blocks) + self.finalize_inner_with::(self.compress_blocks) + } + + #[cfg(feature = "hmac")] + #[cfg_attr(feature = "diag", inline(never))] + #[cfg_attr(not(feature = "diag"), inline)] + pub(crate) fn finalize_secret(&self) -> [u8; 32] { + if kernels::COMPILE_TIME_HW { + return self.finalize_inner_with::(kernels::compile_time_best()); + } + self.finalize_inner_with::(self.compress_blocks) } #[inline(always)] @@ -592,7 +602,7 @@ impl Sha256 { } #[inline] - fn finalize_inner_with(&self, compress_blocks: CompressBlocksFn) -> [u8; 32] { + fn finalize_inner_with(&self, compress_blocks: CompressBlocksFn) -> [u8; 32] { let mut state = self.state; let mut block = self.block; let mut block_len = self.block_len; @@ -618,6 +628,11 @@ impl Sha256 { for (chunk, &word) in out.chunks_exact_mut(4).zip(state.iter()) { chunk.copy_from_slice(&word.to_be_bytes()); } + if ZEROIZE { + crate::traits::ct::zeroize_words_no_fence(&mut state); + crate::traits::ct::zeroize_no_fence(&mut block); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } out } @@ -663,6 +678,24 @@ impl Sha256 { self.reset_update_mode_to_aligned_prefix(prefix); } + #[inline] + #[cfg(feature = "hmac")] + pub(crate) fn zeroize_secret_state(&mut self) { + crate::traits::ct::zeroize_words_no_fence(&mut self.state); + crate::traits::ct::zeroize_no_fence(&mut self.block); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } + + #[inline] + #[cfg(feature = "hmac")] + pub(crate) fn digest_secret(data: &[u8]) -> [u8; 32] { + let mut state = Self::new(); + state.update(data); + let digest = state.finalize_secret(); + state.zeroize_secret_state(); + digest + } + #[inline] #[cfg(all(feature = "hmac", target_arch = "x86_64"))] fn reset_update_mode_to_aligned_prefix(&mut self, prefix: Sha256Prefix) { diff --git a/src/hashes/crypto/sha384.rs b/src/hashes/crypto/sha384.rs index 085d2c39..a48b6a55 100644 --- a/src/hashes/crypto/sha384.rs +++ b/src/hashes/crypto/sha384.rs @@ -168,9 +168,13 @@ impl Sha384 { } #[inline] - fn finalize_inner_with(&self, compress_blocks: CompressBlocksFn) -> [u8; 48] { + fn finalize_inner_with(&self, compress_blocks: CompressBlocksFn) -> [u8; 48] { let total_len = self.bytes_hashed.strict_add(self.block_len as u128); - let state = Sha512::finalize_state_words(self.state, self.block, self.block_len, total_len, compress_blocks); + let mut state = if ZEROIZE { + Sha512::finalize_state_words_secret(self.state, self.block, self.block_len, total_len, compress_blocks) + } else { + Sha512::finalize_state_words(self.state, self.block, self.block_len, total_len, compress_blocks) + }; let mut out = [0u8; 48]; out[0..8].copy_from_slice(&state[0].to_be_bytes()); @@ -179,9 +183,18 @@ impl Sha384 { out[24..32].copy_from_slice(&state[3].to_be_bytes()); out[32..40].copy_from_slice(&state[4].to_be_bytes()); out[40..48].copy_from_slice(&state[5].to_be_bytes()); + if ZEROIZE { + crate::traits::ct::zeroize_words(&mut state); + } out } + #[inline] + #[cfg(feature = "hmac")] + pub(crate) fn finalize_secret(&self) -> [u8; 48] { + self.finalize_inner_with::(self.compress_blocks) + } + #[inline] #[must_use] #[cfg(feature = "hmac")] @@ -219,6 +232,24 @@ impl Sha384 { self.dispatch = prefix.dispatch; } + #[inline] + #[cfg(feature = "hmac")] + pub(crate) fn zeroize_secret_state(&mut self) { + crate::traits::ct::zeroize_words_no_fence(&mut self.state); + crate::traits::ct::zeroize_no_fence(&mut self.block); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } + + #[inline] + #[cfg(feature = "hmac")] + pub(crate) fn digest_secret(data: &[u8]) -> [u8; 48] { + let mut state = Self::new(); + state.update(data); + let digest = state.finalize_secret(); + state.zeroize_secret_state(); + digest + } + #[cfg(all(feature = "hmac", any(test, feature = "diag")))] #[inline] #[allow(dead_code)] @@ -275,7 +306,7 @@ impl Digest for Sha384 { #[inline] fn finalize(&self) -> Self::Output { - self.finalize_inner_with(self.compress_blocks) + self.finalize_inner_with::(self.compress_blocks) } #[inline] diff --git a/src/hashes/crypto/sha512/mod.rs b/src/hashes/crypto/sha512/mod.rs index 053efc3d..ec9c0e4e 100644 --- a/src/hashes/crypto/sha512/mod.rs +++ b/src/hashes/crypto/sha512/mod.rs @@ -359,6 +359,7 @@ impl Sha512 { } #[inline(always)] + #[cfg(any(feature = "ed25519", test))] fn finalize_state( state: [u64; 8], block: [u8; BLOCK_LEN], @@ -372,6 +373,28 @@ impl Sha512 { #[inline(always)] pub(crate) fn finalize_state_words( + state: [u64; 8], + block: [u8; BLOCK_LEN], + block_len: usize, + total_len: u128, + compress_blocks: CompressBlocksFn, + ) -> [u64; 8] { + Self::finalize_state_words_impl::(state, block, block_len, total_len, compress_blocks) + } + + #[inline(always)] + pub(crate) fn finalize_state_words_secret( + state: [u64; 8], + block: [u8; BLOCK_LEN], + block_len: usize, + total_len: u128, + compress_blocks: CompressBlocksFn, + ) -> [u64; 8] { + Self::finalize_state_words_impl::(state, block, block_len, total_len, compress_blocks) + } + + #[inline(always)] + fn finalize_state_words_impl( mut state: [u64; 8], mut block: [u8; BLOCK_LEN], mut block_len: usize, @@ -394,13 +417,31 @@ impl Sha512 { block[112..128].copy_from_slice(&bit_len.to_be_bytes()); compress_blocks(&mut state, &block); + if ZEROIZE_BLOCK { + crate::traits::ct::zeroize(&mut block); + } state } #[inline(always)] - fn finalize_inner_with(&self, compress_blocks: CompressBlocksFn) -> [u8; 64] { + fn finalize_inner_with(&self, compress_blocks: CompressBlocksFn) -> [u8; 64] { let total_len = self.bytes_hashed.strict_add(self.block_len as u128); - Self::finalize_state(self.state, self.block, self.block_len, total_len, compress_blocks) + let mut state = if ZEROIZE { + Self::finalize_state_words_secret(self.state, self.block, self.block_len, total_len, compress_blocks) + } else { + Self::finalize_state_words(self.state, self.block, self.block_len, total_len, compress_blocks) + }; + let digest = Self::state_to_digest(&state); + if ZEROIZE { + crate::traits::ct::zeroize_words(&mut state); + } + digest + } + + #[inline] + #[cfg(any(feature = "hmac", feature = "ed25519"))] + pub(crate) fn finalize_secret(&self) -> [u8; 64] { + self.finalize_inner_with::(self.compress_blocks) } #[inline(always)] @@ -449,6 +490,24 @@ impl Sha512 { self.dispatch = prefix.dispatch; } + #[inline] + #[cfg(any(feature = "hmac", feature = "ed25519"))] + pub(crate) fn zeroize_secret_state(&mut self) { + crate::traits::ct::zeroize_words_no_fence(&mut self.state); + crate::traits::ct::zeroize_no_fence(&mut self.block); + core::sync::atomic::compiler_fence(core::sync::atomic::Ordering::SeqCst); + } + + #[inline] + #[cfg(any(feature = "hmac", feature = "ed25519"))] + pub(crate) fn digest_secret(data: &[u8]) -> [u8; 64] { + let mut state = Self::new(); + state.update(data); + let digest = state.finalize_secret(); + state.zeroize_secret_state(); + digest + } + #[cfg(all(feature = "hmac", any(test, feature = "diag")))] #[inline] #[allow(dead_code)] @@ -768,7 +827,7 @@ impl Digest for Sha512 { #[inline] fn finalize(&self) -> Self::Output { - self.finalize_inner_with(self.compress_blocks) + self.finalize_inner_with::(self.compress_blocks) } #[inline] diff --git a/src/hex.rs b/src/hex.rs index b37bf606..7dcfb518 100644 --- a/src/hex.rs +++ b/src/hex.rs @@ -22,7 +22,7 @@ use core::fmt; /// Hex decoding error. -#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)] +#[derive(Clone, Copy, PartialEq, Eq, Hash)] #[non_exhaustive] pub enum InvalidHexError { /// Input length is not exactly twice the expected byte count. @@ -36,13 +36,23 @@ pub enum InvalidHexError { }, } +impl fmt::Debug for InvalidHexError { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + match self { + Self::InvalidLength => f.write_str("InvalidLength"), + Self::InvalidChar { index, .. } => f + .debug_struct("InvalidChar") + .field("index", index) + .finish_non_exhaustive(), + } + } +} + impl fmt::Display for InvalidHexError { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { match self { Self::InvalidLength => f.write_str("invalid hex length"), - Self::InvalidChar { byte, index } => { - write!(f, "invalid hex character 0x{byte:02x} at index {index}") - } + Self::InvalidChar { index, .. } => write!(f, "invalid hex character at index {index}"), } } } @@ -460,7 +470,11 @@ mod tests { ); assert_eq!( alloc::format!("{}", InvalidHexError::InvalidChar { byte: b'z', index: 5 }), - "invalid hex character 0x7a at index 5" + "invalid hex character at index 5" + ); + assert_eq!( + alloc::format!("{:?}", InvalidHexError::InvalidChar { byte: b'z', index: 5 }), + "InvalidChar { index: 5, .. }" ); } } diff --git a/src/lib.rs b/src/lib.rs index ccbb13c6..e84d7af8 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -922,10 +922,253 @@ assert!(left.ct_eq(&right).declassify()); "#] pub struct __OwnerEqualityBoundaryAudit; -// Compile-time trait assertions. -// -// Every public type must be Send + Sync + Debug. Most must also be Clone. -// These static assertions fail the build if any contract is broken. +#[cfg(all(doctest, feature = "full"))] +#[doc(hidden)] +#[doc = r#" +The secret-bearing owners below must not acquire a generic `Clone` capability. + +```compile_fail,E0277 +use rscrypto::HmacSha256; + +fn require_clone() {} +require_clone::(); +``` + +```compile_fail,E0277 +use rscrypto::HmacSha384; + +fn require_clone() {} +require_clone::(); +``` + +```compile_fail,E0277 +use rscrypto::HmacSha512; + +fn require_clone() {} +require_clone::(); +``` + +```compile_fail,E0277 +use rscrypto::HmacSha3_224; + +fn require_clone() {} +require_clone::(); +``` + +```compile_fail,E0277 +use rscrypto::HmacSha3_256; + +fn require_clone() {} +require_clone::(); +``` + +```compile_fail,E0277 +use rscrypto::HmacSha3_384; + +fn require_clone() {} +require_clone::(); +``` + +```compile_fail,E0277 +use rscrypto::HmacSha3_512; + +fn require_clone() {} +require_clone::(); +``` + +```compile_fail,E0277 +use rscrypto::HkdfSha256; + +fn require_clone() {} +require_clone::(); +``` + +```compile_fail,E0277 +use rscrypto::HkdfSha384; + +fn require_clone() {} +require_clone::(); +``` + +```compile_fail,E0277 +use rscrypto::HkdfSha512; + +fn require_clone() {} +require_clone::(); +``` + +```compile_fail,E0277 +use rscrypto::Kmac128; + +fn require_clone() {} +require_clone::(); +``` + +```compile_fail,E0277 +use rscrypto::Kmac256; + +fn require_clone() {} +require_clone::(); +``` + +```compile_fail,E0277 +use rscrypto::Pbkdf2Sha256; + +fn require_clone() {} +require_clone::(); +``` + +```compile_fail,E0277 +use rscrypto::Pbkdf2Sha512; + +fn require_clone() {} +require_clone::(); +``` + +```compile_fail,E0277 +use rscrypto::Blake2bParams; + +fn require_clone() {} +require_clone::(); +``` + +```compile_fail,E0277 +use rscrypto::Blake2sParams; + +fn require_clone() {} +require_clone::(); +``` + +```compile_fail,E0277 +use rscrypto::Blake2b; + +fn require_clone() {} +require_clone::(); +``` + +```compile_fail,E0277 +use rscrypto::Poly1305OneTimeKey; + +fn require_clone() {} +require_clone::(); +``` + +```compile_fail,E0277 +use rscrypto::Poly1305; + +fn require_clone() {} +require_clone::(); +``` + +```compile_fail,E0277 +use rscrypto::EcdsaP256SecretKey; + +fn require_clone() {} +require_clone::(); +``` + +```compile_fail,E0277 +use rscrypto::EcdsaP256Keypair; + +fn require_clone() {} +require_clone::(); +``` + +```compile_fail,E0277 +use rscrypto::EcdsaP384SecretKey; + +fn require_clone() {} +require_clone::(); +``` + +```compile_fail,E0277 +use rscrypto::EcdsaP384Keypair; + +fn require_clone() {} +require_clone::(); +``` + +```compile_fail,E0277 +use rscrypto::Ed25519SecretKey; + +fn require_clone() {} +require_clone::(); +``` + +```compile_fail,E0277 +use rscrypto::Ed25519Keypair; + +fn require_clone() {} +require_clone::(); +``` + +```compile_fail,E0277 +use rscrypto::X25519SecretKey; + +fn require_clone() {} +require_clone::(); +``` + +```compile_fail,E0277 +use rscrypto::MlKem512DecapsulationKey; + +fn require_clone() {} +require_clone::(); +``` + +```compile_fail,E0277 +use rscrypto::MlKem768DecapsulationKey; + +fn require_clone() {} +require_clone::(); +``` + +```compile_fail,E0277 +use rscrypto::MlKem1024DecapsulationKey; + +fn require_clone() {} +require_clone::(); +``` + +```compile_fail,E0277 +use rscrypto::MlKem512PreparedDecapsulationKey; + +fn require_clone() {} +require_clone::(); +``` + +```compile_fail,E0277 +use rscrypto::MlKem768PreparedDecapsulationKey; + +fn require_clone() {} +require_clone::(); +``` + +```compile_fail,E0277 +use rscrypto::MlKem1024PreparedDecapsulationKey; + +fn require_clone() {} +require_clone::(); +``` + +```compile_fail,E0277 +use rscrypto::RsaPrivateKey; + +fn require_clone() {} +require_clone::(); +``` + +```compile_fail,E0277 +use rscrypto::RsaPrivateScratch; + +fn require_clone() {} +require_clone::(); +``` +"#] +pub struct __SecretCloneBoundaryAudit; + +// Positive compile-time trait assertions for public capability contracts. +// Secret-bearing non-Clone contracts are compile-fail doctests above. #[cfg(all(test, miri))] mod miri_shadow_tests; diff --git a/src/secret.rs b/src/secret.rs index c7e465a8..2d201f9c 100644 --- a/src/secret.rs +++ b/src/secret.rs @@ -242,6 +242,30 @@ pub fn diag_zeroize_fixed_stack(input: [u8; 32]) -> u8 { core::hint::black_box(secret.as_bytes()[0]) } +#[cfg(feature = "diag")] +#[doc(hidden)] +#[unsafe(no_mangle)] +#[inline(never)] +pub fn diag_zeroize_fixed_move(input: [u8; 32]) -> u8 { + let secret = SecretBytes::new(input); + let mut exposed = secret.expose(); + let output = core::hint::black_box(exposed[0]); + ct::zeroize(&mut exposed); + output +} + +#[cfg(feature = "diag")] +#[doc(hidden)] +#[unsafe(no_mangle)] +#[inline(never)] +pub fn diag_zeroize_early_return(input: [u8; 32], stop: bool) -> u8 { + let secret = SecretBytes::new(input); + if core::hint::black_box(stop) { + return 0; + } + core::hint::black_box(secret.as_bytes()[0]) +} + #[cfg(all(feature = "diag", feature = "alloc"))] #[doc(hidden)] #[unsafe(no_mangle)] diff --git a/src/traits/mac.rs b/src/traits/mac.rs index bbdc1ae1..cba4f1e3 100644 --- a/src/traits/mac.rs +++ b/src/traits/mac.rs @@ -1,7 +1,7 @@ //! Message authentication code traits. //! -//! This trait mirrors [`crate::Digest`]: keyed construction, streaming updates, -//! idempotent finalize, and reset support. +//! This trait provides keyed construction, streaming updates, idempotent +//! finalize, and reset support without requiring keyed state to be cloneable. use core::fmt::Debug; @@ -16,7 +16,11 @@ use crate::traits::VerificationError; /// until verification returns one success or failure result. External /// implementations must provide their own equivalent verification boundary; /// the trait cannot prove its timing behavior. -pub trait Mac: Clone { +/// +/// MAC state deliberately does not require [`Clone`]. Reuse one owner through +/// [`reset`](Self::reset), or construct another state explicitly when an +/// independent keyed owner is required. +pub trait Mac: Sized { /// Tag size in bytes. const TAG_SIZE: usize; diff --git a/tests/api_consistency.rs b/tests/api_consistency.rs index ccf341c1..16f450c7 100644 --- a/tests/api_consistency.rs +++ b/tests/api_consistency.rs @@ -124,6 +124,45 @@ fn assert_mac_api() { assert!(mac.verify(&expected).is_ok()); } +#[cfg(any(feature = "hmac", feature = "hmac-sha3"))] +struct NonCloneMac { + key: u8, + state: u8, +} + +#[cfg(any(feature = "hmac", feature = "hmac-sha3"))] +impl Mac for NonCloneMac { + const TAG_SIZE: usize = 1; + type Tag = [u8; Self::TAG_SIZE]; + + fn new(key: &[u8]) -> Self { + Self { + key: key.first().copied().unwrap_or(0), + state: 0, + } + } + + fn update(&mut self, data: &[u8]) { + self.state = data.iter().fold(self.state, |state, byte| state.wrapping_add(*byte)); + } + + fn finalize(&self) -> Self::Tag { + [self.state ^ self.key] + } + + fn reset(&mut self) { + self.state = 0; + } + + fn verify(&self, expected: &Self::Tag) -> Result<(), VerificationError> { + if self.finalize() == *expected { + Ok(()) + } else { + Err(VerificationError::new()) + } + } +} + #[cfg(feature = "aead")] fn assert_aead_api(key: A::Key, nonce: A::Nonce) where @@ -223,6 +262,12 @@ fn all_macs_follow_new_update_finalize_reset() { assert_mac_api::(); } +#[test] +#[cfg(any(feature = "hmac", feature = "hmac-sha3"))] +fn mac_trait_accepts_non_clone_implementations() { + assert_mac_api::(); +} + #[test] #[cfg(feature = "hmac-sha3")] fn hmac_sha3_macs_follow_new_update_finalize_reset() { diff --git a/tests/secret_redaction.rs b/tests/secret_redaction.rs new file mode 100644 index 00000000..6b154ac8 --- /dev/null +++ b/tests/secret_redaction.rs @@ -0,0 +1,336 @@ +use core::fmt::Debug; + +fn assert_debug_snapshot(value: &impl Debug, expected: &str) { + assert_eq!(format!("{value:?}"), expected); +} + +#[test] +fn generic_secret_owner_debug_is_redacted() { + assert_debug_snapshot(&rscrypto::SecretBytes::new([0x53; 16]), "SecretBytes(****)"); +} + +#[test] +fn keyed_state_debug_snapshots_are_redacted() { + const KEY_16: [u8; 16] = [0x53; 16]; + const KEY_32: [u8; 32] = [0x53; 32]; + + #[cfg(feature = "aes-gcm")] + { + use rscrypto::{Aes128Gcm, Aes128GcmKey, Aes256Gcm, Aes256GcmKey}; + + let key = Aes128GcmKey::from_bytes(KEY_16); + assert_debug_snapshot(&key, "Aes128GcmKey(****)"); + assert_debug_snapshot(&Aes128Gcm::new(&key), "Aes128Gcm { .. }"); + + let key = Aes256GcmKey::from_bytes(KEY_32); + assert_debug_snapshot(&key, "Aes256GcmKey(****)"); + assert_debug_snapshot(&Aes256Gcm::new(&key), "Aes256Gcm { .. }"); + } + + #[cfg(feature = "aes-gcm-siv")] + { + use rscrypto::{Aes128GcmSiv, Aes128GcmSivKey, Aes256GcmSiv, Aes256GcmSivKey}; + + let key = Aes128GcmSivKey::from_bytes(KEY_16); + assert_debug_snapshot(&key, "Aes128GcmSivKey(****)"); + assert_debug_snapshot(&Aes128GcmSiv::new(&key), "Aes128GcmSiv { .. }"); + + let key = Aes256GcmSivKey::from_bytes(KEY_32); + assert_debug_snapshot(&key, "Aes256GcmSivKey(****)"); + assert_debug_snapshot(&Aes256GcmSiv::new(&key), "Aes256GcmSiv { .. }"); + } + + #[cfg(feature = "chacha20poly1305")] + { + use rscrypto::{ChaCha20Poly1305, ChaCha20Poly1305Key}; + + let key = ChaCha20Poly1305Key::from_bytes(KEY_32); + assert_debug_snapshot(&key, "ChaCha20Poly1305Key(****)"); + assert_debug_snapshot(&ChaCha20Poly1305::new(&key), "ChaCha20Poly1305 { .. }"); + } + + #[cfg(feature = "xchacha20poly1305")] + { + use rscrypto::{XChaCha20Poly1305, XChaCha20Poly1305Key}; + + let key = XChaCha20Poly1305Key::from_bytes(KEY_32); + assert_debug_snapshot(&key, "XChaCha20Poly1305Key(****)"); + assert_debug_snapshot(&XChaCha20Poly1305::new(&key), "XChaCha20Poly1305 { .. }"); + } + + #[cfg(feature = "aegis256")] + { + use rscrypto::{Aegis256, Aegis256Key}; + + let key = Aegis256Key::from_bytes(KEY_32); + assert_debug_snapshot(&key, "Aegis256Key(****)"); + assert_debug_snapshot(&Aegis256::new(&key), "Aegis256 { .. }"); + } + + #[cfg(feature = "ascon-aead")] + { + use rscrypto::{AsconAead128, AsconAead128Key}; + + let key = AsconAead128Key::from_bytes(KEY_16); + assert_debug_snapshot(&key, "AsconAead128Key(****)"); + assert_debug_snapshot(&AsconAead128::new(&key), "AsconAead128 { .. }"); + } + + #[cfg(feature = "hmac")] + { + use rscrypto::{HmacSha256, HmacSha384, HmacSha512, Mac}; + + assert_debug_snapshot(&HmacSha256::new(&KEY_32), "HmacSha256 { .. }"); + assert_debug_snapshot(&HmacSha384::new(&KEY_32), "HmacSha384 { .. }"); + assert_debug_snapshot(&HmacSha512::new(&KEY_32), "HmacSha512 { .. }"); + } + + #[cfg(feature = "hmac-sha3")] + { + use rscrypto::{HmacSha3_224, HmacSha3_256, HmacSha3_384, HmacSha3_512, Mac}; + + assert_debug_snapshot(&HmacSha3_224::new(&KEY_32), "HmacSha3_224 { .. }"); + assert_debug_snapshot(&HmacSha3_256::new(&KEY_32), "HmacSha3_256 { .. }"); + assert_debug_snapshot(&HmacSha3_384::new(&KEY_32), "HmacSha3_384 { .. }"); + assert_debug_snapshot(&HmacSha3_512::new(&KEY_32), "HmacSha3_512 { .. }"); + } + + #[cfg(feature = "hkdf")] + { + use rscrypto::{HkdfSha256, HkdfSha384, HkdfSha512}; + + assert_debug_snapshot(&HkdfSha256::new(b"salt", &KEY_32), "HkdfSha256 { .. }"); + assert_debug_snapshot(&HkdfSha384::new(b"salt", &KEY_32), "HkdfSha384 { .. }"); + assert_debug_snapshot(&HkdfSha512::new(b"salt", &KEY_32), "HkdfSha512 { .. }"); + } + + #[cfg(feature = "kmac")] + { + use rscrypto::{Kmac128, Kmac256}; + + assert_debug_snapshot(&Kmac128::new(&KEY_32, b"custom"), "Kmac128 { .. }"); + assert_debug_snapshot(&Kmac256::new(&KEY_32, b"custom"), "Kmac256 { .. }"); + } + + #[cfg(feature = "pbkdf2")] + { + use rscrypto::{Pbkdf2Sha256, Pbkdf2Sha512}; + + assert_debug_snapshot(&Pbkdf2Sha256::new(&KEY_32), "Pbkdf2Sha256 { .. }"); + assert_debug_snapshot(&Pbkdf2Sha512::new(&KEY_32), "Pbkdf2Sha512 { .. }"); + } + + #[cfg(feature = "poly1305")] + { + use rscrypto::{Poly1305, Poly1305OneTimeKey}; + + let key = Poly1305OneTimeKey::from_bytes(KEY_32); + assert_debug_snapshot(&key, "Poly1305OneTimeKey(****)"); + assert_debug_snapshot(&Poly1305::new(key), "Poly1305 { .. }"); + } +} + +#[test] +fn keyed_hash_debug_snapshots_are_redacted() { + const KEY: [u8; 32] = [0x53; 32]; + + #[cfg(feature = "blake2s")] + { + let params = rscrypto::Blake2sParams::new().key(&KEY); + assert_debug_snapshot( + ¶ms, + "Blake2sParams { key_len: 32, salt: [0, 0, 0, 0, 0, 0, 0, 0], personal: [0, 0, 0, 0, 0, 0, 0, 0] }", + ); + assert_debug_snapshot(¶ms.build_128(), "Blake2s128 { .. }"); + assert_debug_snapshot(¶ms.build_256(), "Blake2s256 { .. }"); + } + + #[cfg(feature = "blake2b")] + { + let params = rscrypto::Blake2bParams::new().key(&KEY); + assert_debug_snapshot( + ¶ms, + "Blake2bParams { key_len: 32, salt: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], personal: [0, 0, 0, 0, 0, \ + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] }", + ); + assert_debug_snapshot(¶ms.build_256(), "Blake2b256 { .. }"); + assert_debug_snapshot(¶ms.build_512(), "Blake2b512 { .. }"); + assert_debug_snapshot(¶ms.build(32), "Blake2b { output_len: 32, .. }"); + } + + #[cfg(feature = "blake3")] + { + assert_debug_snapshot(&rscrypto::Blake3::new_keyed(&KEY), "Blake3 { .. }"); + assert_debug_snapshot(&rscrypto::Blake3::keyed_xof(&KEY, b"message"), "Blake3XofReader { .. }"); + } +} + +#[test] +fn private_key_and_shared_secret_debug_snapshots_are_redacted() { + #[cfg(feature = "ecdsa-p256")] + { + let secret = rscrypto::EcdsaP256SecretKey::from_bytes([1u8; 32]).expect("valid P-256 scalar"); + assert_debug_snapshot(&secret, "EcdsaP256SecretKey(****)"); + let keypair = rscrypto::EcdsaP256Keypair::from_secret_key(secret); + assert_debug_snapshot( + &keypair, + &format!("EcdsaP256Keypair {{ public: {:?}, .. }}", keypair.public_key()), + ); + } + + #[cfg(feature = "ecdsa-p384")] + { + let secret = rscrypto::EcdsaP384SecretKey::from_bytes([1u8; 48]).expect("valid P-384 scalar"); + assert_debug_snapshot(&secret, "EcdsaP384SecretKey(****)"); + let keypair = rscrypto::EcdsaP384Keypair::from_secret_key(secret); + assert_debug_snapshot( + &keypair, + &format!("EcdsaP384Keypair {{ public: {:?}, .. }}", keypair.public_key()), + ); + } + + #[cfg(feature = "ed25519")] + { + let secret = rscrypto::Ed25519SecretKey::from_bytes([0x53; 32]); + assert_debug_snapshot(&secret, "Ed25519SecretKey(****)"); + let keypair = rscrypto::Ed25519Keypair::from_secret_key(secret); + assert_debug_snapshot( + &keypair, + &format!("Ed25519Keypair {{ public: {:?}, .. }}", keypair.public_key()), + ); + } + + #[cfg(feature = "x25519")] + { + let secret = rscrypto::X25519SecretKey::from_bytes([0x53; 32]); + assert_debug_snapshot(&secret, "X25519SecretKey(****)"); + let shared = secret + .diffie_hellman(&rscrypto::X25519PublicKey::basepoint()) + .expect("basepoint exchange must produce a nonzero shared secret"); + assert_debug_snapshot(&shared, "X25519SharedSecret(****)"); + } + + #[cfg(feature = "ml-kem")] + { + assert_debug_snapshot( + &rscrypto::MlKem512DecapsulationKey::from_bytes([0x53; rscrypto::MlKem512DecapsulationKey::LENGTH]), + "MlKem512DecapsulationKey(****)", + ); + assert_debug_snapshot( + &rscrypto::MlKem768DecapsulationKey::from_bytes([0x53; rscrypto::MlKem768DecapsulationKey::LENGTH]), + "MlKem768DecapsulationKey(****)", + ); + assert_debug_snapshot( + &rscrypto::MlKem1024DecapsulationKey::from_bytes([0x53; rscrypto::MlKem1024DecapsulationKey::LENGTH]), + "MlKem1024DecapsulationKey(****)", + ); + assert_debug_snapshot( + &rscrypto::MlKem512SharedSecret::from_bytes([0x53; rscrypto::MlKem512SharedSecret::LENGTH]), + "MlKem512SharedSecret(****)", + ); + assert_debug_snapshot( + &rscrypto::MlKem768SharedSecret::from_bytes([0x53; rscrypto::MlKem768SharedSecret::LENGTH]), + "MlKem768SharedSecret(****)", + ); + assert_debug_snapshot( + &rscrypto::MlKem1024SharedSecret::from_bytes([0x53; rscrypto::MlKem1024SharedSecret::LENGTH]), + "MlKem1024SharedSecret(****)", + ); + + macro_rules! assert_prepared_decapsulation_key_redacted { + ($profile:ty, $expected:literal) => {{ + use rscrypto::Kem as _; + + let (_, key) = <$profile>::generate_keypair(|out| { + for (index, byte) in out.iter_mut().enumerate() { + *byte = 0x53u8.wrapping_add(index as u8); + } + Ok::<(), rscrypto::MlKemError>(()) + }) + .expect("deterministic ML-KEM key generation must succeed"); + let prepared = key.prepare().expect("generated ML-KEM key must validate"); + assert_debug_snapshot(&prepared, $expected); + }}; + } + + assert_prepared_decapsulation_key_redacted!(rscrypto::MlKem512, "MlKem512PreparedDecapsulationKey(****)"); + assert_prepared_decapsulation_key_redacted!(rscrypto::MlKem768, "MlKem768PreparedDecapsulationKey(****)"); + assert_prepared_decapsulation_key_redacted!(rscrypto::MlKem1024, "MlKem1024PreparedDecapsulationKey(****)"); + } +} + +#[test] +fn borrowed_secret_context_debug_snapshot_is_redacted() { + #[cfg(feature = "argon2")] + assert_debug_snapshot( + &rscrypto::Argon2Context::new(b"pepper-value", b"associated-value"), + "Argon2Context { secret: \"[REDACTED]\", secret_len: 12, associated_data_len: 16 }", + ); +} + +#[test] +fn secret_input_error_snapshots_do_not_echo_input_bytes() { + #[cfg(feature = "aes-gcm")] + { + let error = "5353535353535353535353535353535z" + .parse::() + .expect_err("invalid secret hex must fail"); + assert_debug_snapshot(&error, "InvalidChar { index: 31, .. }"); + assert_eq!(error.to_string(), "invalid hex character at index 31"); + } + + #[cfg(feature = "ecdsa-p256")] + { + let error = rscrypto::EcdsaP256SecretKey::from_bytes([0u8; 32]).expect_err("zero scalar must fail"); + assert_debug_snapshot(&error, "InvalidSecretKey"); + assert_eq!(error.to_string(), "invalid ECDSA secret key"); + + struct SeedBearingError; + + impl Debug for SeedBearingError { + fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result { + f.write_str("SeedBearingError(53535353)") + } + } + + impl core::fmt::Display for SeedBearingError { + fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result { + f.write_str("seed-bearing error: 53535353") + } + } + + impl core::error::Error for SeedBearingError {} + + let generation_error = rscrypto::EcdsaP256SecretKey::try_generate_with(|_| Err(SeedBearingError)) + .expect_err("random-source failure must be preserved as an opaque error"); + assert_debug_snapshot(&generation_error, "Random(..)"); + assert_eq!( + generation_error.to_string(), + "ECDSA key-generation random source failed" + ); + assert!(core::error::Error::source(&generation_error).is_none()); + } + + #[cfg(feature = "ml-kem")] + { + let error = rscrypto::MlKem512DecapsulationKey::try_from_slice(&[0x53; 31]) + .expect_err("wrong-length decapsulation key must fail"); + assert_debug_snapshot(&error, "InvalidDecapsulationKey"); + assert_eq!(error.to_string(), "ML-KEM decapsulation key failed validation"); + } + + #[cfg(feature = "kmac")] + { + let error = + rscrypto::Kmac256::verify_tag(b"secret key", b"custom", b"message", &[]).expect_err("empty KMAC tag must fail"); + assert_debug_snapshot(&error, "VerificationError"); + assert_eq!(error.to_string(), "verification failed"); + } + + #[cfg(feature = "aes-gcm")] + { + let error = rscrypto::OpenError::verification(); + assert_debug_snapshot(&error, "Verification(VerificationError)"); + assert_eq!(error.to_string(), "verification failed"); + } +}