Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions .cargo/config-avx.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# AVX WITHOUT AVX2 (Sandy Bridge class) — the `simd_avx.rs` realization.
#
# Usage:
# env -u RUSTFLAGS cargo --config .cargo/config-avx.toml <cmd>
# bash scripts/masking-parity.sh avx-qemu (runs under qemu -cpu SandyBridge)
#
# Selects the arm `all(target_feature = "avx", not(target_feature = "avx2"))`:
# Sandy Bridge / Ivy Bridge, AMD Bulldozer–Jaguar, VMs that hide AVX2. Plan:
# `.claude/plans/simd-avx1-backend-v1.md`. Overlay semantics as in
# `config-v3.toml`: only the target-cpu is set here; the crypto cfgs come from
# `.cargo/config.toml` through cargo's rustflags join. `env -u RUSTFLAGS` is
# load-bearing for the same reason.
[target.'cfg(target_arch = "x86_64")']
rustflags = ["-Ctarget-cpu=sandybridge"]
28 changes: 28 additions & 0 deletions .claude/blackboard.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,23 @@
## 2026-10-10 — `simd_avx.rs`: AVX-without-AVX2 realization (Sandy/Ivy Bridge, Bulldozer–Jaguar), plus three pre-existing x86 fixes

- **What:** a seventh realization, selected at compile time by `all(target_feature = "avx", not(target_feature = "avx2"))`. You get it with `target-cpu=native` on such a CPU, or with `.cargo/config-avx.toml` (`sandybridge`). The types and the `simd.rs` exports are unchanged; the 49 AVX2-only items in `simd_avx2.rs` and 19 in `simd_avx512.rs` are cfg-gated, and `simd_avx.rs` supplies the same methods from SSE2–SSE4.1 halves plus AVX1 float-domain ops. There are no runtime feature checks. The `cpu_guard` AVX2 startup floor applies only when the AVX2 arm is compiled in. Baseline builds (no `avx` cfg) are UNCHANGED: they keep the AVX2 arm and the floor. Whether the SSE tier should take them over is an open operator question.
- **Evidence:**
- `scripts/masking-parity.sh avx-qemu` passes all 15 groups under `qemu-x86_64-static -cpu SandyBridge`, with header `avx2=false avx512f=false`.
- 422 `simd` lib tests pass on that arm under qemu.
- The CI `realization/avx` row runs both, plus an objdump witness. The witness checks the parity binary for AVX2 integer ymm ops, FMA3 and F16C, and finds none. It can fire: the AVX2-op check finds 20,162 hits on a v3 binary. Its FMA/F16C check has never been seen to fire, because the parity program never calls `mul_add`; the unit test under qemu covers that case instead.
- Clippy `-D warnings` is clean on v3, v4, avx and baseline. Lib tests pass on v3 (2510) and v4 (2561). Codegen witness avx2/avx512 and parity v3/v4 PASS.
- **sentinel-qa BLOCK, fixed:** `F32x8::mul_add` called `_mm256_fmadd_ps` ungated, and F32x8 is exported on this arm, so a Sandy Bridge build faulted. It now has `cfg(target_feature = "fma")` with a per-lane `f32::mul_add` fallback (fused, one rounding). Test `f32x8_mul_add_is_fused` separates fused from unfused rounding. Disable: forcing the FMA body on the AVX build gives SIGILL under qemu. The F16C sites in `simd_avx512.rs` are runtime-detected and need nothing.
- **Pre-existing bugs found and fixed:**
- `simd_avx2::dot_i8`, a safe `pub fn` with no in-tree callers: `vpmaddubsw` saturated at `a = b = 127` (4080 instead of 516128), and a shorter `b` was read past its end. This is inventory row 410. Both arms now use `pmovsxbw` + `pmaddwd`, fold into an i64 every 16,384 chunks, and sum over `min` length. Test `simd_avx2_dot_i8_is_exact` is red on the old code on both arms.
- Five x86 256-bit methods had **no test on any stable arm**: `U16x16::{mullo, blend_epi32}`, `U8x32::shuffle_bytes`, `I8x32::cmp_gt` and `I16x16::cmp_gt`. Found because their disable runs stayed green. Test `x86_256bit_integer_methods_match_scalar` now checks them against scalar references on every x86 arm.
- **Disable runs on `simd_avx.rs`, all red, each restored:** cmpgt bias dropped (6 tests), `reduce_min` shuffle, saturating sub → wrapping, `transpose8` unpack swap, blend operand swap, I8 cmp_gt operand swap, I16 cmp_gt half swap, shuffle operand swap, mullo → mulhi. The first run of the I16 half swap crashed qemu itself (`QEMU internal SIGSEGV`) under the multithreaded harness. That is not a detection; single-threaded, the assertion fails as it should.
- **Unsafe:** 29 new blocks in `simd_avx.rs`, all SAFETY-commented, verdict IRREDUCIBLE. They are appended to `.claude/knowledge/unsafe-inventory/sites.tsv` with a README addendum.
- **Open:**
- The SSE-tier scope decision (all non-AVX builds vs only v2 builds).
- The `saturating_abs` SSSE3 patch, awaiting approval.
- `dot_i8` changed its results; it has no in-tree callers, so the effect on external consumers is unknown.
- **Plan:** `.claude/plans/simd-avx1-backend-v1.md`.

## 2026-10-08 — `PillarReport::deferred` + `certified()`; `Cascade::query` documents that Stroke 1 can drop true hits

- **Pillars:** Pillar-15/16/17 run no probe but report `passed = true`. A gate on `passed` counted them as certified. `PillarReport` now has `deferred: bool` (true only for those three) and `certified() = passed && !deferred`. `print()` shows `DEFERRED`. `passed` keeps its meaning, so existing callers do not change.
Expand Down Expand Up @@ -3630,3 +3650,11 @@ qemu `-cpu Nehalem` 132, `SandyBridge` 132, `IvyBridge` 132, `Haswell` 0,
`max` 0. Side effect: a baseline build on a PRE-AVX CPU now gets the message
too (the guard is not VEX-encoded there). The optional pre-AVX to-do above
remains only for builds compiled with `target-cpu` v3/v4/native.

## simd_avx.rs (AVX-without-AVX2 backend) — started 2026-10-10, plan `.claude/plans/simd-avx1-backend-v1.md`

Operator chose Option A after #348. P0 landed: `.cargo/config-avx.toml`
(`-Ctarget-cpu=sandybridge`) and `scripts/masking-parity.sh avx-qemu` (run under
`qemu-x86_64-static -cpu SandyBridge`). Before the backend exists the arm FAILS
with exit 132 from `cpu_guard`'s AVX2 floor (measured) — the expected red.
No CI row yet; it is added when the arm goes green.
29 changes: 29 additions & 0 deletions .claude/knowledge/unsafe-inventory/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -144,3 +144,32 @@ A flag is a lead, not a verdict. The first four were re-read at source and hold.
per type to two.
4. SAFETY comments, macro-first.
5. `SAFE-API` swaps; `SAFE-CRATE` only with approval.

## Addendum 2026-10-10 — `src/simd_avx.rs` (AVX-without-AVX2 arm)

The snapshot above predates this file. Its **29 sites** are appended to
`sites.tsv` (rows at the end, line numbers from the commit that added them).
All are `unsafe { }` blocks over SSE2–SSE4.1 / AVX1 intrinsics, all carry a
`// SAFETY:` comment, and all get verdict **IRREDUCIBLE**: per the probe table,
an x86 intrinsic in a plain fn is still E0133 on rustc 1.99 even when the
feature is compiled in. Their footing is the arm predicate
`all(target_feature = "avx", not(target_feature = "avx2"))`: the module only
exists in builds where AVX (and, through it, SSE4.2) is enabled for the whole
binary, and `cpu_guard` checks those features at startup.

Also changed by the same arc: `F32x8::mul_add` in `src/simd_avx512.rs` now
issues `_mm256_fmadd_ps` only under `cfg(target_feature = "fma")` and otherwise
uses `f32::mul_add` per lane. Before, it was an ungated FMA3 intrinsic in a
type exported on the AVX arm (sentinel-qa BLOCK; a Sandy Bridge build faulted,
verified under `qemu -cpu SandyBridge`). The site count for that file is
unchanged; its line numbers in `sites.tsv` have moved by a few rows.

**Resolved by the same arc — `src/simd_avx2.rs` row 410 (`dot_i8`).** The
bounds flag (a shorter `b` read 32 bytes past its end inside a safe `pub fn`)
is fixed: both x86 arms now sum over `min(a.len(), b.len())` bytes with
`chunks_exact`, so every load is in bounds. The same commit fixes a wrong-value
bug the flag did not name: the `vpmaddubsw` form saturated i16 at
`a = b = 127` (returned 4080 instead of 516128 over 32 bytes). The rows' second
flag ("safe pub fn uses AVX2 with no cfg") is now covered: the AVX2 bodies are
cfg-gated off the AVX-without-AVX2 arm, and a baseline (no `avx` cfg) build
keeps `cpu_guard`'s AVX2 startup floor.
29 changes: 29 additions & 0 deletions .claude/knowledge/unsafe-inventory/sites.tsv
Original file line number Diff line number Diff line change
Expand Up @@ -1300,3 +1300,32 @@ src/zip/ndproducer.rs 360 UPSTREAM fn src UNSAFE-FN-API missing - unsafe fn: cal
src/zip/ndproducer.rs 364 UPSTREAM fn src UNSAFE-FN-API missing - unsafe fn: caller upholds pointer/bounds/validity precondition; raw-pointer core design ptr_arith unsafe fn uget_ptr(&self, i: &Self::Dim) -> *const A {
src/zip/ndproducer.rs 409 UPSTREAM fn src UNSAFE-FN-API missing - unsafe fn: caller upholds pointer/bounds/validity precondition; raw-pointer core design ptr_arith unsafe fn as_ref(&self, ptr: *mut A) -> *mut A {
src/zip/ndproducer.rs 413 UPSTREAM fn src UNSAFE-FN-API missing - unsafe fn: caller upholds pointer/bounds/validity precondition; raw-pointer core design ptr_arith unsafe fn uget_ptr(&self, i: &Self::Dim) -> *mut A {
src/simd_avx.rs 42 FORK block src IRREDUCIBLE present - SSE2-SSE4.1/AVX1 intrinsics on the AVX-without-AVX2 arm; still E0133 on rustc 1.99 (see probe table), so the block cannot be removed x86_intrinsic unsafe { (_mm256_castsi256_si128(v), _mm256_extractf128_si256::<1>(v)) }
src/simd_avx.rs 49 FORK block src IRREDUCIBLE present - SSE2-SSE4.1/AVX1 intrinsics on the AVX-without-AVX2 arm; still E0133 on rustc 1.99 (see probe table), so the block cannot be removed x86_intrinsic unsafe { _mm256_set_m128i(hi, lo) }
src/simd_avx.rs 59 FORK block src IRREDUCIBLE present - SSE2-SSE4.1/AVX1 intrinsics on the AVX-without-AVX2 arm; still E0133 on rustc 1.99 (see probe table), so the block cannot be removed x86_intrinsic unsafe { join($op(al, bl), $op(ah, bh)) }
src/simd_avx.rs 69 FORK block src IRREDUCIBLE present - SSE2-SSE4.1/AVX1 intrinsics on the AVX-without-AVX2 arm; still E0133 on rustc 1.99 (see probe table), so the block cannot be removed x86_intrinsic unsafe { _mm256_castps_si256($op(_mm256_castsi256_ps($a), _mm256_castsi256_ps($b))) }
src/simd_avx.rs 79 FORK block src IRREDUCIBLE present - SSE2-SSE4.1/AVX1 intrinsics on the AVX-without-AVX2 arm; still E0133 on rustc 1.99 (see probe table), so the block cannot be removed x86_intrinsic unsafe { (_mm_movemask_epi8(lo) as u32 & 0xFFFF) | ((_mm_movemask_epi8(hi) as u32) << 16)
src/simd_avx.rs 93 FORK block src IRREDUCIBLE present - SSE2-SSE4.1/AVX1 intrinsics on the AVX-without-AVX2 arm; still E0133 on rustc 1.99 (see probe table), so the block cannot be removed x86_intrinsic let mut sum = unsafe {
src/simd_avx.rs 133 FORK block src IRREDUCIBLE present - SSE2-SSE4.1/AVX1 intrinsics on the AVX-without-AVX2 arm; still E0133 on rustc 1.99 (see probe table), so the block cannot be removed x86_intrinsic let mut result = unsafe {
src/simd_avx.rs 174 FORK block src IRREDUCIBLE present - SSE2-SSE4.1/AVX1 intrinsics on the AVX-without-AVX2 arm; still E0133 on rustc 1.99 (see probe table), so the block cannot be removed x86_intrinsic unsafe {
src/simd_avx.rs 192 FORK block src IRREDUCIBLE present - SSE2-SSE4.1/AVX1 intrinsics on the AVX-without-AVX2 arm; still E0133 on rustc 1.99 (see probe table), so the block cannot be removed x86_intrinsic,raw_load_store,ptr_arith unsafe {
src/simd_avx.rs 212 FORK block src IRREDUCIBLE present - SSE2-SSE4.1/AVX1 intrinsics on the AVX-without-AVX2 arm; still E0133 on rustc 1.99 (see probe table), so the block cannot be removed x86_intrinsic,raw_load_store,ptr_arith unsafe { _mm256_loadu_pd(v.0.as_ptr().add(4 * half) as *const f64) }
src/simd_avx.rs 219 FORK block src IRREDUCIBLE present - SSE2-SSE4.1/AVX1 intrinsics on the AVX-without-AVX2 arm; still E0133 on rustc 1.99 (see probe table), so the block cannot be removed x86_intrinsic let c = unsafe {
src/simd_avx.rs 235 FORK block src IRREDUCIBLE present - SSE2-SSE4.1/AVX1 intrinsics on the AVX-without-AVX2 arm; still E0133 on rustc 1.99 (see probe table), so the block cannot be removed x86_intrinsic,raw_load_store,ptr_arith unsafe { _mm256_storeu_pd(out[4 * bj + k].0.as_mut_ptr().add(4 * bi) as *mut f64, v) };
src/simd_avx.rs 276 FORK block src IRREDUCIBLE present - SSE2-SSE4.1/AVX1 intrinsics on the AVX-without-AVX2 arm; still E0133 on rustc 1.99 (see probe table), so the block cannot be removed x86_intrinsic,raw_load_store,ptr_arith core::array::from_fn(|k| unsafe { _mm_loadu_si128(self.0.as_ptr().add(4 * k) as *const __m
src/simd_avx.rs 284 FORK block src IRREDUCIBLE present - SSE2-SSE4.1/AVX1 intrinsics on the AVX-without-AVX2 arm; still E0133 on rustc 1.99 (see probe table), so the block cannot be removed x86_intrinsic unsafe {
src/simd_avx.rs 297 FORK block src IRREDUCIBLE present - SSE2-SSE4.1/AVX1 intrinsics on the AVX-without-AVX2 arm; still E0133 on rustc 1.99 (see probe table), so the block cannot be removed x86_intrinsic unsafe {
src/simd_avx.rs 314 FORK block src IRREDUCIBLE present - SSE2-SSE4.1/AVX1 intrinsics on the AVX-without-AVX2 arm; still E0133 on rustc 1.99 (see probe table), so the block cannot be removed x86_intrinsic let q = unsafe { _mm_movemask_ps(_mm_castsi128_ps(_mm_cmpgt_epi32(a[k], b[k]))) as u32 };
src/simd_avx.rs 332 FORK block src IRREDUCIBLE present - SSE2-SSE4.1/AVX1 intrinsics on the AVX-without-AVX2 arm; still E0133 on rustc 1.99 (see probe table), so the block cannot be removed x86_intrinsic unsafe {
src/simd_avx.rs 343 FORK block src IRREDUCIBLE present - SSE2-SSE4.1/AVX1 intrinsics on the AVX-without-AVX2 arm; still E0133 on rustc 1.99 (see probe table), so the block cannot be removed x86_intrinsic unsafe {
src/simd_avx.rs 360 FORK block src IRREDUCIBLE present - SSE2-SSE4.1/AVX1 intrinsics on the AVX-without-AVX2 arm; still E0133 on rustc 1.99 (see probe table), so the block cannot be removed x86_intrinsic Self(unsafe { _mm256_permute2f128_si256::<IMM>(self.0, other.0) })
src/simd_avx.rs 370 FORK block src IRREDUCIBLE present - SSE2-SSE4.1/AVX1 intrinsics on the AVX-without-AVX2 arm; still E0133 on rustc 1.99 (see probe table), so the block cannot be removed x86_intrinsic Self(unsafe {
src/simd_avx.rs 394 FORK block src IRREDUCIBLE present - SSE2-SSE4.1/AVX1 intrinsics on the AVX-without-AVX2 arm; still E0133 on rustc 1.99 (see probe table), so the block cannot be removed x86_intrinsic unsafe {
src/simd_avx.rs 478 FORK block src IRREDUCIBLE present - SSE2-SSE4.1/AVX1 intrinsics on the AVX-without-AVX2 arm; still E0133 on rustc 1.99 (see probe table), so the block cannot be removed x86_intrinsic let ones = unsafe { _mm256_set1_epi16(-1) };
src/simd_avx.rs 496 FORK block src IRREDUCIBLE present - SSE2-SSE4.1/AVX1 intrinsics on the AVX-without-AVX2 arm; still E0133 on rustc 1.99 (see probe table), so the block cannot be removed x86_intrinsic,raw_load_store,ptr_arith unsafe {
src/simd_avx.rs 527 FORK block src IRREDUCIBLE present - SSE2-SSE4.1/AVX1 intrinsics on the AVX-without-AVX2 arm; still E0133 on rustc 1.99 (see probe table), so the block cannot be removed x86_intrinsic let bias = unsafe { _mm256_set1_epi8(i8::MIN) };
src/simd_avx.rs 562 FORK block src IRREDUCIBLE present - SSE2-SSE4.1/AVX1 intrinsics on the AVX-without-AVX2 arm; still E0133 on rustc 1.99 (see probe table), so the block cannot be removed x86_intrinsic unsafe {
src/simd_avx.rs 573 FORK block src IRREDUCIBLE present - SSE2-SSE4.1/AVX1 intrinsics on the AVX-without-AVX2 arm; still E0133 on rustc 1.99 (see probe table), so the block cannot be removed x86_intrinsic unsafe {
src/simd_avx.rs 605 FORK block src IRREDUCIBLE present - SSE2-SSE4.1/AVX1 intrinsics on the AVX-without-AVX2 arm; still E0133 on rustc 1.99 (see probe table), so the block cannot be removed x86_intrinsic Self(unsafe { join(_mm_blendv_epi8(bl, al, ml), _mm_blendv_epi8(bh, ah, mh)) })
src/simd_avx.rs 688 FORK block src IRREDUCIBLE present - SSE2-SSE4.1/AVX1 intrinsics on the AVX-without-AVX2 arm; still E0133 on rustc 1.99 (see probe table), so the block cannot be removed x86_intrinsic unsafe {
src/simd_avx.rs 755 FORK block src IRREDUCIBLE present - SSE2-SSE4.1/AVX1 intrinsics on the AVX-without-AVX2 arm; still E0133 on rustc 1.99 (see probe table), so the block cannot be removed x86_intrinsic unsafe {
70 changes: 70 additions & 0 deletions .claude/plans/simd-avx1-backend-v1.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# Plan: `simd_avx.rs` — AVX-without-AVX2 backend (v1)

Operator decision 2026-10-10 (after PR #348): Option A, a real backend for CPUs
with AVX but not AVX2 (Sandy Bridge / Ivy Bridge, AMD Bulldozer–Jaguar, VMs that
hide AVX2), selected at compile time. No runtime feature checks in the SIMD code.

## Selection

* Arm predicate: `all(target_arch = "x86_64", target_feature = "avx",
not(target_feature = "avx2"))` — written out at each gate (no build.rs, no
cfg-alias crate).
* Reached by: `target-cpu=native` on such a CPU, or the new
`.cargo/config-avx.toml` (`-Ctarget-cpu=sandybridge`).
* Unchanged: AVX-512 builds, AVX2 builds, and BASELINE builds (no `avx` cfg).
A baseline build keeps the AVX2 arm and the `cpu_guard` AVX2 startup floor —
switching it to this arm would slow every baseline build on AVX2 hosts.
(Open question for the operator; default is "unchanged".)
* `cpu_guard::BACKEND_FLOOR` applies only when the AVX2 arm is compiled in. On
this arm the normal compiled-feature table already requires `avx`.

## Shape: impl blocks, not types

Measured: the AVX2-only intrinsics sit in impl blocks; the TYPES are shared and
referenced across the arm (`F32x16`→`I32x16`/`U32x16`, `F64x8`→`U64x8`,
`U16x32`→`U8x64`, `U8x32`→`U16x16`). Redefining types would cascade, so:

* Type definitions and `simd.rs` export lists do not change.
* In `simd_avx2.rs` / `simd_avx512.rs`, every method that calls an AVX2-only
intrinsic moves into an impl block gated `not(<arm predicate>)`. Mixed blocks
are split; their portable methods stay shared (no duplication).
* `simd_avx.rs` provides the same methods, same signatures, gated on the arm:
two `__m128i` halves via SSE2/SSSE3/SSE4.1 (all present on every AVX CPU),
plus AVX1 for loads/stores/float-domain bit ops.

Affected — ⊘ the first table here counted doc-comment mentions and listed
`U8x64`, `U32x16`, `U32x8`, `U16x8`, `palette_lookup_u8x8` and a 1-op `F32x8`
item. Re-measured with comments stripped (`tools/avx2_gate.py`): those are
scalar or comment-only and needed nothing. What actually needed a gate:

| item | gated in | notes |
|---|---|---|
| `U8x32` | `simd_avx2.rs` | 15 methods + 5 traits; biggest |
| `U16x16` | `simd_avx2.rs` | 8 methods + 12 traits |
| `U64x8` | `simd_avx2.rs` | `rotl_half`, `mul_lo32`, `transpose8`, `Shl`/`Shr<Self>` (per-lane: no SSE `sllv`) |
| `I32x16` | `simd_avx2.rs` | `reduce_min/max`, `gt_bitmask` |
| `popcount`, `dot_i8` (free fns) | `simd_avx2.rs` | re-exported from `simd_avx` on the arm |
| `I8x32`, `I16x16` | `simd_avx512.rs` | 6 + 5 methods, 4 traits each |
| `F32x8::mul_add` | `simd_avx512.rs` | FMA3, not AVX2 — missed by the AVX2-only scan, caught by sentinel-qa; now `cfg(target_feature = "fma")` with a per-lane `f32::mul_add` fallback |

49 gates in `simd_avx2.rs`, 19 in `simd_avx512.rs`. F16C sites in
`simd_avx512.rs` are runtime-detected (`is_x86_feature_detected!`) and need none.

## Work items

- [x] P0 TESTS FIRST: a `native-avx` arm in `crates/simd-masking-parity`
(`--config .cargo/config-avx.toml`, run under `qemu-x86_64-static -cpu
SandyBridge`), wired into `scripts/masking-parity.sh` and the
`simd-matrix` CI workflow. Expect it to SIGILL before the backend exists.
- [x] P1 `.cargo/config-avx.toml`; arm predicate; `cpu_guard` floor gated.
- [x] P2 split the affected impl blocks (no behaviour change on other arms;
codegen-witness avx2/avx512 must stay byte-for-byte green).
- [x] P3 `simd_avx.rs` per type, one type per chunk (U8x32, U16x16, U64x8,
U32x8, U32x16, I32x16, U8x64, popcount/dot_i8, I8x32, I16x16, U16x8,
palette_lookup_u8x8). Every `unsafe` gets SAFETY + sentinel-qa audit.
- [x] P4 gates: lib tests on the avx arm under qemu SandyBridge; parity on all
six existing arms + the new one; v3/v4 clippy; disable-run per type.
- [x] P5 docs: CLAUDE.md realization list, contract doc, blackboard, unsafe
inventory rows for the new file.

Estimate: ~1,000–1,500 lines, 2–3 sessions.
Loading
Loading