Skip to content

Deduplicate CSR column search in MatrixCSR - #4461

Draft
garth-wells wants to merge 3 commits into
mainfrom
garth/csr-position-dedup
Draft

Deduplicate CSR column search in MatrixCSR#4461
garth-wells wants to merge 3 commits into
mainfrom
garth/csr-position-dedup

Conversation

@garth-wells

@garth-wells garth-wells commented Aug 31, 2026

Copy link
Copy Markdown
Member

Summary

Split out from #4438: insert_csr, insert_blocked_csr, insert_nonblocked_csr,
and MatrixCSR's ghost-row unpacking each duplicated the same
std::lower_bound-based column search. This factors it into a shared
la::impl::csr_position() helper, and fixes a drive-by bug found while
doing so.

This was dropped from #4438 during rebasing/scope-narrowing of that PR
(now assembly-focused only) — re-proposing standalone since it's an
independent, self-contained improvement to la::MatrixCSR.

Changes

  • Factor the repeated column-search logic into la::impl::csr_position().
  • A hand-rolled "branchless" (conditional-move) rewrite of this search was
    tried first, on the theory that the column found is unpredictable from
    one call to the next so a branchy std::lower_bound mispredicts. It
    measured as a reproducible ~13% regression in
    scripts/bench_backend.py's csr_reasm (P1, N=100): GCC 13 did not turn
    the idiom into a cmov, confirmed by disassembly — it kept a real
    conditional branch, structurally near-identical to libstdc++'s own
    lower_bound, just without libstdc++'s tuning. Reverted to a thin
    wrapper around std::lower_bound instead; the deduplication alone
    measured a genuine, reproducible +6.2% on P1/N=100 csr_reasm (no
    significant change on P3/N=64, where kernel cost dominates insertion
    cost).
  • clang if-converts csr_position's std::lower_bound into a branchless
    cmov chain, ~35-50% slower here than GCC's branchy codegen (the cmov
    serializes each iteration's load behind the prior comparison, killing
    speculation on this short, cache-resident row). Confirmed via a
    standalone reproducer and disassembly; a hand-rolled loop with
    __builtin_expect recovers full speed, but reimplementing lower_bound
    over a compiler codegen quirk isn't worth it, so this is left as-is and
    documented in a code comment so it isn't re-tried without new evidence.
  • Drive-by: MatrixCSR::add() passed _row_ptr.size() as its debug-only
    row-range bound, one row looser than set()'s correct
    size_local()+num_ghosts(). Aligned add() to match, closing a
    theoretical one-row out-of-bounds read for row == num_all_rows() in
    Developer builds.

Simplify block-size handling with auto/std::integral_constant

insert_csr/insert_blocked_csr took their block sizes as non-type
template parameters (template <int BS0, int BS1, ...>, always
compile-time), while insert_nonblocked_csr took plain runtime int bs0, int bs1 (via std::div). Converted all three to the
BlockSizeArg auto convention already used by spmv/spmvT in this
same file (a parameter that accepts either a runtime integral or a
std::integral_constant<U, N>), with decltype(+BS0)-typed loop
counters. insert_nonblocked_csr's std::div calls become plain /,
%, since std::div requires concrete arithmetic types incompatible
with a generic auto parameter. MatrixCSR::set/add's public
template <int BS0, int BS1> API is unchanged; they now pass
std::integral_constant<int, BS0>{} into insert_csr/
insert_blocked_csr instead of explicit template arguments.

Testing

  • Formatting: clang-format --dry-run --Werror passes on all touched
    files.
  • The csr_position dedup + row-bound fix commits previously passed the
    full local C++ test suite (27476 assertions) and python/test
    unit/la + unit/fem (2594 tests) while part of Assembly performance optimisations #4438, before being
    reverted there purely to narrow that PR's scope to assembly
    performance.
  • The block-size-handling simplification could not be exercised through
    the full local test suite in this environment (local MPI toolchain is
    currently broken here — a Homebrew open-mpi upgrade invalidated the
    cached PETSc/MPI paths — unrelated to this change). Instead verified
    standalone: compiled matrix_csr_impl.h directly with both GCC 16 and
    Apple clang under -std=c++20 -Wall -Wextra -Wsign-compare (clean, no
    warnings), and cross-checked insert_csr, insert_blocked_csr, and
    insert_nonblocked_csr — each called with both a runtime int and a
    std::integral_constant block size — against an independently
    constructed dense reference matrix; all six combinations reproduce the
    same matrix.

🤖 Generated with Claude Code

garth-wells and others added 3 commits August 31, 2026 16:48
insert_csr, insert_blocked_csr, insert_nonblocked_csr, and MatrixCSR's
ghost-row unpacking each duplicated the same std::lower_bound-based
column search. Factored into a shared la::impl::csr_position() helper.

A hand-rolled "branchless" (conditional-move) rewrite of this search
was tried first, on the theory that the column found is unpredictable
from one call to the next so a branchy std::lower_bound mispredicts.
It measured as a reproducible ~13% regression in
scripts/bench_backend.py's csr_reasm (P1, N=100): GCC 13 did not turn
the idiom into a cmov, confirmed by disassembly -- it kept a real
conditional branch, structurally near-identical to libstdc++'s own
lower_bound, just without libstdc++'s tuning. Reverted to a thin
wrapper around std::lower_bound instead; the deduplication alone
measured a genuine, reproducible +6.2% on P1/N=100 csr_reasm (no
significant change on P3/N=64, where kernel cost dominates insertion
cost). The failed branchless attempt is documented in a code comment
so it isn't re-tried without new evidence.

Drive-by: MatrixCSR::add() passed _row_ptr.size() as its debug-only
row-range bound, one row looser than set()'s correct
size_local()+num_ghosts(); aligned add() to match, closing a
theoretical one-row out-of-bounds read for row == num_all_rows() in
Developer builds.

Testing: cpp/test full suite (27476 assertions) and python/test
unit/la + unit/fem (2594 tests) pass unmodified.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
clang if-converts csr_position's std::lower_bound into a branchless
cmov chain, ~35-50% slower here than GCC's branchy codegen (the cmov
serializes each iteration's load behind the prior comparison, killing
speculation on this short, cache-resident row). Confirmed via a
standalone reproducer and disassembly; a hand-rolled loop with
__builtin_expect recovers full speed, but reimplementing lower_bound
over a compiler codegen quirk isn't worth it, so left as-is.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
insert_csr and insert_blocked_csr took BS0/BS1 as non-type template
parameters (always compile-time), while insert_nonblocked_csr took
bs0/bs1 as plain runtime int parameters (always via std::div). This
put block-size handling on two different, inconsistent footings within
the same file, even though the BlockSizeArg concept (a runtime
integral or a std::integral_constant<U, N>) already existed here for
spmv/spmvT.

Convert all three insert_* functions to take BlockSizeArg auto
parameters, matching spmv/spmvT's existing convention, with
decltype(+BS0)-typed loop counters. insert_nonblocked_csr's std::div
calls are replaced with plain /, % (std::div requires concrete
arithmetic types, incompatible with a generic BlockSizeArg auto
parameter). MatrixCSR::set/add's public template<int BS0, int BS1> API
is unchanged; they now pass std::integral_constant<int, BS0/BS1>{}
into insert_csr/insert_blocked_csr instead of explicit template
arguments.

Verified standalone (local MPI toolchain is currently broken in this
environment, unrelated to this change): built matrix_csr_impl.h with
both GCC 16 and Apple clang under -std=c++20 -Wall -Wextra
-Wsign-compare, and cross-checked all three insertion functions
(compile-time and runtime block-size instantiations) against an
independently-constructed dense reference matrix -- all results match.
clang-format --dry-run --Werror passes.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@jhale

jhale commented Aug 31, 2026

Copy link
Copy Markdown
Member

"a drive-by bug" - nice to see Claude getting into stopping organised crime.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants