Deduplicate CSR column search in MatrixCSR - #4461
Draft
garth-wells wants to merge 3 commits into
Draft
Conversation
insert_csr, insert_blocked_csr, insert_nonblocked_csr, and MatrixCSR's ghost-row unpacking each duplicated the same std::lower_bound-based column search. Factored into a shared la::impl::csr_position() helper. A hand-rolled "branchless" (conditional-move) rewrite of this search was tried first, on the theory that the column found is unpredictable from one call to the next so a branchy std::lower_bound mispredicts. It measured as a reproducible ~13% regression in scripts/bench_backend.py's csr_reasm (P1, N=100): GCC 13 did not turn the idiom into a cmov, confirmed by disassembly -- it kept a real conditional branch, structurally near-identical to libstdc++'s own lower_bound, just without libstdc++'s tuning. Reverted to a thin wrapper around std::lower_bound instead; the deduplication alone measured a genuine, reproducible +6.2% on P1/N=100 csr_reasm (no significant change on P3/N=64, where kernel cost dominates insertion cost). The failed branchless attempt is documented in a code comment so it isn't re-tried without new evidence. Drive-by: MatrixCSR::add() passed _row_ptr.size() as its debug-only row-range bound, one row looser than set()'s correct size_local()+num_ghosts(); aligned add() to match, closing a theoretical one-row out-of-bounds read for row == num_all_rows() in Developer builds. Testing: cpp/test full suite (27476 assertions) and python/test unit/la + unit/fem (2594 tests) pass unmodified. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
clang if-converts csr_position's std::lower_bound into a branchless cmov chain, ~35-50% slower here than GCC's branchy codegen (the cmov serializes each iteration's load behind the prior comparison, killing speculation on this short, cache-resident row). Confirmed via a standalone reproducer and disassembly; a hand-rolled loop with __builtin_expect recovers full speed, but reimplementing lower_bound over a compiler codegen quirk isn't worth it, so left as-is. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
insert_csr and insert_blocked_csr took BS0/BS1 as non-type template
parameters (always compile-time), while insert_nonblocked_csr took
bs0/bs1 as plain runtime int parameters (always via std::div). This
put block-size handling on two different, inconsistent footings within
the same file, even though the BlockSizeArg concept (a runtime
integral or a std::integral_constant<U, N>) already existed here for
spmv/spmvT.
Convert all three insert_* functions to take BlockSizeArg auto
parameters, matching spmv/spmvT's existing convention, with
decltype(+BS0)-typed loop counters. insert_nonblocked_csr's std::div
calls are replaced with plain /, % (std::div requires concrete
arithmetic types, incompatible with a generic BlockSizeArg auto
parameter). MatrixCSR::set/add's public template<int BS0, int BS1> API
is unchanged; they now pass std::integral_constant<int, BS0/BS1>{}
into insert_csr/insert_blocked_csr instead of explicit template
arguments.
Verified standalone (local MPI toolchain is currently broken in this
environment, unrelated to this change): built matrix_csr_impl.h with
both GCC 16 and Apple clang under -std=c++20 -Wall -Wextra
-Wsign-compare, and cross-checked all three insertion functions
(compile-time and runtime block-size instantiations) against an
independently-constructed dense reference matrix -- all results match.
clang-format --dry-run --Werror passes.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Member
|
"a drive-by bug" - nice to see Claude getting into stopping organised crime. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Split out from #4438:
insert_csr,insert_blocked_csr,insert_nonblocked_csr,and
MatrixCSR's ghost-row unpacking each duplicated the samestd::lower_bound-based column search. This factors it into a sharedla::impl::csr_position()helper, and fixes a drive-by bug found whiledoing so.
This was dropped from #4438 during rebasing/scope-narrowing of that PR
(now assembly-focused only) — re-proposing standalone since it's an
independent, self-contained improvement to
la::MatrixCSR.Changes
la::impl::csr_position().tried first, on the theory that the column found is unpredictable from
one call to the next so a branchy
std::lower_boundmispredicts. Itmeasured as a reproducible ~13% regression in
scripts/bench_backend.py'scsr_reasm(P1, N=100): GCC 13 did not turnthe idiom into a cmov, confirmed by disassembly — it kept a real
conditional branch, structurally near-identical to libstdc++'s own
lower_bound, just without libstdc++'s tuning. Reverted to a thinwrapper around
std::lower_boundinstead; the deduplication alonemeasured a genuine, reproducible +6.2% on P1/N=100
csr_reasm(nosignificant change on P3/N=64, where kernel cost dominates insertion
cost).
csr_position'sstd::lower_boundinto a branchlesscmov chain, ~35-50% slower here than GCC's branchy codegen (the cmov
serializes each iteration's load behind the prior comparison, killing
speculation on this short, cache-resident row). Confirmed via a
standalone reproducer and disassembly; a hand-rolled loop with
__builtin_expectrecovers full speed, but reimplementinglower_boundover a compiler codegen quirk isn't worth it, so this is left as-is and
documented in a code comment so it isn't re-tried without new evidence.
MatrixCSR::add()passed_row_ptr.size()as its debug-onlyrow-range bound, one row looser than
set()'s correctsize_local()+num_ghosts(). Alignedadd()to match, closing atheoretical one-row out-of-bounds read for
row == num_all_rows()inDeveloper builds.
Simplify block-size handling with
auto/std::integral_constantinsert_csr/insert_blocked_csrtook their block sizes as non-typetemplate parameters (
template <int BS0, int BS1, ...>, alwayscompile-time), while
insert_nonblocked_csrtook plain runtimeint bs0, int bs1(viastd::div). Converted all three to theBlockSizeArg autoconvention already used byspmv/spmvTin thissame file (a parameter that accepts either a runtime integral or a
std::integral_constant<U, N>), withdecltype(+BS0)-typed loopcounters.
insert_nonblocked_csr'sstd::divcalls become plain/,%, sincestd::divrequires concrete arithmetic types incompatiblewith a generic
autoparameter.MatrixCSR::set/add's publictemplate <int BS0, int BS1>API is unchanged; they now passstd::integral_constant<int, BS0>{}intoinsert_csr/insert_blocked_csrinstead of explicit template arguments.Testing
clang-format --dry-run --Werrorpasses on all touchedfiles.
csr_positiondedup + row-bound fix commits previously passed thefull local C++ test suite (27476 assertions) and
python/testunit/la+unit/fem(2594 tests) while part of Assembly performance optimisations #4438, before beingreverted there purely to narrow that PR's scope to assembly
performance.
the full local test suite in this environment (local MPI toolchain is
currently broken here — a Homebrew
open-mpiupgrade invalidated thecached PETSc/MPI paths — unrelated to this change). Instead verified
standalone: compiled
matrix_csr_impl.hdirectly with both GCC 16 andApple clang under
-std=c++20 -Wall -Wextra -Wsign-compare(clean, nowarnings), and cross-checked
insert_csr,insert_blocked_csr, andinsert_nonblocked_csr— each called with both a runtimeintand astd::integral_constantblock size — against an independentlyconstructed dense reference matrix; all six combinations reproduce the
same matrix.
🤖 Generated with Claude Code