Resolve bitpacking functions once for bitwidth - #9721
Conversation
BitPacked arrays are type erased, so decoding went through the fastlanes `unchecked_*` entry points, which re-dispatch on the runtime bit width with a 65-arm match for every 1024-value block (and for every `scalar_at`). Add `BitPackedKernels`, a set of function pointers to the const-width kernel instantiations (`unpack`, `unpack_single`, `unfor_pack`), lazily resolved once per array through a `OnceLock` on `BitPackedData` and shared by every decode path: canonicalization, mapped cast, take, filter, scalar_at, is_constant, between, and the fused FoR decompress. The fused compare kernel resolves its `unpack_cmp` instantiation once per call, since it is generic over the comparison closure. The resolved kernels take slices and check block lengths, so the unsafe runtime-width calls are gone from the decoding logic. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R9mScTPzB4Vw4PCdSyc862 Signed-off-by: Claude <noreply@anthropic.com>
`bitpack_primitive` still dispatched on the runtime bit width through `unchecked_pack` for every 1024-value block. Resolve the const-width pack kernel once per call via `BitPackedPhysical::resolve_pack` instead, so no path outside kernel resolution matches on the width. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R9mScTPzB4Vw4PCdSyc862 Signed-off-by: Claude <noreply@anthropic.com>
Merging this PR will degrade performance by 14.44%
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ | WallTime | arrow_checked_add_u32_neon[16384] |
13.5 µs | 20.4 µs | -33.79% |
| ❌ | Simulation | decompress[datetime_for_bp] |
220.1 µs | 310 µs | -29.01% |
| ❌ | Simulation | new_bp_prim_test_between[i64, 2048] |
90.2 µs | 120 µs | -24.83% |
| ❌ | Simulation | new_alp_prim_test_between[f64, 2048] |
107.1 µs | 137.5 µs | -22.14% |
| ❌ | Simulation | decompress_rd[f64, (2000, 0.0)] |
121.9 µs | 152.6 µs | -20.13% |
| ❌ | Simulation | decompress_rd[f64, (2000, 0.1)] |
121.9 µs | 152.6 µs | -20.13% |
| ❌ | Simulation | decompress_rd[f64, (2000, 0.01)] |
121.9 µs | 152.6 µs | -20.09% |
| ❌ | Simulation | decompress[alp_for_bp_f64] |
145 µs | 177 µs | -18.08% |
| ❌ | WallTime | arrow_checked_add_u32_avx2[16384] |
17.7 µs | 21.4 µs | -17.34% |
| ❌ | Simulation | alp_rd_decompress_f64 |
169.6 µs | 200.7 µs | -15.51% |
| ❌ | Simulation | new_bp_prim_test_between[i64, 16384] |
170.8 µs | 201.3 µs | -15.12% |
| ❌ | Simulation | new_alp_prim_test_between[f64, 16384] |
193.9 µs | 224.8 µs | -13.73% |
| ❌ | Simulation | compare[1] |
206.8 µs | 237.9 µs | -13.09% |
| ❌ | Simulation | compare[2] |
212.3 µs | 243.2 µs | -12.71% |
| ❌ | Simulation | compare[2] |
215.8 µs | 246.4 µs | -12.44% |
| ❌ | Simulation | compare[1] |
210.8 µs | 240.4 µs | -12.34% |
| ❌ | Simulation | compare[3] |
219.7 µs | 250.4 µs | -12.26% |
| ❌ | Simulation | compare[3] |
223.1 µs | 254 µs | -12.16% |
| ❌ | Simulation | compare[4] |
225.5 µs | 256.2 µs | -11.98% |
| ❌ | Simulation | compare[4] |
228.7 µs | 259.5 µs | -11.85% |
| ... | ... | ... | ... | ... | ... |
ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing claude/bitpacked-fastlanes-function-pointers-ckpnee (117a5ce) with develop (50cd6d7)
Footnotes
-
206 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
-
4 benchmarks were run, but are now archived. If they were deleted in another branch, consider rebasing to remove them from the report. Instead if they were added back, click here to restore them. ↩
Inspecting the release assembly showed two misses. `BitPackedData::kernels` was not inlined, so `scalar_at` paid an out-of-line call before the indirect kernel call; mark it `#[inline]`. Every kernel wrapper also carried the `vortex_panic!` formatting for a block-length mismatch inline, which reserved a stack frame on the hot path; move it into a `#[cold]` out-of-line function so the wrappers reduce to a length compare and a tail jump into the fastlanes kernel. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01R9mScTPzB4Vw4PCdSyc862 Signed-off-by: Claude <noreply@anthropic.com>
Instead of using unchecked_* variants of bitpacking functions resolve the functions once. Avoids finding the right generic function for given bitwidth on every call