[connortsui20]

Summary

• Tracking Issue: Epic: Row-oriented scalar functions #9128
• Independent validity candidate: Attach RowFn validity directly to canonical output #10742

Routes float32 comparisons through the existing word-packing collectors. This recovers the measured float32 gains from the RowFn comparison investigation with two private dispatch changes. It is independent of the comparison migration and fusion work in #10676 and #10675.

Changes

Adds F32 to the existing collector selections for array pairs and array/constant comparisons. Integer collector policies remain unchanged. The floating-point documentation now accounts for NaN signs and payloads.

Draft pending fresh isolated benchmarks on this head. Saved measurements show a consistent float32 gain, but also some unrelated ARM regressions. The later combined packing/validity candidate had an AVX2 regression that has not been attributed to either change individually.

Saved benchmark evidence and validation

The isolated packing snapshot 4c98df49e379853f8e990d9764791b5330511771 was compared with frozen develop at a1693389301e446eec41373cb274d98a303d3ca0. Measurements used Rust 1.99.1 / LLVM 23.1.1, 16 CGUs, no LTO, CPU 2, and 21 balanced paired blocks on Graviton3 C7g and Sapphire Rapids C7i. This was native wall time, not CodSpeed simulation.

Across 48 float32 cases at 8,192 and 65,536 rows, covering array pairs, both constant orientations, equality/ordering, and four validity shapes, the median block ratios of equally weighted geometric means were:

Target	Candidate / baseline time	95% paired bootstrap interval
NEON	0.611	0.610–0.611
AVX2	0.632	0.630–0.633
AVX-512	0.679	0.678–0.681

Among all 375 probe cases, the isolated snapshot had no flagged regressions on AVX2 or AVX-512. ARM had four, including roughly 4.7% slower u64 multiply and 4.6% slower large f64 equality. Those paths do not use the changed float32 collector, so their movement is not evidence of extra float32 work, but it still counts against a no-regression claim.

A separate combined packing/validity candidate at 8d8ef8151454bbc73e1cd0b236567542a1401cbe, based on d834a11faa79370b4d2e0bbcc73195f451a8a212, retained the float32 gain but regressed large nullable u64 add by about 18% on AVX2. Profiling found a byte-identical retry hot loop with essentially unchanged instructions per operation and more cycles. The precise placement-sensitive mechanism remains unresolved. That combined result does not isolate this PR.

The combined candidate passed 225 RowFn tests and 232 binary-function tests on that newer base. Separate native probes checked scalar-reference results and float/encoding/tail edge cases. The changed file passed the pinned formatter. These are saved checks, not fresh validation of this individual PR.

This PR is rebased onto c750fbd80882f78efd6a2703bc0f3e17f1aca9f9. The patch is unchanged by the rebase. Tests and benchmarks were not rerun after it.

[codspeed]

Merging this PR will improve performance by 20.81%

4 warnings, 1 note

⚠️ Walltime results may be unreliable

Some benchmarks used the Walltime instrument outside CodSpeed Macro Runners, which can make the results inconsistent.

Learn more about Macro Runners

See affected benchmarks

⚠️ Unable to measure the execution of 2 benchmarks

These benchmarks measured no execution time, so their results are not comparable and count as unchanged. This usually happens when the compiler removes the code under test.

Prevent compiler optimizations

See affected benchmarks

⚠️ 20 benchmarks are dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

⚠️ Base and head ran on different runner types

Some benchmarks with significant performance changes were compared across different runner types, so the changes might not come from the code. Run base and head on the same runner type, or use CodSpeed Macro Runners for more stable results.

Learn more about Macro Runners

Open the report in CodSpeed to investigate

ℹ️ 40 benchmarks have no flamegraph

The flamegraph wasn't generated: the symbols of the profiling trace could not be resolved, because no information about the language frames was collected.

See affected benchmarks

⚡ 11 improved benchmarks
✅ 2397 untouched benchmarks
⏩ 12 skipped benchmarks¹

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
⚡	Walltime	compare_f32_avx2	4.4 µs	2.9 µs	+51.76%
⚡	Walltime	compare_f32_avx512	3.4 µs	2.4 µs	+43.97%
⚡	Walltime	compare_f32_neon	5.2 µs	4.2 µs	+24.21%
⚡	Walltime	filter_neon[8]	1,115 ns	917 ns	+21.59%
⚡	CPU Simulation	test_repeated_scan_scalar_at[int]	280.2 µs	233.7 µs	+19.89%
⚡	CPU Simulation	test_scan_filter_to_arrow_array[int_float]	4 ms	3.5 ms	+16%
⚡	CPU Simulation	test_repeated_scan_filter[int_float]	3.9 ms	3.4 ms	+13.65%
⚡	Walltime	filter_neon[64]	2 µs	1.8 µs	+11.94%
⚡	CPU Simulation	test_repeated_scan_filter[int]	2.7 ms	2.4 ms	+11.19%
⚡	Walltime	filter_neon[80]	2.7 µs	2.4 µs	+11.02%
⚡	CPU Simulation	test_scan_filter_to_arrow_array[int]	2.8 ms	2.5 ms	+11.01%

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing ct/row-fn-cleanup/f32-packing (bc683bb) with develop (f3cbb32)}

Footnotes

1. 12 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[robert3005]

This is part of #10675