Repository navigation
Description
Several vectorizable loops in the writer duplicate or belong next to the existing SimdOperations kernels (widenInto, narrowInto, maxUnsigned, ...). Moving them gives one place for a future Vector API implementation (ADR 0005, #483, #484) and removes copies.
Candidates (by value)
-
minMax:writer/.../encode/ArrayStats.minMax(~70 lines, NEON-tuned in Compressor stats: per-value cost above Rust's (float top-frequency, int widening) #476: int-domain loops, U32 sign-bit flip, U8/U16 masking). Runs on every write. AddminMax(Object data, PType ptype)next tomaxUnsigned. Also makePrimitiveEncodingEncoder.minMaxStats(naive branchy loop, used byZoneMapStatCodec.columnMinMaxand the masked/dict zone-map paths) call it, so there is one unsigned-handling implementation. Floats need their own kernel (NaN / -0.0 semantics). -
widen:ArrayStats.widenduplicatesSimdOperations.widenInto, except the source is a heap array. TryMemorySegment.ofArray(arr)+widenIntofirst (no new API); fall back to a heap-array overload if JMH shows C2 does not vectorize it. -
allEqual(all values equal, raw-bits compare for floats):ConstantEncodingEncoder.constantBits(primitives)ConstantEncodingEncoder.isConstantBoolandMaskedEncodingEncoder.isConstantValidity(identical copies)AlpRdEncodingEncoder.allEqual(short[], re-readsvalues[0]every iteration)
Early-exit loops do not auto-vectorize, so this is mostly dedup today; the SIMD payoff comes with the Vector API implementation. The bool/validity pair can be merged regardless.
-
runs(ArrayStats, count of adjacent changes): only one caller, lowest priority.
Not candidates: the per-element unsigned masking in Dict*Array / BitpackedEncodingDecoder (accessors, not bulk loops).
Constraints
- Keep the ptype switch outside the loop (hot-loop rule).
- Output must stay byte-identical (zone-map and interop tests cover it).
- Dispatch is an interface call via
VectorSupport.operations(): benchmark small inputs (per-zone) against the inlined static versions, and A/B on all real files (RealFileScanBenchmark/ write benchmarks), per past experience that 4 files hid a regression. - CHANGELOG/docs in the same commit if user-visible.
Activity
dfa1 commentedon Oct 10, 2026
Addendum after reading
core.compute.PrimitiveArrays:widenhas three copies, not two:SimdOperations.widenInto(segment source),ArrayStats.widen(writer, heap array) andPrimitiveArrays.toLongs(Object, PType, EncodingId)(heap array, allocates). All three should end up on one kernel.PrimitiveArrays.compact(one filter loop per width) and thefromLongs/fromLongsArray/fromBitsArray/fromIntsArraynarrowing loops are further kernel candidates with no SIMD entry point today.EncodingId, noObjectreturn, caller-supplied output,IllegalArgumentExceptiononly).PrimitiveArraysstays the thin validating layer that attributesVortexExceptionand boxes results, astoLongsIntoalready does by delegating towidenInto.Naming (brainstorm, undecided):
SimdOperations/ScalarOperations/VectorSupportare misleading (scalar impl behind a "Simd" name;VectorSupportis a selector, not a probe). Candidates:BufferKernels/ScalarKernels/VectorKernelswithBufferKernels.instance(). PlainKernelsis avoided because Rust uses "kernel" for encoding-levelExecuteParentKernel/FilterKernel; Rust's buffer-level analogues arevortex-compute::lane_kernelsandvortex-buffer::CpuKernel. Do the rename as its own commit first.dfa1 commentedon Oct 10, 2026
Naming decision (supersedes the undecided naming note above). The current
SimdOperations/ScalarOperations/VectorSupporttrio was inherited from Hardwood (dev.hardwood.internal.encoding.simd), but "Simd" is misleading when the default is scalar, and "Scalar" collides with Vortex's scalar-value vocabulary (ProtoScalarValue). PlainKernelscollides with Rust's encoding-levelExecuteParentKernel;Array*collides withreader.array.Array. Every method takes aPType, matchingPTypeIO:PTypeOperationsScalarPTypeOperationsVectorPTypeOperationsPTypeOperationsSupport— onlyoperations();isAvailable()/implementationName()droppedcore.simd→core.ptypeThe selector stays a standalone class (not a static factory on the interface) so a Java 22 multi-release overlay can replace it, as Hardwood does. "Factory" rejected: it returns one selected instance, not a new one. Do the rename as its own commit before the loop migration.
dfa1 commentedon Oct 10, 2026
Status on branch
refactor/516-simd-naming(7 commits, not merged):ScalarOperations->AutoVectorizedSimdOperations,VectorSupport->SimdOperationsSupportwithpreferred(); package stayscore.simd.SimdOperationsas pure moves:minMax(signed order over the widened values, so U64 stays signed),widenArrayInto,narrowArrayInto,allEqual(raw bits for floats, plus aboolean[]overload),sum/sumFloating,runs.sumoverflow-checks every addition in order (Rust drops the sum on any overflowing partial sum); float sums stay strictly left to right;runscompares floats by value,allEqualby raw bits.PrimitiveArrays.compact(a filter, different shape) andfromBitsArray's F32/F64 bit reinterpretation; the per-element accessors inDict*Array/BitpackedEncodingDecoder; float min/max (NaN semantics).sum(a separate, measured change).build: compile and document the Vector API class on JDK 26 (#516)
ci: run the SIMD benchmark with one fork by default (#516)
perf: sum narrow integers without converting lane widths (#516)
ci: pin the x86 vector width so benchmark runs are comparable (#516)
perf: sum ints by splitting 64-bit lanes (experiment, measured in CI) (…
Revert "perf: sum ints by splitting 64-bit lanes (experiment, measure…
refactor: rename the SIMD kernel classes (#516)
refactor: one SimdOperations kernel per primitive-array loop, for all…
feat: an opt-in Vector API implementation of the SIMD kernels (#516)
ci: benchmark the SIMD kernels on x86, ARM and Apple silicon (#516)
refactor: rename the SIMD kernel classes (#516)
refactor: one SimdOperations kernel per primitive-array loop, for all…
feat: an opt-in Vector API implementation of the SIMD kernels (#516)
ci: benchmark the SIMD kernels on x86, ARM and Apple silicon (#516)
build: run the tests without jdk.incubator.vector for now (#516)
refactor: rename the SIMD kernel classes (#516)
refactor: one SimdOperations kernel per primitive-array loop, for all…
feat: an opt-in Vector API implementation of the SIMD kernels (#516)
ci: benchmark the SIMD kernels on x86, ARM and Apple silicon (#516)
build: run the tests without jdk.incubator.vector for now (#516)
dfa1 commentedon Oct 11, 2026
Done in #524 (merged as 5 commits on main): the loops live in
SimdOperationsfor all eleven ptypes, plus an opt-inVectorApiSimdOperations. The test JVMs run withoutjdk.incubator.vectorfor now; testing both configurations is tracked in #529.