Skip to content

HLL: reject HLL8 register values above 63 - #814

Open
jaideeppyne wants to merge 1 commit into
apache:mainfrom
jaideeppyne:fix/hll8-reject-ranks-above-63
Open

jaideeppyne wants to merge 1 commit into
apache:mainfrom
jaideeppyne:fix/hll8-reject-ranks-above-63

Conversation

@jaideeppyne

Copy link
Copy Markdown
Contributor

Problem

An HLL8 register is a rank in 0..63, stored in a byte. getSlotValue masks with VAL_MASK_6, so a byte of 64 (bit 6 set) is read as 0, an empty slot. The distinct-count estimate then drops instead of the image being rejected.

Go's deserializeHll8 and Rust's HllSketch::deserialize already reject a rank above 63. Java heapify and wrap accepted it.

Fix

Hll8Array.heapify and both DirectHll8Array constructors scan the register bytes and throw SketchesArgumentException when any value is above 63. Writers already store masked ranks, so sketches produced by this library are unchanged.

Tests

mvn -Dtest=HllCorruptImageTest test with JDK 25. 13 tests, 0 failures. The new case sets the last register of an HLL-mode image to 64 and checks heapify, wrap, writableWrap, and union heapify. The existing valid-image cases still match.

Readers mask each slot with six bits, so a byte of 64 is read as an empty register and the estimate drops. Heapify and wrap now reject that image. Go and Rust already do.

@leerho leerho left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Rejecting HLL8 images with a register above 63 matches Go and Rust. Valid images can't trip the check, and the promotion path clears memory before building the array. The extra scan in wrap is acceptable and follows apache/datasketches-java#799 · HLL: harden checks of coupon…'s SET-mode check. Folding the two checkSlots loops into one helper would be a nice cleanup but isn't required. If you could do that then I'll merge it.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants