Skip to content

Add nullable UnionArray#8792

Draft
connortsui20 wants to merge 5 commits into
agent/union-scalar-nullabilityfrom
agent/union-array-nullability
Draft

Add nullable UnionArray#8792
connortsui20 wants to merge 5 commits into
agent/union-scalar-nullabilityfrom
agent/union-array-nullability

Conversation

@connortsui20

@connortsui20 connortsui20 commented Jul 16, 2026

Copy link
Copy Markdown
Member

Tracking issue: #8769

Implements the array portion of #8769 on top of #8791.

  • adds the canonical sparse UnionArray
  • stores outer union validity exclusively in the nullable u8 type IDs child
  • treats child validity as inner/selected-value nullability, not outer validity
  • ignores the type-ID payload at outer-null rows during validation
  • preserves outer and inner nulls through scalar_at, serde, constants, chunking, filtering, slicing, masking, recursive canonicalization, and compression
  • recursively builds valid defaults for unselected children and returns an error for genuinely uninhabited schemas
  • canonicalizes empty unions without inventing placeholder values
  • supports non-nullable-index take; nullable-index take remains an explicit unsupported error for now

@connortsui20
connortsui20 force-pushed the agent/union-scalar-nullability branch from a05b340 to bb2292d Compare July 16, 2026 15:10
@connortsui20
connortsui20 force-pushed the agent/union-array-nullability branch from 30dd491 to 4aa2973 Compare July 16, 2026 15:42
@connortsui20
connortsui20 force-pushed the agent/union-scalar-nullability branch from bb2292d to 6c0a2ea Compare July 16, 2026 15:47
@connortsui20
connortsui20 force-pushed the agent/union-array-nullability branch 2 times, most recently from 4bdd331 to 2e6b711 Compare July 16, 2026 16:02
@connortsui20
connortsui20 force-pushed the agent/union-scalar-nullability branch from 6c0a2ea to 056c755 Compare July 16, 2026 16:02
@connortsui20
connortsui20 force-pushed the agent/union-array-nullability branch from 2e6b711 to 79645be Compare July 16, 2026 16:08
@connortsui20
connortsui20 force-pushed the agent/union-scalar-nullability branch 2 times, most recently from 877e544 to af088ad Compare July 16, 2026 17:23
@connortsui20
connortsui20 force-pushed the agent/union-array-nullability branch from 79645be to 76048bd Compare July 16, 2026 17:23
@connortsui20
connortsui20 changed the base branch from agent/union-scalar-nullability to agent/union-dtype-nullability July 16, 2026 17:24
@connortsui20
connortsui20 force-pushed the agent/union-dtype-nullability branch from e035b08 to c5f26f5 Compare July 16, 2026 17:36
@connortsui20
connortsui20 force-pushed the agent/union-array-nullability branch from 76048bd to 39677cc Compare July 16, 2026 17:36
Base automatically changed from agent/union-dtype-nullability to agent/union-scalar-nullability July 16, 2026 17:36
@connortsui20 connortsui20 added the changelog/feature A new feature label Jul 16, 2026
@connortsui20
connortsui20 force-pushed the agent/union-scalar-nullability branch from 3a22a55 to 50d65b5 Compare July 16, 2026 17:43
@connortsui20
connortsui20 force-pushed the agent/union-array-nullability branch from 39677cc to 19c9269 Compare July 16, 2026 17:43
@codspeed-hq

codspeed-hq Bot commented Jul 16, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 1670 untouched benchmarks
⏩ 52 skipped benchmarks1


Comparing agent/union-array-nullability (40479b3) with agent/union-scalar-nullability (d5458d2)

Open in CodSpeed

Footnotes

  1. 52 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@connortsui20
connortsui20 force-pushed the agent/union-scalar-nullability branch from 50d65b5 to 0e2816c Compare July 16, 2026 18:54
@connortsui20
connortsui20 force-pushed the agent/union-array-nullability branch from 19c9269 to 3dccb32 Compare July 16, 2026 18:54
@connortsui20
connortsui20 force-pushed the agent/union-scalar-nullability branch from 0e2816c to 5b5d8c0 Compare July 16, 2026 18:57
@connortsui20
connortsui20 force-pushed the agent/union-array-nullability branch from 3dccb32 to c24fce2 Compare July 16, 2026 18:57
@connortsui20
connortsui20 force-pushed the agent/union-scalar-nullability branch from 5b5d8c0 to 39be7e2 Compare July 16, 2026 20:01
@connortsui20
connortsui20 force-pushed the agent/union-array-nullability branch from c24fce2 to 0015a47 Compare July 16, 2026 20:03
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
@connortsui20
connortsui20 force-pushed the agent/union-scalar-nullability branch from 39be7e2 to d5458d2 Compare July 16, 2026 20:23
@connortsui20
connortsui20 force-pushed the agent/union-array-nullability branch from 0015a47 to 40479b3 Compare July 16, 2026 20:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/feature A new feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant