Skip to content

fix(spark): reject MapType instead of emitting a childless Arrow List#8811

Open
jackylee-ch wants to merge 1 commit into
vortex-data:developfrom
jackylee-ch:spark-schema-map-fail-fast
Open

fix(spark): reject MapType instead of emitting a childless Arrow List#8811
jackylee-ch wants to merge 1 commit into
vortex-data:developfrom
jackylee-ch:spark-schema-map-fail-fast

Conversation

@jackylee-ch

Copy link
Copy Markdown
Contributor

SparkToArrowSchema mapped MapType to a childless ArrowType.List, which is an invalid Arrow list (a List must have exactly one child). Drop the branch so MapType falls through to the existing fail-fast UnsupportedOperationException, and add a test.

SparkToArrowSchema mapped MapType to a childless ArrowType.List, which is
an invalid Arrow list (List requires exactly one child). Drop the branch so
MapType falls through to the fail-fast UnsupportedOperationException, and
add a test.

Signed-off-by: jackylee <qcsd2011@gmail.com>
@jackylee-ch jackylee-ch reopened this Jul 17, 2026
@codspeed-hq

codspeed-hq Bot commented Jul 17, 2026

Copy link
Copy Markdown

Merging this PR will improve performance by 11.17%

⚡ 1 improved benchmark
✅ 1669 untouched benchmarks
⏩ 52 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation true_count_vortex_buffer[128] 580.6 ns 522.2 ns +11.17%

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.


Comparing jackylee-ch:spark-schema-map-fail-fast (1e47bbc) with develop (3975895)

Open in CodSpeed

Footnotes

  1. 52 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant