Skip to content

GH-51037: [C++] Replace RapidJSON with simdjson in JSON parser - #51038

Open
Reranko05 wants to merge 6 commits into
apache:mainfrom
Reranko05:gh-35460-parser
Open

GH-51037: [C++] Replace RapidJSON with simdjson in JSON parser#51038
Reranko05 wants to merge 6 commits into
apache:mainfrom
Reranko05:gh-35460-parser

Conversation

@Reranko05

@Reranko05 Reranko05 commented Aug 29, 2026

Copy link
Copy Markdown
Collaborator

Rationale for this change

This PR continues the simdjson migration by replacing the RapidJSON-based parsing implementation used by the JSON parser.

The existing parser uses RapidJSON's SAX/handler interface to parse JSON values and populate Arrow builders. This change replaces that implementation with simdjson's ondemand API while retaining the existing builder and type-inference logic.

Changes

  • Replace the RapidJSON parser and handler interface with simdjson's ondemand API.
  • Parse JSON documents using simdjson::ondemand::parser::iterate_many.
  • Use ResolveSimdjsonResult() consistently when handling simdjson results.
  • Preserve support for nested objects and arrays.
  • Preserve explicit-schema and inferred-field behavior.
  • Preserve unexpected-field handling for Error, Ignore, and InferType.
  • Continue storing numeric values as raw JSON tokens.
  • Trim trailing whitespace from numeric raw tokens to preserve existing behavior.
  • Preserve JSON parse error propagation through Status::Invalid.
  • Remove the parser's RapidJSON-specific dependencies.

Fixes: #51037

@Reranko05 Reranko05 added the CI: Extra: C++ Run extra C++ CI label Aug 29, 2026
@Reranko05
Reranko05 force-pushed the gh-35460-parser branch 2 times, most recently from 733788e to d02c90a Compare August 29, 2026 21:09
@Reranko05
Reranko05 marked this pull request as ready for review August 30, 2026 06:34
Copilot AI lite review requested due to automatic review settings August 30, 2026 06:34
@Reranko05
Reranko05 requested review from pitrou and rok as code owners August 30, 2026 06:34

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[C++] Replace RapidJSON with simdjson in JSON parser

2 participants