Skip to content

GH-50816: [C++][IPC] Improve sparse tensor index validation - #50817

Open
rok wants to merge 9 commits into
apache:mainfrom
rok:cpp-ipc-validate-sparse-tensor-indices
Open

rok wants to merge 9 commits into
apache:mainfrom
rok:cpp-ipc-validate-sparse-tensor-indices

Conversation

@rok

@rok rok commented Aug 5, 2026 •

Copy link
Copy Markdown
Member

Rationale for this change

Invalid sparse tensor indices should be rejected consistently before sparse-to-dense conversion.

What changes are included in this PR?

  • Validate sparse index bounds and index tensor buffers.
  • Validate compressed pointer consistency.
  • Validate CSF axis ordering.
  • Check sparse values and dense tensor sizes before conversion.
  • Keep CSF IPC index buffer sizing consistent with the index element width.

Are these changes tested?

Regression tests were added for invalid CSF axis ordering and out-of-range CSR and CSF indices. C++ formatting and lint checks pass.

Are there any user-facing changes?

Invalid sparse tensor inputs that were previously accepted are now rejected with an Invalid status.

@rok
rok force-pushed the cpp-ipc-validate-sparse-tensor-indices branch from 473cec3 to 83eba91 Compare November 2, 2026 18:43
@rok
rok marked this pull request as ready for review November 2, 2026 20:12
@rok
rok requested review from pitrou and a balanced review from Copilot November 2, 2026 20:12
@rok

rok commented Nov 2, 2026

Copy link
Copy Markdown
Member Author

@pitrou here's a proposal for several sparse index checkers.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Sparse tensor validation can read past the buffer because the index and indptr helpers assume packed storage without rejecting non-contiguous vectors.

1 open issue
What changed in this PR

This PR strengthens C++ sparse tensor validation for GH-50816, rejecting malformed indices before dense conversion and correcting CSF index buffer sizing in IPC.

Changes:

  • Validate index bounds, buffers, compressed pointers, axis ordering, and conversion sizes.
  • Respect COO coordinate strides and encode CSF IPC buffer lengths in bytes.
  • Add regression tests for malformed inputs and IPC round trips.
File Description
cpp/​src/​arrow/​tensor/​coo_converter.cc Respect coordinate strides during dense conversion.
cpp/​src/​arrow/​tensor.h Declare the shared permutation validator.
cpp/​src/​arrow/​tensor.cc Implement the shared permutation validator.
cpp/​src/​arrow/​tensor_test.cc Test valid and invalid permutations.
cpp/​src/​arrow/​sparse_tensor.h Declare and connect sparse index validation.
cpp/​src/​arrow/​sparse_tensor.cc Validate sparse indices, buffers, and conversion sizes.
cpp/​src/​arrow/​sparse_tensor_test.cc Add sparse validation and conversion regressions.
cpp/​src/​arrow/​ipc/​tensor_test.cc Test CSF buffer sizing and invalid IPC inputs.
cpp/​src/​arrow/​ipc/​metadata_internal.cc Correct CSF buffer lengths and validate layouts.
cpp/​src/​arrow/​extension/​tensor_internal.h Remove the relocated validator declaration.
cpp/​src/​arrow/​extension/​tensor_internal.cc Remove the relocated validator implementation.

Review effort: Balanced

Comment thread cpp/src/arrow/sparse_tensor.cc Resolved
Copilot AI balanced review requested due to automatic review settings November 2, 2026 20:17

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Maintainer review is needed to clarify compatibility expectations for previously serialized CSF messages and older readers when IPC buffer lengths change from element counts to byte counts.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants