Repository navigation
[cpyrt] Parse buffer formats properly instead of character search - #140
Merged
guitargeek merged 1 commit intoOct 19, 2026
Merged
Conversation
guitargeek
force-pushed
the
buffer-format-compat
branch
3 times, most recently
from
October 7, 2026 15:57
06c6385 to
eae190d
Compare
aaronj0
previously approved these changes
Oct 19, 2026
aaronj0
left a comment
Collaborator
There was a problem hiding this comment.
LGTM! This is a nice improvement over the previous heuristic approach and now supports all formats under the PEP 3118 as shown in the test. Can you fix the merge conflicts?
numpy and other buffer producers canonicalize the PEP 3118 format char
of 64-bit integer dtypes (e.g. int64 is always reported as 'l' on 64-bit
platforms), but C++ parameters declared as int64_t resolve to 'long long'
on macOS and 'long' on Linux. The exact format-char matching in the
array converters therefore rejected valid buffers when the resolved type
and the canonicalized buffer format differed, breaking e.g. passing
numpy int64 arrays to 'const int64_t*' parameters on macOS.
Replace the ad-hoc buffer-format matching with a small PEP 3118 parser
in Utility::BufferFormatCompatible(). Formats are reduced to their
element type (skipping byte-order prefixes like ctypes' '<q', repeat
counts, and the 'Z' complex marker), classified by kind
(signed/unsigned integer, float, complex, ...) and native size, and
accepted when both match the requested type code. Use it where buffer
formats are matched: Utility::GetBuffer() (standard buffer protocol),
CArraySetArg() (low-level views), the 1- and 2-dimensional branches of
the array converter, and StdSpanConverter (std::span parameters).
This subsumes all previously scattered special cases:
- int/long equivalence on Windows and 32-bit Linux in GetBuffer()
- long/long long interchange for numpy's canonicalized int64 format
- 'z'/'Zf' complex-float naming
- signed char ('b') as bool, kept as an explicit documented quirk
As side effects, ctypes' explicitly-endian '<q'/'<d' buffers are now
accepted natively, and complex buffers no longer leak into float
parameters through the 'Zd'-contains-'d' substring accident (std::span,
which performs no item-size check, previously read complex128 buffers
as double).
guitargeek
force-pushed
the
buffer-format-compat
branch
from
October 19, 2026 12:07
eae190d to
620bad0
Compare
Collaborator
Author
|
Merge conflicts fixed! Could you re-review? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
numpy and other buffer producers canonicalize the PEP 3118 format char of 64-bit integer dtypes (e.g. int64 is always reported as 'l' on 64-bit platforms), but C++ parameters declared as int64_t resolve to 'long long' on macOS and 'long' on Linux. The exact format-char matching in the array converters therefore rejected valid buffers when the resolved type and the canonicalized buffer format differed, breaking e.g. passing numpy int64 arrays to 'const int64_t*' parameters on macOS.
Replace the ad-hoc buffer-format matching with a small PEP 3118 parser in Utility::BufferFormatCompatible(). Formats are reduced to their element type (skipping byte-order prefixes like ctypes' '<q', repeat counts, and the 'Z' complex marker), classified by kind (signed/unsigned integer, float, complex, ...) and native size, and accepted when both match the requested type code. Use it where buffer formats are matched: Utility::GetBuffer() (standard buffer protocol), CArraySetArg() (low-level views), the 1- and 2-dimensional branches of the array converter, and StdSpanConverter (std::span parameters).
This subsumes all previously scattered special cases:
As side effects, ctypes' explicitly-endian '<q'/'<d' buffers are now accepted natively, and complex buffers no longer leak into float parameters through the 'Zd'-contains-'d' substring accident (std::span, which performs no item-size check, previously read complex128 buffers as double).