You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Add store methods for writing into an existing buffer #3831
I think this is not the best way to support a pattern where the caller wants to use a specific memory type. IMO it would be better if the caller was responsible to creating the buffer, and they pass a reference to that buffer to the store.
We should do this with a get_into method with the following signature:
GetResult here is a data structure that conveys useful information about the IO operation. It would convey how much of the buffer was filled, whether the result was truncated, etc.
In concrete terms, I don't want to change the store ABC much, but I do think we could incrementally evolve our storage layer by increasingly relying on protocols. See #3758.
codecs should probably also have encode_into and decode_into, that follow the same pattern.
This is directly relevant for zarrs style unordered-via-threading performance/behavior i.e., the ShardingCodec can encode directly to an output buffer without caring about its order
Activity
d-v-b commentedon Mar 25, 2026
codecs should probably also have
encode_intoanddecode_into, that follow the same pattern.ilan-gold commentedon Mar 27, 2026
This is directly relevant for
zarrsstyleunordered-via-threading performance/behavior i.e., theShardingCodeccanencodedirectly to an output buffer without caring about its order