Skip to content

Consolidate Zarr metadata at the node a store path addresses - #11835

Draft
jhamman wants to merge 2 commits into
pydata:mainfrom
jhamman:fix/consolidate-at-store-path
Draft

jhamman wants to merge 2 commits into
pydata:mainfrom
jhamman:fix/consolidate-at-store-path

Conversation

@jhamman

@jhamman jhamman commented Oct 31, 2026

Copy link
Copy Markdown
Member

Description

Writing with consolidated=True to a store that addresses a node below the store root, such as zarr.storage.StorePath(store, "grp"), only consolidated metadata at the store root. Reading the same store back looks for consolidated metadata at grp, so it fell back to non-consolidated metadata with a RuntimeWarning, or raised a ValueError with consolidated=True. This happens with both zarr formats 2 and 3.

import numpy as np
import xarray as xr
import zarr

store = zarr.storage.MemoryStore()
xr.Dataset({"a": ("x", np.arange(3))}).to_zarr(
    zarr.storage.StorePath(store, "grp"), consolidated=True
)
xr.open_zarr(zarr.storage.StorePath(store, "grp"))  # RuntimeWarning before this PR

Now xarray also consolidates at the node the store addresses: the group's path, minus any group=. It still consolidates at the store root as before, so reading with open_zarr(store, group="grp") keeps working.

Checklist

  • Tests added
  • User visible changes (including notable bug fixes) are documented in whats-new.rst

AI Disclosure

  • This PR contains AI-generated content.
    • I have tested any AI-generated content in my PR.
    • I take responsibility for any AI-generated content in my PR.
      Tools: Claude Code

[This is Claude Code on behalf of Joe Hamman]

🤖 Generated with Claude Code

jhamman and others added 2 commits October 31, 2026 08:58
When writing with consolidated=True to a store that addresses a node below
the root of the underlying store (e.g. zarr.storage.StorePath(store, "grp")),
also consolidate metadata at that node. Reading the same store looks for
consolidated metadata there; before, it only existed at the store root, so
reading fell back to non-consolidated metadata with a RuntimeWarning, or
raised with consolidated=True.

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Claude <noreply@anthropic.com>
@github-actions github-actions Bot added topic-backends topic-zarr Related to zarr storage library io labels Oct 31, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

io topic-backends topic-zarr Related to zarr storage library

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant