From c876340fc52961dd8b420ec653c90694b42de78a Mon Sep 17 00:00:00 2001 From: Davis Vann Bennett Date: Mon, 26 Oct 2026 20:03:04 +0100 Subject: [PATCH 1/3] docs: 3.4.2 release notes Build the 3.4.2 section of the release notes with towncrier from the twelve fragments on main, join towncrier's hard-wrapped entries into single lines, and drop private attribute names from the #4483 entry. Assisted-by: ClaudeCode:claude-opus-5-5 Co-Authored-By: Claude Opus 5.5 --- changes/3850.doc.md | 1 - changes/4051.doc.md | 6 ------ changes/4166.doc.md | 1 - changes/4344.bugfix.md | 8 -------- changes/4408.bugfix.md | 1 - changes/4450.bugfix.md | 1 - changes/4483.bugfix.md | 1 - changes/4495.bugfix.md | 1 - changes/4520.doc.md | 1 - changes/4543.misc.md | 1 - changes/4547.misc.md | 1 - changes/4557.bugfix.md | 1 - docs/release-notes.md | 23 +++++++++++++++++++++++ 13 files changed, 23 insertions(+), 24 deletions(-) delete mode 100644 changes/3850.doc.md delete mode 100644 changes/4051.doc.md delete mode 100644 changes/4166.doc.md delete mode 100644 changes/4344.bugfix.md delete mode 100644 changes/4408.bugfix.md delete mode 100644 changes/4450.bugfix.md delete mode 100644 changes/4483.bugfix.md delete mode 100644 changes/4495.bugfix.md delete mode 100644 changes/4520.doc.md delete mode 100644 changes/4543.misc.md delete mode 100644 changes/4547.misc.md delete mode 100644 changes/4557.bugfix.md diff --git a/changes/3850.doc.md b/changes/3850.doc.md deleted file mode 100644 index ff5564792b..0000000000 --- a/changes/3850.doc.md +++ /dev/null @@ -1 +0,0 @@ -Document the removal of `LRUStoreCache` in the v3 migration guide, and point to the experimental `zarr.experimental.cache_store.CacheStore` as its replacement. diff --git a/changes/4051.doc.md b/changes/4051.doc.md deleted file mode 100644 index 023dff113a..0000000000 --- a/changes/4051.doc.md +++ /dev/null @@ -1,6 +0,0 @@ -Added a deprecation policy to the contributing guide. It defines when a -deprecation cycle is required for public API changes, a shared framework for -deciding whether a removal is worthwhile, the mechanics a deprecation must -follow (warning class, message contents, documentation, and changelog entry), -and a minimum deprecation period of at least 6 months and at least one minor -release before removal. diff --git a/changes/4166.doc.md b/changes/4166.doc.md deleted file mode 100644 index 32738d3635..0000000000 --- a/changes/4166.doc.md +++ /dev/null @@ -1 +0,0 @@ -Clarify how Zarr format 3 codec pipeline roles map to `filters`, `serializer`, and `compressors` when creating arrays. diff --git a/changes/4344.bugfix.md b/changes/4344.bugfix.md deleted file mode 100644 index 28e031b318..0000000000 --- a/changes/4344.bugfix.md +++ /dev/null @@ -1,8 +0,0 @@ -Resizing an array with `delete_outside_chunks=True` now generates only the -out-of-bounds chunk coordinates and deletes them with a bounded pool of -workers, instead of materializing the entire old and new grids and all -deletion tasks. This bounds auxiliary memory during shrinking and mixed -growing/shrinking resizes while preserving retained chunks. Growing, or -shrinking within the last chunk of each axis, no longer schedules any -deletion work. When a deletion fails, the remaining deletions are now -cancelled before the error is raised; previously they ran to completion. diff --git a/changes/4408.bugfix.md b/changes/4408.bugfix.md deleted file mode 100644 index 4b59c86f98..0000000000 --- a/changes/4408.bugfix.md +++ /dev/null @@ -1 +0,0 @@ -`zarr.open_array` and `zarr.open` (sync and async) now apply the `config` argument when opening an existing array. Previously `config` was only used when a new array was created, so `zarr.open_array(store, config={"read_missing_chunks": False})` silently ran an existing array with the global defaults. `config` is now a named parameter of both functions and of `Array.open`/`AsyncArray.open`. `zarr.open` raises a `TypeError` naming the problem when `config` is given for a group. diff --git a/changes/4450.bugfix.md b/changes/4450.bugfix.md deleted file mode 100644 index 4364d1a9a6..0000000000 --- a/changes/4450.bugfix.md +++ /dev/null @@ -1 +0,0 @@ -Fixed several ways `ZipStore` could lose data or fail. A store created with `mode="w"` or `mode="x"` now keeps its entries when it is used again after `close()`, `move()`, or unpickling, instead of truncating the archive. Every method now opens the archive on first use, under a lock, so `get`, `get_partial_values`, `set_if_not_exists`, and `clear` no longer raise `AttributeError` on a store that was never opened, and threads sharing a store no longer open the archive twice. An unpickled store now opens its archive on first use rather than when it is unpickled, so writing a dask array to a `ZipStore` no longer produces an unreadable file. Where reopening an archive would silently drop its entries, the store now raises instead: `RuntimeError` after a failed `close()`, `zipfile.BadZipFile` if the file is not a complete zip archive, and `io.UnsupportedOperation` for a file object that is not readable and seekable. The type of the `mode` parameter now includes `"x"`. Builds on [#3593](https://github.com/zarr-developers/zarr-python/pull/3593) by [@oelhammouchi](https://github.com/oelhammouchi). diff --git a/changes/4483.bugfix.md b/changes/4483.bugfix.md deleted file mode 100644 index 7f90482d1d..0000000000 --- a/changes/4483.bugfix.md +++ /dev/null @@ -1 +0,0 @@ -FusedCodecPipeline reads no longer hang in a process forked after a read populated the module-level codec thread pool. `_pool`, `_pool_size`, and `_pool_lock` are now reset in the child via an `os.register_at_fork` hook. Previously the child inherited an executor whose threads did not exist, so small reads were queued on a pool nothing would run. diff --git a/changes/4495.bugfix.md b/changes/4495.bugfix.md deleted file mode 100644 index ff9e45e1c2..0000000000 --- a/changes/4495.bugfix.md +++ /dev/null @@ -1 +0,0 @@ -Structured data types now default string (`S`, `U`) and raw-bytes (`V`) fields to zero bytes. Before, an array with a raw-bytes field failed to create without an explicit fill value, and string fields defaulted to `"0"`. diff --git a/changes/4520.doc.md b/changes/4520.doc.md deleted file mode 100644 index 1c3ad1a06d..0000000000 --- a/changes/4520.doc.md +++ /dev/null @@ -1 +0,0 @@ -Correct the docstrings of `zarr.save`, `zarr.save_array` and `zarr.save_group` to say the functions write to any store. Previously the docstrings stated the functions wrote to only the local file system, which was inaccurate. diff --git a/changes/4543.misc.md b/changes/4543.misc.md deleted file mode 100644 index 54be050e61..0000000000 --- a/changes/4543.misc.md +++ /dev/null @@ -1 +0,0 @@ -Fixed the Read the Docs builds of the zarr-indexing, zarr-metadata and zarr-http-server documentation sites, which failed after an unpinned `pymdown-extensions` release, and made pull request builds of those sites skip when the pull request does not change the package. diff --git a/changes/4547.misc.md b/changes/4547.misc.md deleted file mode 100644 index f98644f479..0000000000 --- a/changes/4547.misc.md +++ /dev/null @@ -1 +0,0 @@ -The big-endian (s390x) CI job now installs zarr with its dependencies resolved against `pyproject.toml` and runs `pip check`, instead of trusting whatever versions the Debian-based test image provides. The image builds a numcodecs that satisfies zarr's constraints from source where Debian's package does not. diff --git a/changes/4557.bugfix.md b/changes/4557.bugfix.md deleted file mode 100644 index b2c76744ef..0000000000 --- a/changes/4557.bugfix.md +++ /dev/null @@ -1 +0,0 @@ -The `zarr.storage` module can now be serialized with `pickle` and `cloudpickle`. Previously, code that referenced the module (for example a Dask task that used `zarr.storage.ObjectStore`) failed with `TypeError: cannot pickle 'VerboseModule' object`. diff --git a/docs/release-notes.md b/docs/release-notes.md index 562f28996e..a83fddb419 100644 --- a/docs/release-notes.md +++ b/docs/release-notes.md @@ -4,6 +4,29 @@ +## 3.4.2 (2026-10-26) + +### Bugfixes + +- Resizing an array with `delete_outside_chunks=True` now generates only the out-of-bounds chunk coordinates and deletes them with a bounded pool of workers, instead of materializing the entire old and new grids and all deletion tasks. This bounds auxiliary memory during shrinking and mixed growing/shrinking resizes while preserving retained chunks. Growing, or shrinking within the last chunk of each axis, no longer schedules any deletion work. When a deletion fails, the remaining deletions are now canceled before the error is raised; previously they ran to completion. ([#4344](https://github.com/zarr-developers/zarr-python/pull/4344)) +- `zarr.open_array` and `zarr.open` (sync and async) now apply the `config` argument when opening an existing array. Previously `config` was only used when a new array was created, so `zarr.open_array(store, config={"read_missing_chunks": False})` silently ran an existing array with the global defaults. `config` is now a named parameter of both functions and of `Array.open`/`AsyncArray.open`. `zarr.open` raises a `TypeError` naming the problem when `config` is given for a group. ([#4408](https://github.com/zarr-developers/zarr-python/pull/4408)) +- Fixed several ways `ZipStore` could lose data or fail. A store created with `mode="w"` or `mode="x"` now keeps its entries when it is used again after `close()`, `move()`, or unpickling, instead of truncating the archive. Every method now opens the archive on first use, under a lock, so `get`, `get_partial_values`, `set_if_not_exists`, and `clear` no longer raise `AttributeError` on a store that was never opened, and threads sharing a store no longer open the archive twice. An unpickled store now opens its archive on first use rather than when it is unpickled, so writing a dask array to a `ZipStore` no longer produces an unreadable file. Where reopening an archive would silently drop its entries, the store now raises instead: `RuntimeError` after a failed `close()`, `zipfile.BadZipFile` if the file is not a complete zip archive, and `io.UnsupportedOperation` for a file object that is not readable and seekable. The type of the `mode` parameter now includes `"x"`. Builds on [#3593](https://github.com/zarr-developers/zarr-python/pull/3593) by [@oelhammouchi](https://github.com/oelhammouchi). ([#4450](https://github.com/zarr-developers/zarr-python/pull/4450)) +- Reads through `FusedCodecPipeline` no longer hang in a process forked after an earlier read had started the codec thread pool. The pool is now reset in the child process after a fork. Previously the child inherited an executor whose threads did not exist, so small reads were queued on a pool nothing would run. ([#4483](https://github.com/zarr-developers/zarr-python/pull/4483)) +- Structured data types now default string (`S`, `U`) and raw-bytes (`V`) fields to zero bytes. Before, an array with a raw-bytes field failed to create without an explicit fill value, and string fields defaulted to `"0"`. ([#4495](https://github.com/zarr-developers/zarr-python/pull/4495)) +- The `zarr.storage` module can now be serialized with `pickle` and `cloudpickle`. Previously, code that referenced the module (for example a Dask task that used `zarr.storage.ObjectStore`) failed with `TypeError: cannot pickle 'VerboseModule' object`. ([#4557](https://github.com/zarr-developers/zarr-python/pull/4557)) + +### Improved Documentation + +- Document the removal of `LRUStoreCache` in the v3 migration guide, and point to the experimental `zarr.experimental.cache_store.CacheStore` as its replacement. ([#3850](https://github.com/zarr-developers/zarr-python/pull/3850)) +- Added a deprecation policy to the contributing guide. It defines when a deprecation cycle is required for public API changes, a shared framework for deciding whether a removal is worthwhile, the mechanics a deprecation must follow (warning class, message contents, documentation, and changelog entry), and a minimum deprecation period of at least 6 months and at least one minor release before removal. ([#4051](https://github.com/zarr-developers/zarr-python/pull/4051)) +- Clarify how Zarr format 3 codec pipeline roles map to `filters`, `serializer`, and `compressors` when creating arrays. ([#4166](https://github.com/zarr-developers/zarr-python/pull/4166)) +- Correct the docstrings of `zarr.save`, `zarr.save_array` and `zarr.save_group` to say the functions write to any store. Previously the docstrings stated the functions wrote to only the local file system, which was inaccurate. ([#4520](https://github.com/zarr-developers/zarr-python/pull/4520)) + +### Misc + +- [#4543](https://github.com/zarr-developers/zarr-python/pull/4543), [#4547](https://github.com/zarr-developers/zarr-python/pull/4547) + + ## 3.4.1 (2026-10-08) ### Bugfixes From c1281ae270853853172e16d548a49b5aa9cb42b6 Mon Sep 17 00:00:00 2001 From: Davis Vann Bennett Date: Mon, 26 Oct 2026 21:18:17 +0100 Subject: [PATCH 2/3] docs: add #4465 to the 3.4.2 release notes Render the two fragments #4465 added (named after issues #4404 and #4539) into the 3.4.2 section, linked to #4465, the pull request that merged them. Assisted-by: ClaudeCode:claude-opus-5-5 Co-Authored-By: Claude Opus 5.5 --- changes/4404.feature.md | 1 - changes/4539.bugfix.md | 1 - docs/release-notes.md | 5 +++++ 3 files changed, 5 insertions(+), 2 deletions(-) delete mode 100644 changes/4404.feature.md delete mode 100644 changes/4539.bugfix.md diff --git a/changes/4404.feature.md b/changes/4404.feature.md deleted file mode 100644 index cfa685edbc..0000000000 --- a/changes/4404.feature.md +++ /dev/null @@ -1 +0,0 @@ -Reading data encoded with the `pickle` codec now requires an explicit opt-in, because unpickling data from an untrusted store can execute arbitrary code ([GHSA-jqc8-qjfv-5m6r](https://github.com/zarr-developers/numcodecs/security/advisories/GHSA-jqc8-qjfv-5m6r)). Enable it with `zarr.config.set({"array.allow_unsafe_pickle_decode": True})` or the environment variable `ZARR_ARRAY__ALLOW_UNSAFE_PICKLE_DECODE=True`. Without the opt-in, reading such data raises `zarr.errors.PickleCodecDisabledError`. numcodecs has its own opt-in (`numcodecs.pickles.ALLOW_UNSAFE_PICKLE_DECODE`), which must also be enabled. Writing with the `pickle` codec is unchanged. diff --git a/changes/4539.bugfix.md b/changes/4539.bugfix.md deleted file mode 100644 index 62f2472ef6..0000000000 --- a/changes/4539.bugfix.md +++ /dev/null @@ -1 +0,0 @@ -The minimum supported numcodecs version is now 0.16.6, and 0.17.0 is excluded. Those versions leave `Pickle.decode` enabled by default, which allows arbitrary code execution when reading an untrusted Zarr v2 array ([GHSA-jqc8-qjfv-5m6r](https://github.com/zarr-developers/numcodecs/security/advisories/GHSA-jqc8-qjfv-5m6r)). diff --git a/docs/release-notes.md b/docs/release-notes.md index a83fddb419..caf8480316 100644 --- a/docs/release-notes.md +++ b/docs/release-notes.md @@ -6,6 +6,10 @@ ## 3.4.2 (2026-10-26) +### Features + +- Reading data encoded with the `pickle` codec now requires an explicit opt-in, because unpickling data from an untrusted store can execute arbitrary code ([GHSA-jqc8-qjfv-5m6r](https://github.com/zarr-developers/numcodecs/security/advisories/GHSA-jqc8-qjfv-5m6r)). Enable it with `zarr.config.set({"array.allow_unsafe_pickle_decode": True})` or the environment variable `ZARR_ARRAY__ALLOW_UNSAFE_PICKLE_DECODE=True`. Without the opt-in, reading such data raises `zarr.errors.PickleCodecDisabledError`. numcodecs has its own opt-in (`numcodecs.pickles.ALLOW_UNSAFE_PICKLE_DECODE`), which must also be enabled. Writing with the `pickle` codec is unchanged. ([#4465](https://github.com/zarr-developers/zarr-python/pull/4465)) + ### Bugfixes - Resizing an array with `delete_outside_chunks=True` now generates only the out-of-bounds chunk coordinates and deletes them with a bounded pool of workers, instead of materializing the entire old and new grids and all deletion tasks. This bounds auxiliary memory during shrinking and mixed growing/shrinking resizes while preserving retained chunks. Growing, or shrinking within the last chunk of each axis, no longer schedules any deletion work. When a deletion fails, the remaining deletions are now canceled before the error is raised; previously they ran to completion. ([#4344](https://github.com/zarr-developers/zarr-python/pull/4344)) @@ -13,6 +17,7 @@ - Fixed several ways `ZipStore` could lose data or fail. A store created with `mode="w"` or `mode="x"` now keeps its entries when it is used again after `close()`, `move()`, or unpickling, instead of truncating the archive. Every method now opens the archive on first use, under a lock, so `get`, `get_partial_values`, `set_if_not_exists`, and `clear` no longer raise `AttributeError` on a store that was never opened, and threads sharing a store no longer open the archive twice. An unpickled store now opens its archive on first use rather than when it is unpickled, so writing a dask array to a `ZipStore` no longer produces an unreadable file. Where reopening an archive would silently drop its entries, the store now raises instead: `RuntimeError` after a failed `close()`, `zipfile.BadZipFile` if the file is not a complete zip archive, and `io.UnsupportedOperation` for a file object that is not readable and seekable. The type of the `mode` parameter now includes `"x"`. Builds on [#3593](https://github.com/zarr-developers/zarr-python/pull/3593) by [@oelhammouchi](https://github.com/oelhammouchi). ([#4450](https://github.com/zarr-developers/zarr-python/pull/4450)) - Reads through `FusedCodecPipeline` no longer hang in a process forked after an earlier read had started the codec thread pool. The pool is now reset in the child process after a fork. Previously the child inherited an executor whose threads did not exist, so small reads were queued on a pool nothing would run. ([#4483](https://github.com/zarr-developers/zarr-python/pull/4483)) - Structured data types now default string (`S`, `U`) and raw-bytes (`V`) fields to zero bytes. Before, an array with a raw-bytes field failed to create without an explicit fill value, and string fields defaulted to `"0"`. ([#4495](https://github.com/zarr-developers/zarr-python/pull/4495)) +- The minimum supported numcodecs version is now 0.16.6, and 0.17.0 is excluded. Those versions leave `Pickle.decode` enabled by default, which allows arbitrary code execution when reading an untrusted Zarr v2 array ([GHSA-jqc8-qjfv-5m6r](https://github.com/zarr-developers/numcodecs/security/advisories/GHSA-jqc8-qjfv-5m6r)). ([#4465](https://github.com/zarr-developers/zarr-python/pull/4465)) - The `zarr.storage` module can now be serialized with `pickle` and `cloudpickle`. Previously, code that referenced the module (for example a Dask task that used `zarr.storage.ObjectStore`) failed with `TypeError: cannot pickle 'VerboseModule' object`. ([#4557](https://github.com/zarr-developers/zarr-python/pull/4557)) ### Improved Documentation From 1881c7bd309cc6c5f2e457d8ba4a357a280866cf Mon Sep 17 00:00:00 2001 From: Davis Vann Bennett Date: Tue, 27 Oct 2026 13:48:39 +0100 Subject: [PATCH 3/3] docs: date the 3.4.2 release notes 2026-10-27 Assisted-by: ClaudeCode:claude-opus-5-5 Co-Authored-By: Claude Opus 5.5 --- docs/release-notes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/release-notes.md b/docs/release-notes.md index caf8480316..195847735a 100644 --- a/docs/release-notes.md +++ b/docs/release-notes.md @@ -4,7 +4,7 @@ -## 3.4.2 (2026-10-26) +## 3.4.2 (2026-10-27) ### Features