Repository navigation
Conversation
…ign the v2 readers with v3 Fixes the five reports in #417, each found by a differential run of zarr-metadata.js against zarr-metadata 0.6.0: - A v2 struct record's problem is located where the document writes it, `("dtype", 0, 1)`, not under a `fields` key the document lacks. The `_configuration_loc` prefix hook becomes `_in_configuration(loc, place)`, which maps a place in a configuration to its document location; `nested` stays keyed by configuration place. - The v2 dtype string `"struct"` is refused as a non-typestr. - A listed v3 group whose copy has a problem other than an unknown key is reported for that problem alone, not also as reading otherwise. - A union of fixed-length arrays tries only the branches of the value's length, and names them as "an array of 2 or 3 elements". - v2 unknown `.zarray` members may hold non-finite numbers, as v3 unclaimed members may; `.zmetadata` keys with a leading `/`, an empty name, or a `.`/`..` segment are refused, as v3 listing keys are; and `attributes` in a `.zgroup` is `invalid_value`, as in a `.zarray`. Assisted-by: ClaudeCode:claude-opus-5-5
… one invalid value A name that encodes its configuration is one token, so a part its definition does not declare is a wrong value of the name, not an unknown key a tolerant caller may skip. `<f4[ns]` and `|O4` were refused only as `unknown_key` problems naming internal keys (`unit`, `scale_factor`, `itemsize`); they now get one `invalid_value` problem naming the part as the typestr spells it, through a new `_name_part` kind hook. Every other problem with an encoded configuration is an `invalid_value` too, so `<M8` is no longer a `missing_key`. Assisted-by: ClaudeCode:claude-opus-5-5
…cument writes it A codec written as its name alone, or as an object without `configuration`, has an empty configuration the document does not write. A `bytes` codec there with values of several bytes reported its missing `endian` at `codecs.N.configuration.endian`, which resolves in nothing; it is now the configuration itself, missing, at `codecs.N.configuration`, as a gzip codec without its required configuration is. A problem with the configuration as a whole is located at the codec. Assisted-by: ClaudeCode:claude-opus-5-5
…g type everywhere `refused_kind` compared JSON types, so an integral float such as `"zarr_format": 2.0` matched an integer `Literal` and was an `invalid_value`, while a union holding the same `Literal` filtered the float out by its integer shape and reported an `invalid_type`, as an `int` member does for `4.0`. An integer is now a type of its own in `refused_kind`, so the kind no longer depends on where the literal sits. Assisted-by: ClaudeCode:claude-opus-5-5
…uments Hypothesis changes valid documents of every kind a reader reads (v2 and v3 arrays, structs, shards, groups, consolidated metadata) in one to three places, and checks that every problem is located in the document it was read from, a missing key where it would go, that none is reported twice, and that an unknown key is located at a key of an object. Targeted properties pin the rules earlier releases broke: a typestr's problems are wrong values of the dtype, a float for an integer `Literal` is the wrong type wherever it sits, a member nothing claims may hold any number in both formats, and both formats refuse the same node paths. Run against the code before the recent fixes, 11 of the 16 properties fail. `read_node_metadata_v2` still locates problems in the merged document, not in the files it reads, so that property is a strict xfail. Assisted-by: ClaudeCode:claude-opus-5-5
…nd read name parts from the name Review fixes for the two previous commits: - `_read_carried` took the parts a name encodes but its definition does not declare from the problems' kinds, so a definition whose rules yielded an `unknown_key` at `()`, an index, or an undeclared key crashed the read. The undeclared parts are now the encoded keys the configuration does not declare, read before the rest is checked, and every problem the rules find is a wrong value of the name. - Moving a problem out of a configuration the document does not write applied only to a codec's chunk rules. `in_document` now does it for `resolve`'s rules and the pipeline's chunk rules alike, replacing the pipeline's own helper. - The typestr property also checks that a typestr is refused exactly when it has a problem. Assisted-by: ClaudeCode:claude-opus-5-5
…les it reads
`read_node_metadata_v2`, `read_repaired_node_metadata_v2` and the v2
models' `from_key_value` read a mapping of files but located problems in
the merged document, so a `.zarray` holding `attributes` beside a bad
`.zattrs` gave two problems at `("attributes",)`, a location in neither
file. Each problem is now in the file that holds it, as the consolidated
reader locates it in its entry, through one `in_files` helper they share.
An array node's reading gains `at`, `(".zarray",)`, under which
`fields()` locates each field, so `with_problems` pairs fields with their
problems. The model reads its merged document, so its own reading stays
at `()` and no longer equals the node reading. Pickling keeps `at`.
Each `from_key_value` now reads through the same per-type node reader,
and `read_group_metadata_v2` builds its model without checking the
document a second time. The node-files property is no longer an xfail.
Assisted-by: ClaudeCode:claude-opus-5-5
Assisted-by: ClaudeCode:claude-opus-5-5
…rs#4574 Two entries also lead with what a user sees: a listed group with its own problem, and a struct record of the wrong length. Assisted-by: ClaudeCode:claude-opus-5-5
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🤖 AI text below 🤖
The zarr-metadata readers now locate every problem in the document or files they were given, and report each mistake once, with the same kind wherever it occurs. The v2 readers now agree with the v3 readers where the two disagreed without a reason in the spec.
Fixes #417. The agent also fixed four further bugs that a sweep of the tests found pinned:
13cdc5ab2): v2 struct record problems were located under afieldskey the document lacks; the string"struct"was read as a v2 dtype; a listed v3 group with its own problem was also reported as reading otherwise; a struct record of the wrong length named only the three-element form; and v2 and v3 disagreed on non-finite numbers in unknown members, on.zmetadatakeys such as/a/.zarray, and on the kind of a misplacedattributes.e10628723,b4d75d38d):<f4[ns]and|O4were refused only asunknown_keyproblems naming internal keys, so a caller tolerating unknown keys accepted them. Each is now oneinvalid_valuethat names the part.7e233b133,b4d75d38d): abytescodec written as"bytes"reported its missingendianinside a configuration the document does not write.494e72a1c):"zarr_format": 2.0was aninvalid_valuewhere a union holding the same literal gaveinvalid_type.195efdb1a):read_node_metadata_v2and the v2from_key_valuelocated problems in the merged document, not the files. An array node's reading gainsat, under whichfields()locates each field; a node reading no longer equalsreading.metadata.reading.tests/model/test_problem_invariants.py(daef5fdc8) is a Hypothesis suite. It changes valid documents of every kind and checks that every problem is located in the input and reported once, plus one property per rule above. Run against the code before these fixes, 11 of its 16 properties fail.Items 1, 5 (key normalization) and the v2 node change are breaking: problem locations and kinds change.
just checkpasses on Python 3.11 and 3.14.Bookkeeping
mainata2f2415c8. The agent fast-forwarded the fork'smainto the same commit, so the diff is these commits alone.zarr-developers/zarr-python/pull/{n}, so they need renaming again if this lands upstream under another number.🤖 Generated with Claude Code