You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
feat!: Align data dictionaries with the current data-dict spec (#316) - #330
Data dictionary files now follow the current data-dict spec, in both packages:
tables is a list of tables each with a required name field (previously a mapping of name to definition)
column range is a two-element [min, max] list (either end may be null for an open range)
values may be a value-to-label map ({M: Male, F: Female}); the values (keys) are used for categories
constraints may include assertion objects (assert/language/description), rendered as expr (description)
pkg-py parses the spec natively via pydantic models (TableSpec.name, list-form tables, ConstraintAssertion). A mode="before" validator rejects legacy mapping-form tables with a descriptive migration error pointing at the spec and the data-dict validate-spec CLI.
pkg-r stays validation-free by design: a single internal dd_tables_by_name() helper keys the spec's table list by name at the existing consumption sites (QueryChatSystemPrompt, querychat_tools, DBISource::build_column_details_impl).
Docs (context.qmd / context.Rmd), the shipped nutrition example YAML, and both changelogs are updated to the new format.
DataDict files now follow the current data-dict spec
(https://data-dict.tidyverse.org/spec.html):
- tables is a list of tables each with a name field (previously a
mapping of name to definition)
- column range is a two-element [min, max] list (either end may be null)
- values may be a value-to-label map ({M: Male, F: Female})
- constraints may include assertion objects ({assert, language,
description})
pkg-py parses the spec natively via pydantic models and rejects legacy
mapping-form files with a descriptive migration error. pkg-r stays
validation-free, keying the table list by name via dd_tables_by_name()
at existing consumption sites. Docs, examples, vignettes, and
changelogs updated in both packages.
… range bounds
Addresses Copilot review on #330: the bundled Python nutrition app's
data dict still used mapping-form tables (now rejected at load), and a
one-sided range like [0, null] lost its known bound in both the LLM
schema text and the user-facing schema card (now rendered as '0 to ?').
- ConstraintAssertion: enable populate_by_name so assert_= construction
and model_validate(model_dump()) round-trips work
- ColumnSpec.values: coerce numeric YAML map keys (e.g. {0: Control})
to strings instead of rejecting them
- pkg-r: add regression tests asserting rendered output for open-ended
ranges, map-form value keys, and assertion constraints
Addresses Copilot review on #330: the linked data-dict spec uses
-.inf/.inf (not null) for open bounds and asks that open bounds be
described in prose. Both packages now normalize infinite range ends to
an open bound (rendered as '?', same as null), and the data dict guide
documents the sentinels and the prose requirement.
Fix YAML indentation and separate migration example blocks
pkg-py/src/querychat/_data_dict.py:178
Nonblocking: the migration example indents columns farther than name, so its replacement is invalid YAML. Align those keys; showing before and after as separate blocks also makes the example easier to copy.
Addresses Copilot review on #330: the after example indented columns
farther than name (invalid YAML); before/after are now separate blocks
that copy cleanly.
…ading
Addressing review findings on the spec-alignment PR:
- `$version` handling: files declaring a version that is invalid, predates
the first spec version, or is newer than the newest spec version querychat
understands now warn and are read on a best-effort basis (unknown fields
are ignored). This keeps additive spec evolution maintenance-free — the
known-version constant is calibration for the warning, not a hard gate,
so it only needs bumping opportunistically. The legacy mapping-form
`tables` check in pkg-py remains a hard error.
- pkg-r: read_data_dict() now warns on those same `$version` problems, on
legacy mapping-form `tables`, and on nameless table entries — all of
which previously degraded silently (the whole dict was ignored).
- pkg-r: dd_tables_by_name() keeps the first entry on duplicate table
names, matching pkg-py's DataDict.get_table().
- Move the changelog entries to Breaking changes sections and document
the `$version` behavior in the context guides.
Replace the hand-rolled version parser with packaging.version.Version
(already in the dependency tree; now declared directly). Also split the
floor comparison out into a fixed _FIRST_SPEC_VERSION / .dd_first_spec_version
constant — comparing against the known-version constant would have wrongly
warned on legitimate 0.1.0 files once the known version bumps.
Old/malformed files have a clear migration path, so error instead of
warning — a warning at app startup is easy to miss, and reading on would
silently produce a degraded dictionary. This also gives pkg-r and pkg-py
a single uniform policy: old or malformed files fail with a descriptive
error; files declaring a newer spec `$version` warn and are read on a
best-effort basis.
- Python: invalid and predating `$version` collapse into a single
rejection in `_check_spec_version()` (the legacy mapping-form `tables`
error is unchanged).
- R: `dd_check_spec_compat()` now aborts — instead of warning — on an
invalid or predating `$version`, legacy mapping-form `tables`
(previously the whole dict was silently ignored), and table entries
without a `name` (matching pydantic's required `name`).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #316
Summary
Data dictionary files now follow the current data-dict spec, in both packages:
tablesis a list of tables each with a requirednamefield (previously a mapping of name to definition)rangeis a two-element[min, max]list (either end may be null for an open range)valuesmay be a value-to-label map ({M: Male, F: Female}); the values (keys) are used for categoriesconstraintsmay include assertion objects (assert/language/description), rendered asexpr (description)pkg-py parses the spec natively via pydantic models (
TableSpec.name, list-formtables,ConstraintAssertion). Amode="before"validator rejects legacy mapping-formtableswith a descriptive migration error pointing at the spec and thedata-dict validate-specCLI.pkg-r stays validation-free by design: a single internal
dd_tables_by_name()helper keys the spec's table list by name at the existing consumption sites (QueryChatSystemPrompt,querychat_tools,DBISource::build_column_details_impl).Docs (
context.qmd/context.Rmd), the shipped nutrition example YAML, and both changelogs are updated to the new format.Verification
DataDict.from_yaml("data-dict.yaml")parses this in Python; the old mapping form raises aValidationErrorwith migration instructions.test_data_dict.py/test-DataDict.Rsuites covering the new forms).