Repository navigation
test(diagram): reference model and properties for Diagram.expand - #1621
Open
dimitri-yatsenko wants to merge 4 commits into
Open
dimitri-yatsenko wants to merge 4 commits into
dimitri-yatsenko wants to merge 4 commits into
Conversation
Add hypothesis to the test dependency group and to the test and dev extras, for the property tests of Diagram.expand. tests/conftest.py registers three profiles, selected with HYPOTHESIS_PROFILE (default: ci when CI is set, else dev): - dev: 50 examples, for fast local runs; - ci: 500 examples, derandomized, no deadline; - thorough: 2000 random examples, with failing examples saved under the gitignored .hypothesis/ directory. pixi.lock is re-solved. Besides hypothesis and sortedcontainers in the test and dev environments, it now records the test-py310 and test-py314 environments that pyproject.toml already declares, and packaging in the default environment; no existing package changes version.
Add an executable reference model of Diagram.expand (L0 of the test
program) that needs no database:
- tests/unit/expand_model.py: TableSpec, FK and Schema; databases as
{table: {pk_tuple: row}}; ref_e under MATCH SIMPLE (A2); the owning
reference (A1), with parts that violate it traversed as plain tables;
the row relations child, parent, inst, down and up; D, D1, U and both
as breadth-first closures; the checkers ri_ok, group_whole,
group_closed, lift_closed and q_acyclic; apply_delete; the
self.upstream of a row not inserted yet; and the Dependencies graph
built the way Dependencies.load builds it.
- tests/unit/test_expand_spec.py: Hypothesis generators for schemas and
consistent databases, and properties for T1-T8. Minimality (T4, T5) is
checked by brute force on databases of at most eight rows. T8 calls
datajoint.dependencies.topo_sort with MySQL and PostgreSQL quoting.
Two examples show that T8 needs A1, and parametrized checks confirm
that the generators reach each shape the properties rely on.
- tests/unit/test_expand_shapes_model.py: one test per shape (S1-S7,
G1-G6, K1, K2, K5, K6) with the expected rows written by hand.
Rewrite tests/unit/test_expand_shapes_model.py to mirror the Traversal
Shape Catalog (traversal-shapes.md) one to one: each shape's schema,
rows and seeds, the rows it states, and its node set, including the
tables listed as none. The catalog's claims about copies, relabeling,
delete modes and the rounds of an iterating group are checked too. The
seed evaluations that need a database (string seeds, Top, the warning)
and X1 are left to the integration suite; X2's cycle in Q is checked.
Extend the reference model:
- inst(r) = {r} for a row in no group, as diagram.md defines it;
- table_reach, the node set of an expanded diagram;
- RowGraph.delete with the three part_integrity modes;
- one_pass, the computation over Q evaluated on rows, with the
iteration rule of diagram.md or the exact one;
- condition semantics for the R1 copy rule and for relabeling keys;
- Schema(..., a1="chain"), A1 generalized to owning chains with
renames (#1429), with the owning attributes composed along the chain.
In test_expand_spec.py, T1-T8 now run under both readings of A1, T1 and
T2 include D1, and T6 checks monotonicity and minimality. New properties
check the one pass, the copy rule and relabeling. Three strict xfails
record where diagram.md's rules disagree with the definitions: a sibling
FK that does not carry the master key links instances without renaming
it; relabeling drops a key on a secondary attribute of the parent; and
a NULL key value on a nullable FK relabels to the NULL child rows.
Follow the revised traversal specification (datajoint-docs#249): - A1 is ownership: a part is owned if an FK in its primary key references the master or an owned part of the same master, renames allowed. Its owning attributes come from that chain, preferring an unrenamed -> master and otherwise the first owning reference. This replaces the strict reading and the a1 switch; a part that references its master only below the line is no longer in its group. - The lift goes through the owning attributes, and a group iterates if and only if some FK inside it does not map owning attributes onto owning attributes. The relabel rule is the revised one. - The shape catalog gains the rewritten G6 (a part its master does not own) and G7 (an owned part-of-part with renamed attributes, #1429). The three xfails for the spec's iteration and relabel rules are gone: the docs now state the model's rules, and the tests assert them. Two strict xfails remain for T8: topo_sort still groups every part with its master by name, while the spec makes a part that is not owned a plain table (#1619). test_expand_spec.py opens with a description of the test layers and of the Hypothesis profiles.
dimitri-yatsenko
requested review from
MilagrosMarin,
lum-agilyti and
ttngu207
October 17, 2026 22:47
This was referenced Oct 17, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The first part of the test harness for
Diagram.expand: an executable reference model of the traversal specification in datajoint/datajoint-docs#249, and tests that need no database.What it adds
tests/unit/expand_model.py, the reference model. It definesD,D1,Uandbothas closures over rows, together with:tests/unit/test_expand_spec.pychecks the derived results T1–T8 and those rules with Hypothesis, on random schemas and databases.tests/unit/test_expand_shapes_model.pymirrors the Traversal Shape Catalog of docs(specs): normative spec of Diagram.expand, with a traversal shape catalog datajoint-docs#249 one to one, under the same shape IDs. Every expected row is copied from the catalog by hand, not computed by the model.Two T8 examples are strict
xfails, becausetopo_sortstill groups every part with its master by name (#1619). They start passing when #1620 merges, and the strict marker then flags them for removal.Running
HYPOTHESIS_PROFILEselects the profile:devciCIis setthorough.hypothesis/Dependencies
hypothesisjoins thetestdependency group and thetestanddevextras.pixi.lockis re-solved. The lock on master was stale, so the diff also records thetest-py310andtest-py314environments andpackaging. No package version changes in thedefault,devortestenvironments.Next
The database layers (L1–L3) follow in later PRs. They run the engine against the same model on MySQL and PostgreSQL.