feat: add competency criteria models for CBE authoring layer - #800
feat: add competency criteria models for CBE authoring layer#800jesperhodge wants to merge 9 commits into
Conversation
|
Thanks for the pull request, @jesperhodge! This repository is currently maintained by Once you've gone through the following steps feel free to tag them in a comment and let them know that your changes are ready for engineering review. 🔘 Get product approvalIf you haven't already, check this list to see if your contribution needs to go through the product review process.
🔘 Provide contextTo help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:
🔘 Get a green buildIf one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green. 🔘 Update the status of your PRYour PR is currently marked as a draft. After completing the steps above, update its status by clicking "Ready for Review", or removing "WIP" from the title, as appropriate. Where can I find more information?If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources: When can I expect my changes to be merged?Our goal is to get community contributions seen and reviewed as efficiently as possible. However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:
💡 As a result it may take up to several weeks or months to complete a review and merge your PR. |
| OR = "OR", _("Or") | ||
|
|
||
|
|
||
| def validate_rule_payload(rule_type: str, payload: Any) -> None: |
There was a problem hiding this comment.
I don't like payload: Any, there should be a type for the dict.
And I want to validate against that type.
| .. no_pii: | ||
| """ | ||
|
|
||
| # Set at from_db() time to the scope this row had when it was loaded from the database, so |
There was a problem hiding this comment.
Comment too hard to understand
| # from_db() is a classmethod, so it sets this through a local `instance` variable rather than | ||
| # `self`, which pylint's protected-access check can't tell apart from reaching into another | ||
| # object's internals. | ||
| loaded_scope: tuple[int | None, int | None, int | None] | None = None |
There was a problem hiding this comment.
What is loaded_scope and why is it a tuple of ints?
| Organization, | ||
| null=True, | ||
| blank=True, | ||
| on_delete=models.PROTECT, |
There was a problem hiding this comment.
organization & course on_delete should also be CASCADE, with Python logic elsewhere ensuring that no learners are linked to this (if they are linked, organization / course can still be deleted but rule profile stays.)
There was a problem hiding this comment.
I think this is not quite right. CASCADE should result in archival, not deletion.
Question to flag for later: what should happen if someone actually wants to modify (or rather, delete and then replace) a rule profile? Assuming this gets archived, do the archived ones still need to be unique? In that case they are never modifiable and we need some hard delete or overwrite mechanism, I guess.
| CourseRun, | ||
| null=True, | ||
| blank=True, | ||
| on_delete=models.PROTECT, |
There was a problem hiding this comment.
This also needs to be CASCADE but then it should stay if there are any actual learner's already connected to the mastery. Same as elsewhere
| """Capture the scope this row had when loaded, so clean()/save() can detect an edit to it.""" | ||
| instance = super().from_db(db, field_names, values) | ||
| # field_names holds attnames (e.g. "organization_id"), not field names. Only capture when |
There was a problem hiding this comment.
I have no idea what this is supposed to mean. Clarify what the intent is here.
There was a problem hiding this comment.
Why do we need to override from_db at all?
| ) | ||
| return instance | ||
|
|
||
| def _check_scope_immutable(self) -> None: |
There was a problem hiding this comment.
This seems pretty complicated code. Can it be simplified following KISS principle?
| help_text=_("The profile this criterion uses by default. Null only when overrides are set instead."), | ||
| ) | ||
| rule_type_override = models.CharField(max_length=32, choices=RuleType, null=True, blank=True) | ||
| rule_payload_override = models.JSONField(null=True, blank=True) |
There was a problem hiding this comment.
Use attrs, as in my other comment.
| null=True, | ||
| blank=True, | ||
| db_column="competency_rule_profile_id", | ||
| on_delete=models.PROTECT, |
There was a problem hiding this comment.
I think this is okay because rule profiles should be archived not deleted in general.
| ) | ||
| uuid = immutable_uuid_field() | ||
|
|
||
| history = HistoricalRecords(excluded_fields=["scope_code"]) |
There was a problem hiding this comment.
I wonder if the history and the archived flag somehow conflict with each other.
Also I wonder about this immutability idea anyway: if this should be immutable why a history?
Maybe immutability is not a clear concept in the issue. This will need further clarification from the architect.
| # ============================================================================================== | ||
|
|
||
|
|
||
| def test_group_parent_cascade(tag: Tag) -> None: |
There was a problem hiding this comment.
Test names are bad. They should all state the expected behavior. I don't care if that makes them long. E.g. "test_criteria_group_deletion" is bad, while "test_delete_criteria_group_cascades_to_child_groups" is good.
There was a problem hiding this comment.
Here's some reading for proper naming of tests. https://learn.microsoft.com/en-us/dotnet/core/testing/unit-testing-best-practices
|
|
||
| def test_group_parent_cascade(tag: Tag) -> None: | ||
| """ | ||
| Deleting a CompetencyCriteriaGroup cascades to any child group referencing it via `parent`: |
There was a problem hiding this comment.
At least some of the tests need to be a bit more integrative. In that: sure, we have tested that the cascade is there, but it's not clear why. There should be at least some tests that look at the actual bad outcome that we want to avoid: for example, do we suddenly have orphaned child groups that do not serve any purpose?
|
|
||
|
|
||
| # ============================================================================================== | ||
| # Transitive deletion tests required by #641's Deletions criteria: deleting an oel_tagging.Tag, |
There was a problem hiding this comment.
Too unclear. I have no patience to decipher what this comment means. Either it's clear at a glance or it's useless.
Implements the authoring and definition half of the CBE data model from ADR-0002: CompetencyCriteriaGroup (internal AND/OR nodes), CompetencyRuleProfile (reusable scoped evaluation defaults) and CompetencyCriterion (leaf nodes). Also adds the taxonomy_overrides_org column that PR openedx#712 left off CompetencyTaxonomy. CompetencyRuleProfile.scope_code is a generated, never-null column with a plain unique constraint. SQL never treats two NULLs as equal, so a unique constraint over the three nullable scope columns would accept two rows with the same scope, and the conditional UniqueConstraint that would normally fix that compiles to a partial index MySQL does not support. Both structural invariants are database check constraints rather than clean() checks, since DRF serializers, QuerySet.update() and bulk_create() never call full_clean(). Payload shape validation stays in clean(), per the issue. Every new foreign key is on_delete=PROTECT with a TODO(openedx#799) comment. That is a fail-closed placeholder, not a per-key decision; openedx#799 sets the real values once openedx#655 lands. openedx_catalog joins .importlinter's root_packages and the src_layering contract, since CompetencyCriteriaGroup.course is the first foreign key from openedx_learning into that app. django-simple-history moves into base.in: it was only ever a transitive dependency of edx-organizations, and setup.py builds install_requires from base.in. Refs openedx#641 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
openedx#655 closed with an approved design and openedx#799 is now closed as superseded, so both halves of the nine repeated TODO comments were false: openedx#799 does not own the on_delete question, and no follow-up ticket will set "the real" per-foreign-key values. Replaces those nine identical comments with one explanation in the module docstring, which also records the open question openedx#655's design creates for CompetencyCriteriaGroup.tag and CompetencyCriterion.object_tag: that design keeps openedx_tagging ignorant of CBE and promises a plain hard delete for a tag no learner holds mastery against, which PROTECT turns into a ProtectedError whenever an author's criteria tree references the tag and nobody has been graded yet. The PROTECT values themselves are unchanged. They remain the fail-closed default until openedx#655's reviewers settle the question. Refs openedx#641, openedx#655 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
openedx#641 requires at least one test per foreign key asserting that deleting the referenced row matches what the field declares. All nine are PROTECT, so all nine assert ProtectedError, and each inspects ProtectedError.protected_objects rather than only the exception type: a single delete can trip several protected relationships, so a bare pytest.raises would not prove which foreign key did the protecting. Two cases needed isolating to avoid passing for the wrong reason. CatalogCourse.org is itself PROTECT, so the organization test uses an organization with no catalog course attached. Tag.taxonomy is CASCADE, so the competency_taxonomy test omits the tag and group fixtures. A tenth test pins the open openedx#655 question in executable form: deleting a CompetencyTaxonomy whose tag carries a criteria tree raises ProtectedError today, though that design promises the delete succeeds when no learner status exists. It is the test that has to change if the reviewers move CompetencyCriteriaGroup.tag to CASCADE, and says so. Refs openedx#641 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
openedx#655 decided the on_delete question on 2026-09-02, so the four foreign keys between definition tables become CASCADE: CompetencyCriteriaGroup.parent, CompetencyCriteriaGroup.tag, CompetencyCriterion.group and CompetencyCriterion.object_tag. The other five stay PROTECT and are now final. Deleting a Tag nobody holds mastery against has to succeed, and openedx#655's design forbids openedx_tagging from knowing CBE exists, so the tagging-side path cannot clear the criteria tree first. CASCADE lets the delete take the tree with it. parent and group need it too, because Django's collector looks up referencing rows in the database rather than in the set it has already collected, so a parent and child reached in one batch would trip PROTECT and abort the walk partway down. This does not weaken ADR-0002 Decision 7. The four CASCADE links are what carries the collector down to the PROTECT that enforces it, on openedx#642's Student*Status foreign keys one and two levels below the tag, which Django reaches only by walking CASCADE edges. Migration 0002 is edited in place rather than gaining an AlterField, since it is unmerged. The delete tests are reworked accordingly and extended with the transitive cases: a tag delete cascading a whole tree, a group delete at depth taking its descendants and their criteria, and a taxonomy delete reaching through tag to group to criterion. The matching "raises ProtectedError when a learner status exists" halves need openedx#642's tables and belong to that slice, which a comment in the test file records. One cascade test also asserts django-simple-history writes a history_type='-' row per removed row, so the cascade is not silent for audit. Refs openedx#641, openedx#655 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…dx#641 Four changes, all from openedx#641's revision. CompetencyRuleProfile.competency_taxonomy becomes CASCADE, making the split five CASCADE and four PROTECT. The reason is a requirement rather than a mechanism: a rule profile must never be why a taxonomy delete fails. Once taxonomy-scoped profiles exist, deleting a taxonomy has to be blocked only when learner data is connected to it, and that check belongs in Python at the application layer, the way openedx#655 settled it for every other record type. PROTECT would push the decision into the database, which cannot tell the two cases apart. Nothing changes behaviorally in MVP, because the only profile is the system default and its three scope columns are all null. openedx_content and openedx_catalog become independent siblings in the src_layering contract rather than separate ranks. A layers contract is a strict total order, so ranking them asserted both that openedx_content may import openedx_catalog and that openedx_catalog may never import openedx_content. src/openedx_catalog/ARCHITECTURE.md records that direction as explicitly undecided, so the sibling form, which forbids imports both ways, asserts only what is settled. The Meta.db_table override is dropped, so the leaf table is Django's default openedx_learning_competencycriterion. ADR-0002 Decision 4's heading names a domain concept rather than instructing a rename, and no model anywhere in src/ overrides db_table. The competency_taxonomy delete test becomes a cascade test to match. Refs openedx#641 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Replaces the hand-rolled key-walking in validate_rule_payload with one attrs class per rule type, using the modern `from attrs import define, field` style already used in openedx_tagging and openedx_content. attrs is already a declared dependency, so nothing changes in requirements. The spec class is now the definition of the shape: constructing it does the checking, and the expected key set is derived from it via attrs.fields() rather than repeated in a literal. Adding MasteryLevel later is one class plus one registry entry. The field validators stay hand-written rather than using attrs.validators.in_(), because that helper's default message dumps the whole Attribute repr into the error, which a course author would see in the Django admin. Key errors are raised before construction for the same reason: Python's own TypeError names the offending key but leaks "GradeRule.__init__()" along with it. validate_rule_payload is now also called from save() on both models. clean() is reached only via full_clean(), so objects.create() and instance.save() previously bypassed payload validation entirely; this closes both. QuerySet.update(), bulk_create() and DRF serializers remain uncovered, because none of them builds or saves a model instance, and both model docstrings say so rather than implying more. CourseRun.save() is the existing precedent in this repo for validating in save(). One consequence, split rather than papered over: a criterion with rule_type_override set and no payload now raises ValidationError from save() before the check constraint sees it, so that case moves out of test_criterion_profile_xor_override_constraint into its own test. The other three invalid states still reach the constraint and still raise IntegrityError. The seed data migration is unaffected: apps.get_model() returns a historical model that does not carry the custom save(). Refs openedx#641 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Add simple_history to INSTALLED_APPS in the test and dev settings. The CBE models declare HistoricalRecords(), and while the historical models are built under openedx_learning's own app label and so work without the entry, its absence breaks SimpleHistoryAdmin's history views, its template tag libraries and the populate_history/clean_old_history/clean_duplicate_history commands. The package ships no AppConfig and registers no system check, so nothing warns. Rank openedx_content above openedx_catalog in the src_layering contract rather than making them independent siblings. The sibling form forbids imports in both directions, including the one 0007-pathway-catalog-content-split.rst requires: "openedx_content knows about openedx_catalog, never the reverse." Ranking asserts only the settled half, that catalog never reaches up into content, and does not have to be loosened when pathway content lands. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
scope_code becomes an ordinary column written in save(), null when the profile is archived, keeping the plain UniqueConstraint and adding a CheckConstraint tying the two. This fixes three defects. An archived profile used to occupy its scope's unique slot forever, so no replacement could ever be created for that scope; SQL never treats two NULLs as equal, so archived rows now share a scope freely while exactly one live row holds it, identically on MySQL and SQLite. A database GeneratedField was also rewritten whenever Django's collector nulled a nullable scope foreign key before deleting, which it does on any backend where can_defer_constraint_checks is false, colliding with the seeded default row on MySQL while passing on SQLite. A plain column is not rewritten by that update. It also avoids Django never populating a GeneratedField in memory on MySQL. Two on_delete values change, per ADR-0002 Decision 7 as amended by b5fae6b. CompetencyRuleProfile.course becomes CASCADE, which that amendment requires when it says a profile is deleted with "a taxonomy or course" it is scoped to. CompetencyCriteriaGroup.course becomes CASCADE for the same stated reason: a course is only hard-deleted once nothing beneath it needs protecting, so a course-scoped criteria tree is safe to remove with it rather than blocking the delete. Both deviate from openedx#641, which lists them as PROTECT. Drop the loaded_scope cache and the from_db() override; _check_scope_immutable() now always reads the persisted scope, on self._state.db so a non-default alias is not silently skipped. save() calls full_clean() on both models instead of duplicating a hand-picked validation list that could drift from clean(). Move RuleType, the payload spec classes and the parser to rule_payloads.py, so the JSON schema is not trapped behind a module importing five models, and have it return the frozen GradeRule rather than discarding it. Both models derive their choices from the payload-spec registry, so a rule type can never be offered to an author and then rejected on save. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…dge cases Add a test per acceptance criterion, plus the cases the previous per-foreign-key shape could not reach: a taxonomy or course run deleted with a scoped rule profile, two taxonomies deleted together, an archived profile's scope being reused, and an ObjectTag delete leaving a childless group behind. Add test_criteria_trees.py for whole-tree deletion, so a test proves the bad outcome is avoided rather than only that a cascade fired: it builds a root/branch/grandchild tree with criteria at two levels and a mix of profile-assigned and override criteria, deletes in the middle, and asserts exactly which rows survive. Run the deletion paths under MySQL's collector semantics while still on SQLite, by setting can_defer_constraint_checks to False. That is what makes this class of bug visible in the fast local suite instead of only in the MySQL CI job. Rename every test so the name states the expected behavior rather than the mechanism, and move the fixtures duplicated across both files into conftest.py. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
a6a8e76 to
5380a64
Compare
Adds the authoring and definition half of the competency-based education (CBE) data model, per
ADR-0002
and ADR-0003.
Three models, one column on an existing model, two migrations, four configuration edits. No REST
endpoints, no UI, no evaluation logic.
Closes #641.
Warning
The deletion behavior here is work in progress and will change within this PR, pending
discussion with @mgwozdz. Three separate things are still open: the individual
on_deletevalues, the cascade-versus-protect split as a design question, and the archive-versus-delete
story as a whole. ADR-0002 Decision 7 was amended twice in three days (
f9ec022, thenb5fae6b) while this branch was being written, and the second amendment reversed the reasoningbehind the first. Treat the
on_deletetable below as the current position, not a settled one.Nothing else in this PR depends on how that discussion resolves.
Note
Implemented by an AI agent (Claude Code), with a human directing the work and reviewing the
decisions. Please review it as you would any other PR.
The three models
CompetencyCriteriaGroupis an internal AND/OR node of a criteria tree. A tree hangs off onecompetency, which is a
Tagin a competency-enabled taxonomy.CompetencyCriterionis a leaf. It points at oneObjectTag, meaning one specific piece oftagged content, and takes its pass rule either from a shared profile or from its own inline
override pair.
CompetencyRuleProfileis a reusable set of evaluation settings, scoped to at most one of anorganization, a course, or a taxonomy. One row is scoped to none of them: the system default,
seeded by migration, which every criterion falls back to when nothing more specific applies. In
this MVP it is the only profile that exists, so all three scope columns are always null.
Plus
CompetencyTaxonomy.taxonomy_overrides_org, the boolean ADR-0002 Decision 1 asks for.Decisions
scope_codeis an ordinary column written insave(), null when the profile is archived, ratherthan a database
GeneratedField. At most one profile may exist per distinct scope, and a uniqueconstraint over the three nullable scope columns cannot enforce that, because SQL never treats two
NULLs as equal. The idiomatic fix, a conditionalUniqueConstraint, compiles to a partial indexthat MySQL silently skips (ADR-0002 Rejected Alternative 6). Deriving the value works, but deriving
it in the database broke twice: an archived profile kept occupying its scope's unique slot, so no
replacement could ever be created for that scope; and Django's collector nulls a nullable foreign
key before deleting the row it points at, which recomputed
scope_codemid-delete and collided withthe seeded default row. The collector only does this where
can_defer_constraint_checksis false.MySQL has that flag false and SQLite has it true, so this failed only on MySQL. Writing the column
in
save()fixes both: the collector's update no longer rewrites it, and archived rows carryNULL, so any number of them share a scope while exactly one live row holds it, identically onevery backend. A
CheckConstrainttiesarchivedandscope_codetogether, so aQuerySet.update()bypassingsave()is refused by the database rather than silently breaking theinvariant. This deviates from #641, which asks for a generated, never-null column; ADR-0002
Decision 3 needs a matching amendment.
Validation and immutability each collapsed to one path.
save()now callsfull_clean()onboth models rather than repeating a hand-picked list of checks that could drift from
clean(),following the precedent
CourseRun.save()sets. Scope immutability drops its cached copy of theloaded scope and its
from_db()override in favor of always reading the persisted scope, onself._state.dbso a non-default database alias is not silently skipped. The rule payload schemamoved out of the models module into
rule_payloads.pyand now returns the parsedGradeRuleratherthan discarding it, so #642's evaluation code can consume typed fields without importing five
models. Both models derive their
rule_typechoices from the payload-spec registry, so a rule typecan never be offered to an author and then rejected on save.
Two
on_deletevalues differ from #641, and this is the part under discussion.CompetencyRuleProfile.coursebecomesCASCADEbecauseb5fae6bsays a profile is deleted alongwith "a taxonomy or course" it is scoped to.
CompetencyCriteriaGroup.coursebecomesCASCADEforthat amendment's own stated reason: a course is only hard-deleted once nothing beneath it needs
protecting, so a course-scoped criteria tree is safe to remove with it rather than blocking the
delete permanently.
CompetencyRuleProfile.organizationstaysPROTECT, since the amendment namesonly taxonomy and course, and an organization is not a competency-definition record. That asymmetry
is deliberate and is one of the things to settle.
CompetencyCriteriaGroup.tag,.parent,.courseCASCADECompetencyCriterion.group,.object_tagCASCADECompetencyRuleProfile.competency_taxonomy,.courseCASCADECompetencyCriterion.rule_profilePROTECTCompetencyRuleProfile.organizationPROTECTOther deviations from #641
.importlinterranksopenedx_contentaboveopenedx_catalograther than making themindependent siblings. The sibling form forbids imports both ways, including the direction
0007-pathway-catalog-content-split.rstrequires, so it would have to be loosened to do analready-decided thing.
RuleTypedeclares onlyGrade. ADR-0002 also namesViewandMasteryLevel, but neither has apayload shape, so declaring them offers an author a choice that always fails on save. Adding one
later means a spec class, a registry entry, and the matching member together.
Known gaps
CompetencyTaxonomywhose taxonomy-scoped profile is assigned to a criterion raisesProtectedError. Django's collector looks up referencing rows in the database rather than in theset it has already decided to delete, so
CompetencyCriterion.rule_profile'sPROTECTfires evenfor criteria being deleted in the same operation. This is unreachable in the MVP, where the only
profile is the system default. A test pins it and names the fix: a fifth reassignment event in
ADR-0002 Decision 4, in an application-layer function.
self._state.db. That needs a second databasealias, and configuring one breaks the whole test session on a pre-existing bug in
openedx_content/backcompat/collections/migrations/0004_collection_key.py, whosegenerate_keysstep queries without
.using(schema_editor.connection.alias)and so always hitsdefault.Worth its own issue.
branch from [Arch] Implementation approach for competency data delete/edit guardrails #655, the
archivedcolumn on the group and leaf models ([BE] Add archived field to CompetencyCriteriaGroup and CompetencyCriterion #716), and Django adminregistration.
Testing
Every acceptance criterion in #641 has a test, written before the implementation. Tests are named
for the behavior they assert rather than the mechanism, and
test_criteria_trees.pycoverswhole-tree deletion, so a test proves the bad outcome is avoided rather than only that a cascade
fired. The deletion paths also run under MySQL's collector semantics while still on SQLite, by
setting
can_defer_constraint_checksto false, which makes this class of bug visible in the fastlocal suite instead of only in CI.
Verified against both backends, since a green SQLite run is not evidence for the
scope_codework: 887 passed on SQLite, 888 on a real MySQL 8.
mypy,pylint,pycodestyle,pydocstyleandisortare clean,lint-importskeeps both contracts, andmakemigrations --checkreports no driftin
openedx_learning. No# noqa,# pylint: disable,# type: ignoreorTODOanywhere in thediff.
make pii_checkstill fails at two pre-existing lint conflicts (openedx_content.Draft,openedx_content.PublishableEntityVersion), both identical onmainand in code this PR does nottouch. The models added here are annotated, including the three
Historical*modelsdjango-simple-historygenerates.