[BE] Extend competency criteria cleanup to course, section, subsection, & tag deletion/archiving #895

When a course author deletes a subsection, this fires a "block-deleted" event, which is heard by the Tagging code, which then deletes any relevant ObjectTag records, or archives them instead when a learner status depends on them (#779 and #782).

1. If there was no learner status related to that association, then this also deletes the relevant CompetencyCriteria, through the existing cascade. The CompetencyCriteriaGroups they leave empty are not touched today. This ticket adds a criterion-deleted listener in the CBE applet on the CompetencyCriterion's post_delete signal that reuses [BE] Delete Competency Criterion #674's upward walk to delete those groups, or archive them per [BE] Delete Competency Criterion #674's rules.
2. If there is a learner status related to that association, the ObjectTag is archived instead ([BE] Branch tag association deletes between archiving and hard delete #779), and today nothing archives the CompetencyCriterion behind it or its groups. This ticket adds an association-archived listener that reacts when an association is archived, archives its CompetencyCriteria, and archives each group above them that is left with no unarchived children, per [BE] Delete Competency Criterion #674's rules. An archived rule stays archived if [BE] Branch tag association deletes between archiving and hard delete #779 later revives the association.

When a course author deletes a section, Studio deletes every subsection inside it, but the Tagging code has a bug that makes it only delete the ObjectTags associated with that section and does not also remove the ObjectTags associated with its subsections. This ticket adds a section-delete handler in openedx-platform that runs before the section is removed, while its contents can still be read, and deletes the ObjectTag records of every block inside it. From there, the algorithm described above runs for each subsection exactly as if it had been deleted on its own.

When an operator deletes a course with the delete_course management command, the command deletes the course without deleting its content, which also means that all of its ObjectTag records continue existing, which is the same bug as the section case above. This ticket adds a course-delete handler in openedx-platform that runs when a course is deleted and deletes every ObjectTag record belonging to that course. From there, the algorithm described above runs for each tagged subsection exactly as if it had been deleted on its own.

A CourseRun is a record in openedx-core's catalog app as opposed to the course and its content ("blocks"), which are records in modulestore. Deleting a course with the delete_course management command also deletes its CourseRun record, unless learner status recorded in that course refuses the delete, and a Django admin can delete a CourseRun directly. Deleting a CourseRun already deletes that course's course-level CompetencyCriteriaGroup, every group beneath it, and the CompetencyCriteria in them, by cascade. It leaves two things behind:

1. The ObjectTags stay on their subsections, which still exist in Studio when an administrator deleted only the course run, with no rule behind them. This ticket removes those associations too, when no other CompetencyCriterion still uses them, so a subsection does not keep showing a competency tag that nothing backs.
2. The competency's shared root group may be left with nothing beneath it, and nothing cleans it up today, because the criteria were deleted through their groups rather than through an ObjectTag, so the listener described above never ran. This ticket adds a group-deleted listener, on the CompetencyCriteriaGroup's delete signals, that removes that root group, or archives it per [BE] Delete Competency Criterion #674's rules.

When a course author re-imports a course, the tag import today removes and re-applies the tags of every tagged block, even when nothing changed. With no un-archiving in scope, that would permanently archive every rule with learner progress on every re-import, and it deletes every rule without learner progress for no reason. This ticket changes the course tag import in openedx-platform to apply only the differences between the export and the course, so an unchanged re-import touches no association. Only a tag the export dropped is removed or archived, and the criterion-deleted and association-archived listeners handle it as they would any other removal. Content added after the export was taken is dropped from the course by the import, and the tags and rules behind it are removed or archived like any other dropped tag, so no rule is left behind a block that is no longer in the course.

When someone re-imports a taxonomy and the import omits a competency tag that is associated with a subsection:

1. If no learner status depends on any of that tag's associations, the Tag is deleted, and because the CompetencyCriteriaGroup's foreign key to the Tag cascades, the competency's entire group tree and every CompetencyCriterion in it are deleted with it. The ObjectTags are not deleted and stay on their subsections with their tag reference cleared, which is existing Tagging behavior. Nothing is left for this ticket to clean up; the criterion-deleted listener must ignore deletions that start from a Tag.
2. If a learner status depends on one of its associations, today the learner status protection refuses the delete and the import fails at that tag. [BE] Branch tag and taxonomy deletes between archiving and hard delete #780 changes that to archive the Tag instead, which leaves the competency's groups and rules live. This ticket adds a tag-archived listener that reacts when [BE] Branch tag and taxonomy deletes between archiving and hard delete #780 archives a competency tag, or a whole taxonomy of competencies, and archives every CompetencyCriterion for each affected competency and every group in its tree. The listener needs [BE] Branch tag and taxonomy deletes between archiving and hard delete #780's archive path to announce which tags or taxonomies it archived, the same kind of hook the association-archived listener needs from [BE] Branch tag association deletes between archiving and hard delete #779. [BE] Branch tag and taxonomy deletes between archiving and hard delete #780 as written branches only in the taxonomy's delete-tags method and the taxonomy delete endpoint, and the taxonomy import removes tags through a different call. This ticket also routes the import's tag removal through that method, so [BE] Branch tag and taxonomy deletes between archiving and hard delete #780's branch applies on re-import.

A taxonomy re-import with no learner status needs nothing of its own: the existing cascade from the Tag removes the competency's groups and rules. This ticket restores nothing it archives. #717 reactivates an archived root when an author adds a new rule under it, and #696 decides whether a course import reactivates an archived course-level group.

This is a suggested approach, not the source of truth. The User Story, Description, and Acceptance Criteria define what must be true when the work is done.

Summary

In openedx-core, four listeners in a new CBE signal_handlers.py keep competency criteria and groups consistent with the tagging and catalog layers. The criterion-deleted and group-deleted listeners react to Django deletes that cascade into CBE. The association-archived and tag-archived listeners react to new tagging signals that #779 and #780 send when they archive an association, a tag, or a taxonomy. Nothing this ticket archives is ever un-archived. In openedx-platform, the section-delete and course-delete handlers make deleting a section in Studio, or a whole course with the delete_course management command, delete the ObjectTags of everything inside it. Course tag import applies only the differences in tags.csv, so an unchanged re-import leaves every rule in place.

Suggested split:

• PR A is the openedx-core deletion side (items 1 and 3), and needs only [BE] Delete Competency Criterion #674.
• PR B is the three openedx-platform changes. It needs nothing new from openedx-core to merge, and merges alongside [BE] Branch tag association deletes between archiving and hard delete #779 and no later than PR C.
• PR C is the openedx-core archive side (items 2 and 4, the tagging signals, and the taxonomy import change), and waits for [BE] Branch tag association deletes between archiving and hard delete #779 and [BE] Branch tag and taxonomy deletes between archiving and hard delete #780.

Approach

PR A, openedx-core: the criterion-deleted and group-deleted listeners.

1. Module and loading. Add src/openedx_learning/applets/cbe/signal_handlers.py for all four listeners, and load it from a new LearningConfig.ready() in src/openedx_learning/apps.py. Use the same unused-import,import-outside-toplevel disable and comment as src/openedx_tagging/apps.py. The module imports from cbe/api.py, which imports the models, so it cannot live in the models package.
2. Error handling and logging, shared by every listener.
   • Every listener runs its work inside transaction.atomic(), which is a savepoint inside the transaction that triggered it. That block is wrapped in try/except Exception with the approved broad-exception-caught disable and a comment saying why, as src/openedx_tagging/import_export/api.py has.
   • On error, the listener logs a warning with exc_info that names the groups it left in place and the triggering ids, then returns. The savepoint undoes any partial work, and the triggering delete, archive, or import still completes.
   • Letting the error propagate would roll back the trigger. Course import calls the tagging API directly, so the import itself would fail.
   • [BE] Delete Competency Criterion #674's _cascade_from_group() returns nothing. A listener therefore records the ids and archived flags of the groups it may touch before calling it, and re-reads them afterward.
   • It logs one info line per group that is now gone or newly archived, naming the group, the action, and the triggering ids. It never logs per criterion, because one Studio deletion can remove many rules at once.
3. The criterion-deleted listener. This is a post_delete receiver for CompetencyCriterion that acts only when the delete started on an ObjectTag.
   • The origin check. Django passes the instance or queryset that .delete() was called on as origin. The check is isinstance(origin, ObjectTag), or isinstance(origin, QuerySet) and issubclass(origin.model, ObjectTag).
   • Origins it ignores. Every other origin returns at once:
     • A criterion: [BE] Delete Competency Criterion #674's delete_competency_criterion() runs its own walk.
     • A group: [BE] Build endpoint for removing a Competency Criteria Group #675, and the helper's own deletes.
     • A Tag or Taxonomy: the competency's whole group tree cascades with the tag, so nothing is left to clean up.
     • A CourseRun: the group-deleted listener handles it.
   • What it does. It looks up the criterion's former group by instance.group_id. If that group is already gone, it returns, because an earlier call or the other listener may have removed it. Otherwise it calls _cascade_from_group(group, "delete") from .api, unchanged.
   • Check when [BE] Branch tag association deletes between archiving and hard delete #779 merges. Confirm that [BE] Branch tag association deletes between archiving and hard delete #779's overridden delete() methods still hard-delete unlocked rows through Django's collector with an ObjectTag origin. A raw SQL delete would never reach this receiver.
4. The group-deleted listener. This is a post_delete receiver for CompetencyCriteriaGroup that acts only when origin is a CourseRun instance or queryset. That happens from Django admin, and from delete_course_run() when a course is deleted with the delete_course management command (step 7). CourseRun.catalog_course is PROTECT, so no other delete cascades into a course run.
   • What it does. For each deleted course-level group (the one with course_id set), it calls _cascade_from_group() with "delete" on that group's parent. The parent is the shared root, which survives the delete. It then removes the ObjectTags that no remaining criterion uses.
   • Why it needs a pre_delete partner. Django deletes the criteria before the groups, so the post_delete instance has no criteria left to read. A pre_delete receiver on the same model, with the same filter, reads the object_tag_id of every criterion beneath the course-level group and stores the ids on the instance. Django passes that same instance to post_delete.
   • How the ObjectTags are removed. The removal reuses [BE] Stop an assignment showing as tagged when the last competency rule measuring it is archived #817's _reconcile_object_tags(). That helper skips ids that any non-archived criterion still uses, and removes the rest through openedx_tagging.api.tag_object(). If [BE] Stop an assignment showing as tagged when the last competency rule measuring it is archived #817 has not merged yet, this PR adds the helper with [BE] Stop an assignment showing as tagged when the last competency rule measuring it is archived #817's semantics.
   • Tolerating the other listener. Both steps must cope with the course-delete handler having run first. The listener looks up the root with filter(...).first(), and the helper reads its candidates with filter(id__in=...), never get(), so ids that are already gone are skipped.
   • No locked associations here. A course run whose groups or criteria hold learner status cannot be deleted, because the status rows protect them. So no association reached here is locked.

PR B, openedx-platform: the section-delete handler, the course-delete handler, and the course tag import change, all in openedx/core/djangoapps/content_tagging/.

5. The section-delete handler. This is a receiver for SignalHandler.pre_item_delete, in handlers.py. Split modulestore sends that signal before it removes a block, while the subtree is still readable. XBLOCK_DELETED fires only after the subtree is gone, and carries only the deleted block's key.
   • What it does. It loads the block with modulestore().get_item(usage_key.for_branch(None), depth=None), walks it with get_children(), and calls the existing _delete_tags() for each descendant. It skips the block itself, whose tags delete_tag_xblock already deletes.
   • Missing block. On ItemNotFoundError it logs a warning and returns, as handle_item_deleted in cms/djangoapps/contentstore/signals/handlers.py does.
   • Why get_children(). That handler uses yield_dynamic_block_descendants, which appears to return no children for A/B test and library content blocks unless they are bound to a user. This is inferred from get_dynamic_block_children in common/djangoapps/util/block_utils.py, and a test below settles it.
   • Firing twice. The signal fires once per branch for sections and subsections, and the second pass finds nothing left to delete.
   • The cost of running early. This is the same tradeoff handle_item_deleted makes: if the modulestore delete then fails, the descendants' unlocked associations and rules are already gone.
6. The course-delete handler. This is a receiver for SignalHandler.course_deleted, in handlers.py. Split's delete_course() sends that signal after it removes the course index. It is the only signal course deletion sends, because course deletion never calls delete_item.
   • Finding the course's tags. The course structure is unreadable by then, so the handler finds the course's tagged objects in the tagging tables. It takes the distinct object_id values of ObjectTag rows that equal the course key, which are course-level tags, or that start with str(course_key).replace("course-v1:", "block-v1:", 1) + "+type@".
   • Why the +type@ delimiter. Without it, deleting course-v1:A+B+C would also match the blocks of course-v1:A+B+C2.
   • Deleting them. It calls api.delete_object_tags() once per object id, and logs one line for the course.
   • No new openedx-core API. Querying ObjectTag directly follows get_all_object_tags() in api.py, which does the same because the tagging API has no by-course lookup.
7. The two receivers of a course deletion. The same course_deleted signal also reaches _listen_for_course_delete in openedx/core/djangoapps/content/course_overviews/signals.py, which deletes the course's CourseRun through delete_course_run(). The order of the two receivers is not guaranteed, and the end state is the same either way.
   • If the CourseRun is deleted first. Its cascade deletes the course's rules and its course-level and bottom-tier groups directly. The group-deleted listener then removes or archives the root, and removes the rules' ObjectTags. The course-delete handler then deletes the remaining tags, which have no rule behind them.
   • If the course-delete handler runs first. The criterion-deleted listener removes each emptied group up to the root, and the CourseRun delete then finds no groups.
   • What the log records. The log records only cleanup, never groups that a CourseRun delete cascades away directly. So the first order logs only the root, and the second order logs every removed group.
   • When the course has learner status. delete_course_run() raises ProtectedError, which send_robust logs. The course deletion completes and the CourseRun row stays. What remains under it is the rules that hold learner status and the groups above them, archived by the association-archived listener.
   • Failures. All of these signals use send_robust, so a failure in either handler is logged and the deletion proceeds.
   • Before [BE] Branch tag association deletes between archiving and hard delete #779 has merged. Until [BE] Branch tag association deletes between archiving and hard delete #779 has merged, deleting the ObjectTag of a rule that holds learner status raises ProtectedError, which stops either handler at that block.
8. The course tag import change. set_exported_object_tags() in api.py, which course import calls once per tags.csv row, applies only the differences instead of deleting every tag on the block and re-applying them.
   • What changes. Drop its opening oel_tagging.delete_object_tags(content_key_str). The loop after it already calls oel_tagging.tag_object() once per taxonomy in the row. tag_object() keeps the associations whose values are listed, adds new ones, and removes only the omitted ones through [BE] Branch tag association deletes between archiving and hard delete #779's archive-or-delete branch.
   • Taxonomies the row no longer carries. For each taxonomy the block still has tags in that the row does not list, call oel_tagging.tag_object(object_id, taxonomy, [], taxonomy_export_id=...). Find those taxonomies with oel_tagging.get_object_tags(), matched by export id. This keeps today's rule that an import replaces a block's tags. The tagging API has no taxonomy-scoped delete_object_tags(), and tag_object() with an empty list does that job.
   • Result. An unchanged re-import deletes and archives nothing, so no rule or group changes. A re-import still removes the associations the export no longer carries, and the criterion-deleted and association-archived listeners follow those removals. Blocks that tags.csv does not list can only have been added after the export, and the import has already dropped them from the course tree: after the rows are processed, the import finds the course's remaining tagged objects with the same prefix query the course-delete handler uses, and calls oel_tagging.delete_object_tags() for each one the file did not list. An export with no tags.csv keeps today's behavior and changes nothing.
   • Why. Without this change, every re-import would delete every rule without learner status and permanently archive every rule with it. [BE] Branch tag association deletes between archiving and hard delete #779 revives the association when the tag is re-applied, but nothing un-archives a rule.

PR C, openedx-core: the association-archived and tag-archived listeners, after #779 and #780. The tagging hooks below land in openedx_tagging code, but they are this ticket's work.

9. The tagging signals. Add src/openedx_tagging/signals.py with three Django signals:

   • object_tags_archived carries object_tag_ids. Send it from the one function that [BE] Branch tag association deletes between archiving and hard delete #779 routes both delete() overrides through.
   • tags_archived carries tag_ids. Send it from [BE] Branch tag and taxonomy deletes between archiving and hard delete #780's archive branch in Taxonomy.delete_tags().
   • taxonomies_archived carries taxonomy_ids. Send it from [BE] Branch tag and taxonomy deletes between archiving and hard delete #780's taxonomy archive branch behind TaxonomyView.perform_destroy(). It is a separate signal because archiving a taxonomy does not set archived on its tags ([BE] Branch tag and taxonomy deletes between archiving and hard delete #780). A tags_archived payload for it would state something untrue, and [BE] Fan out the content-tag-changed event when a taxonomy is archived #781 needs to tell the two cases apart.

   Rules for all three signals:

   • Each is sent with send() right after its bulk .update(), inside the same transaction, so a rollback also undoes the CBE changes.
   • Each carries only the ids whose archived value actually changed, and is not sent when none did.
   • The signals state a tagging fact and know nothing about competencies.
   • They are new public surface of openedx_tagging, so their docstrings define the payload.

10. The taxonomy import change. In src/openedx_tagging/import_export/actions.py, DeleteTag.execute() looks the tag up with _get_tag(), returning as today when it no longer exists. It then calls self.taxonomy.delete_tags([tag.value], with_subtags=True), so [BE] Branch tag and taxonomy deletes between archiving and hard delete #780's archive-or-delete branch also applies on re-import.

    • Why with_subtags=True. It matches today's cascade. An earlier UpdateParentTag action in the same plan has already moved every child that the file keeps, so only children that the file also omits are removed with the tag.
    • Tags already archived. A tag that [BE] Branch tag and taxonomy deletes between archiving and hard delete #780 already archived earlier in the same import must be skipped, not deleted again. _get_tag() reads Taxonomy.tag_set, a reverse manager that [BE] Exclude archived tagging records from read paths and block edits to them #778's read-path filters may not cover, so check the tag's archived flag here rather than relying on [BE] Exclude archived tagging records from read paths and block edits to them #778 to hide it.
    • Search events. The existing pre_delete receiver in src/openedx_tagging/signal_handlers.py already handles a queryset origin, so search events are unchanged.

11. The association-archived listener. It receives object_tags_archived.

    • Archiving the rules. It archives the non-archived criteria on those ObjectTags with simple-history's bulk_update_with_history() and a change reason, never QuerySet.update(), so history records the archive (ADR 0003, Decision 1).
    • Archiving the groups. It then calls _cascade_from_group(group, "archive") once per distinct group. That walk stops at the first group that still has a live child.
    • When there is nothing to do. It finds nothing to do when the criteria are already archived. That covers an archive [BE] Stop an assignment showing as tagged when the last competency rule measuring it is archived #817 caused, because [BE] Stop an assignment showing as tagged when the last competency rule measuring it is archived #817 archives the criterion before it calls tag_object(). It also covers a section delete that archives the same association once per branch.
    • When the association is revived. When [BE] Branch tag association deletes between archiving and hard delete #779 later revives the association because an author re-applied the tag, nothing reacts. The rule and its groups stay archived, and the author creates a new rule, as [BE] Stop an assignment showing as tagged when the last competency rule measuring it is archived #817 already expects.

12. The tag-archived listener. It receives tags_archived and taxonomies_archived. It archives every non-archived criterion and group whose competency tag is one of those tags, or belongs to one of those taxonomies, writes them as in step 11, and logs each group. It filters on tag_id and tag__taxonomy_id, which [BE] Exclude archived tagging records from read paths and block edits to them #778's hiding of archived tags and taxonomies from reads does not affect.

Tests in openedx-core for PR A, in tests/openedx_learning/applets/cbe/test_signal_handlers.py, on a tree of a root, a course-level group, and leaf groups:

• Instance origin. Removing a tag through tag_object() removes the leaf, the course-level group, and the root, and logs one line for each of the three. A sibling leaf that keeps a criterion stops the walk, and then nothing is logged.
• Queryset origin. delete_object_tags() is used by Studio deletion. On an ObjectTag that has criteria in two leaves under one course-level group, it removes both leaves, the course-level group, and the root, and logs one line per removed group.
• Two criteria, one leaf. One ObjectTag that has two criteria in the same leaf removes and logs that leaf once.
• Archive instead of remove. A leaf that also holds a criterion archived earlier is archived instead of removed. So is a leaf with its own StudentCompetencyCriteriaGroupStatus row. A course-level group left holding only archived groups is archived too.
• Two competencies, one ObjectTag. The ObjectTag has a criterion for competency A, alone in its leaf and course-level group, and one for competency B that shares its leaf. Deleting it removes and logs A's leaf and course-level group. B's leaf keeps its other criterion and is not logged.
• Course run delete. Delete a CourseRun, as an instance and as a queryset, whose course-level group is the root's only child. The root is removed and logged.
  • The root is unchanged and not logged when another course's group remains under it.
  • The root is archived and logged when it has its own status row.
  • ObjectTags used only by the deleted run's criteria are removed. One that a criterion in another course run also uses is kept.
• Both orders of a course deletion end with the same rows.
  • Deleting the course's ObjectTags and then its CourseRun logs each removed group, and the CourseRun delete finds nothing to do.
  • Deleting the CourseRun and then the remaining ObjectTags logs only the root.
  • Neither order raises or logs a warning.
• Helper failure. With the helper patched to raise, delete_object_tags() and the course run delete both succeed, the groups stay, and the warning names them.
• Ignored origins. Deletes that start on a criterion (including delete_competency_criterion()), a group, a Tag, or a Taxonomy never call the helper, and log nothing.

Tests in openedx-platform for PR B. They assert on ObjectTags only.

• In tests/test_handlers.py, with ENABLED_SIGNALS = ['pre_item_delete', 'course_deleted']:
  • Deleting a section deletes the tags of its subsections, units, and components, including a component inside an A/B test block and one inside a library content block. Tags on blocks outside the section stay.
  • Deleting a unit deletes its components' tags.
  • Deleting course course-v1:A+B+C deletes the tags on every block in it and on the course key itself. Tags on the blocks of course-v1:A+B+C2 stay.
  • A failure in either handler is logged, and the deletion still succeeds.
• In tests/test_api.py:
  • Importing a course's own tags.csv back into it leaves every ObjectTag row in place, with the same ids.
  • A tag the row no longer carries is removed from that block.
  • A taxonomy the row no longer carries is removed from that block.
  • A block absent from the file loses its tags, and an export with no tags.csv changes nothing.
• Once [BE] Branch tag association deletes between archiving and hard delete #779 has merged: a locked association inside a deleted section or course, or dropped by an import, is archived, not deleted.

Tests in openedx-core for PR C, in the same CBE module and in tests/openedx_tagging/:

• Signals. Each tagging signal is sent once per operation, after its update, with only the ids whose state changed. It is not sent when nothing changed.
• Archiving a locked association. Removing it with tag_object() archives its criterion and each group left with no live children, and logs each archived group.
• Revival. Re-applying the tag revives the association, and leaves the criterion and groups archived.
• [BE] Stop an assignment showing as tagged when the last competency rule measuring it is archived #817 no-op. Take a criterion archived through delete_competency_criterion() whose association is then archived. Nothing more changes and nothing is logged.
• Idempotence. Sending object_tags_archived twice for the same ids changes and logs nothing the second time.
• Taxonomy import.
  • An import with replace that omits a competency tag with a locked association completes, archives the tag, and archives that competency's tree.
  • Omitting an unlocked tag still deletes it and its tree, and no listener logs.
  • Omitting a parent and its child in the same file succeeds.
• Taxonomy archive. Archiving a taxonomy through the taxonomy delete endpoint archives every competency tree in it, and leaves criteria and groups that were already archived unchanged.
• Listener failure. With a listener patched to raise, the triggering archive commits, the criteria are unchanged, and the warning is logged.

Out of scope:

• Any frontend change.
• Changing [BE] Branch tag association deletes between archiving and hard delete #779's or [BE] Branch tag and taxonomy deletes between archiving and hard delete #780's archive-or-delete rules. This ticket only adds signals to them.
• Un-archiving anything this ticket archives, when [BE] Branch tag association deletes between archiving and hard delete #779 revives an association or when a tag or taxonomy is un-archived. [BE] Enforce archived in Competency Criteria creation #717's reactivation of an archived root is unaffected.
• [FE] Show competency criteria whose gradeable subsection has been deleted #826's display of unresolvable criteria.
• Backfilling groups that are already empty.
• ObjectTags left with a null tag after a Tag is hard-deleted, which is existing tagging behavior.

Files to create and modify

Additional context