File: .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py
Overall Score: N/A (this is a rubric tooling bug, not a workshop content issue) — corpus mean affected: 6.15 / 10.0 across 92 files
Flagged Dimensions:
| Dimension |
Score |
Benchmark |
Delta |
| checkpoint_quality (corpus-wide) |
0.0 / 10.0 on 87 of 92 files |
present + ≥4 items scores up to 10.0 |
-10.0 pts on 87/92 pages |
Root Cause (≤ 2 sentences):
CHECKPOINT_RE = re.compile(r"##\s+✅\s*Checkpoint", re.IGNORECASE) only matches a literal ✅ Unicode emoji character immediately after ##, but per AGENTS.md and actual practice in 87 of 96 workshop files, checkpoints are written as ## :white_check_mark: Checkpoint (GFM emoji shortcode), which the regex never matches. Only 5 files happen to use the literal Unicode emoji instead of the shortcode, so those 5 are the only ones correctly scored.
Evidence (quoted from the file):
CHECKPOINT_RE = re.compile(r"##\s+✅\s*Checkpoint", re.IGNORECASE)
Compare to the actual convention used almost everywhere, e.g. in workshop/07-your-first-workflow.md:
## :white_check_mark: Checkpoint
Learning Science Rationale:
This is a measurement-validity issue rather than a pedagogical one: checkpoint_quality carries the joint-highest rubric weight (2.0), so a systematic false negative on 87/92 files means the entire scorecard's overall_score, part means, std devs, and per-file rankings are computed against corrupted data for that dimension. Any curriculum-quality conclusions drawn from the current scores (which pages "need more checkpoints," which parts are weakest) are unreliable until this is fixed, since virtually every page is being scored as if it had no checkpoint at all when nearly all of them do.
Improvement Prompt (for an agent):
Open .github/skills/curriculum-quantitative-assessment/curriculum_assessment.py and find CHECKPOINT_RE = re.compile(r"##\s+✅\s*Checkpoint", re.IGNORECASE). Update this regex (and any other place in the file that matches the same literal ✅ pattern, e.g. search for "✅" throughout the file) so it also matches the GFM emoji shortcode convention `:white_check_mark:` used in nearly all workshop files, for example: re.compile(r"##\s+(?:✅|:white_check_mark:)\s*Checkpoint", re.IGNORECASE). Also check the evidence-gathering logic near "activity:checkpoint" (around line 157-159) for the same literal-emoji-only pattern and update it consistently. After fixing, re-run the rubric script against workshop/*.md and confirm has_checkpoint is now true for files that contain "## :white_check_mark: Checkpoint" (spot check workshop/07-your-first-workflow.md, workshop/04-github-actions-intro.md, and workshop/05-agentic-workflows-intro.md). Do not change the checkpoint_quality scoring formula itself, only the detection regex.
Expected Score After Fix: Corpus mean overall_score should rise meaningfully across nearly all 92 files once checkpoint_quality is scored correctly (previously undercounted); exact new corpus mean depends on re-running the rubric.
Generated by 🔬 Curriculum Quality Evaluator · copilot · auto · 95.4 AIC · ⌖ 8.01 AIC · ⊞ 10.7K · ◷
File:
.github/skills/curriculum-quantitative-assessment/curriculum_assessment.pyOverall Score: N/A (this is a rubric tooling bug, not a workshop content issue) — corpus mean affected:
6.15 / 10.0across 92 filesFlagged Dimensions:
Root Cause (≤ 2 sentences):
CHECKPOINT_RE = re.compile(r"##\s+✅\s*Checkpoint", re.IGNORECASE)only matches a literal ✅ Unicode emoji character immediately after##, but perAGENTS.mdand actual practice in 87 of 96 workshop files, checkpoints are written as## :white_check_mark: Checkpoint(GFM emoji shortcode), which the regex never matches. Only 5 files happen to use the literal Unicode emoji instead of the shortcode, so those 5 are the only ones correctly scored.Evidence (quoted from the file):
Compare to the actual convention used almost everywhere, e.g. in
workshop/07-your-first-workflow.md:Learning Science Rationale:
This is a measurement-validity issue rather than a pedagogical one: checkpoint_quality carries the joint-highest rubric weight (2.0), so a systematic false negative on 87/92 files means the entire scorecard's overall_score, part means, std devs, and per-file rankings are computed against corrupted data for that dimension. Any curriculum-quality conclusions drawn from the current scores (which pages "need more checkpoints," which parts are weakest) are unreliable until this is fixed, since virtually every page is being scored as if it had no checkpoint at all when nearly all of them do.
Improvement Prompt (for an agent):
Expected Score After Fix: Corpus mean
overall_scoreshould rise meaningfully across nearly all 92 files once checkpoint_quality is scored correctly (previously undercounted); exact new corpus mean depends on re-running the rubric.