codecov: widen project threshold to 0.5% for measurement noise - #212
Conversation
The project status is now a required check, so a 0% threshold is unsafe: the measurement is not deterministic. Commit 78bf583 reported 29,373 hits and later 29,512 (+0.41 points) with no code change, because repeat uploads are unioned and the integration suite covers slightly different paths run to run. At 0% that noise fails PRs for no reason. 0.5% still catches real erosion by a wide margin -- the Dashboard.Ui shortfall this gate exists to prevent is ~2.5 points -- while staying far tighter than the original 2%, which is what let coverage drift unnoticed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughThe coverage policy raises the automatic project ratchet threshold from 0% to 0.5%, marks patch coverage as required, and updates comments and documentation to explain the measurement noise floor. ChangesCoverage gate policy
Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #212 +/- ##
==========================================
+ Coverage 88.06% 88.09% +0.02%
==========================================
Files 1022 1022
Lines 33476 33476
Branches 2790 2790
==========================================
+ Hits 29480 29490 +10
+ Misses 3133 3126 -7
+ Partials 863 860 -3 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|



Follow-up to #211, now that
codecov/projectis a required check on master.Problem
projectwas set toauto+ 0% threshold — any decrease fails. But the measurement is not deterministic: commit78bf5839was observed reporting 29,373 hits, then 29,512 (+0.41 points) with no code change. Repeat uploads get unioned, and the integration suite covers slightly different paths run to run.With
projectrequired, that noise blocks PRs for no reason.Change
threshold: 0%→0.5%.Still far tighter than the original
2%(which is what allowed coverage to drift downward unnoticed, sinceautorebaselines every merge), and comfortably below the size of a real regression — the Dashboard.Ui shortfall this gate exists to prevent is ~2.5 points.Docs updated with the reasoning so the number isn't mistaken for arbitrary slack.
🤖 Generated with Claude Code
Summary by CodeRabbit
Documentation
Chores