fix(core): stopword & rare-literal scoring noise (6F.9, A15)#51
Merged
officialCodeWork merged 1 commit intoJul 15, 2026
Merged
Conversation
…ng (6F.9, A15) Self-found validating 0.4.0 on Grafana's frontend: `find "Find silences by matcher"` ranked OrderBySection (renders a bare "BY") top-1 over SilencesFilter, because a lone stopword is a rare literal with high IDF; and `find "The"` returned confident matches on "the". New isLowSignal() in core/text.ts (extends the A14 hasMatchSignal guard): true when a normalized string is empty, punctuation-only, or entirely stopwords (the stopword set is folded through foldPlural so it compares to normalized text). matchComponents drops stopword-only query terms and skips stopword-only rendered-text targets, so the exact-phrase component wins and a stopword-only query declines no-signal. Verified on the real Grafana graph: "Find silences by matcher" -> SilencesFilter top-1 (OrderBySection gone), "The" -> no-signal, and resolve on a stopword-heavy ticket went from confidently-wrong to an honest no-signal. New failure mode A15 + fixture a15-stopword-noise; 5 unit tests. A separate finding (HTML-entity rendered text -> junk tokens) is filed as A16/6F.10 — an extraction bug, out of scope for a scoring PR. Tests 221 green; eval 297/0/0/0, gate OK, all metrics 1.000. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Found while self-validating 0.4.0 on Grafana's frontend (a big real RTK codebase). Two ranking bugs, one root cause — low-specificity tokens treated as real signal:
find "Find silences by matcher"ranked OrderBySection (renders a bareBY) top-1 over SilencesFilter, because a lone stopword is a rare literal with high IDF.find "The"returned confident matches on "the".Fix
New
isLowSignal(normalized)incore/text.ts— extends the A14hasMatchSignalguard: true when a string is empty, punctuation-only, or entirely stopwords (the stopword set is folded throughfoldPluralso it compares equal to normalized text).matchComponents:BYcan't be a match target).IDF and the combination bonus are untouched — this just sets the noise floor.
Verified on the real Grafana graph (not just the fixture)
Find silences by matcher→ SilencesFilter top-1, OrderBySection gone ✅The→no-signal✅resolveon a stopword-heavy ticket went from confidently-wrong → honestno-signal✅Tests
a15-stopword-noise(aBYcomponent, an exact-phrase component, athe-rendering component)isLowSignal, 3 matcher)Follow-up (not in this PR)
Validation surfaced a separate bug — HTML-entity rendered text (
→"nbsp","→"34") creates spurious matches; numeric entities let gibberish sharing those digits match. That's an extraction bug, not scoring, so it's filed as A16 / step 6F.10 and kept out of this PR.🤖 Generated with Claude Code