Skip to content

fix(prompts): scrub site-specific vocabulary and contradictions (plan 021)#92

Open
DavertMik wants to merge 1 commit into
mainfrom
refactor/prompt-generality-scrub
Open

fix(prompts): scrub site-specific vocabulary and contradictions (plan 021)#92
DavertMik wants to merge 1 commit into
mainfrom
refactor/prompt-generality-scrub

Conversation

@DavertMik

Copy link
Copy Markdown
Contributor

Plan 021 — Scrub site-specific vocabulary and contradictions from prompts

CLAUDE.md's cardinal prompt rule is "General, Not Example-Driven". The prompts violated it. This is a prompt-only change (the single code change is the rule composition in Step 4). Every replacement uses the plan's exact wording.

What changed

  1. Positional-index contradiction resolved. rules.ts said both "never use positional indices [1], [2]" (line 22) and "XPath should use positional indices [1], [2], [3]" (<xpath_rules>). Now one policy: positional indices are a last-resort disambiguator.
  2. Debug-session leftovers removed from rules.ts: imr_manual12 project slug, md-icon-cog/md-icon-plus icon classes, a[href*="requirements"], :has-text("Manual"), "New Template" → neutral placeholders (icon-settings/icon-add, /items/12345, "Active", "New Item").
  3. Domain-neutral vocabulary in pilot/tester/planner examples: Testomatio's "test suite"/"Create Suite"/"Template API Testing"/"Matched tests 0" → generic item/post/"Matched 0" (matching the existing house style, e.g. "New user 'john@example.com'…").
  4. locatorRule composed from named partscontextSimplificationRule + locatorRuleWithoutContextSimplification are now exported; Driller imports the pre-composed variant instead of .replace(/<context_simplification>.../)-stripping the constant. locatorRule verified byte-identical to the original for all its other consumers (Tester/Pilot system prompts).
  5. Duplicate rule blocks removed. capabilityGroundingRule + dataProtectionRules were injected in Tester's system prompt and its scenario user-message (same conversation), likewise Pilot's system prompt and planTest. Removed the user-message copies (verified each targets a conversation that already carries the system copy). Pilot's verdict system prompt keeps its own copy — separate conversation.

Verification

  • bun test tests/integration/63 pass / 0 fail (the gate for prompt changes — real prompt assembly through aimock).
  • bun test tests/unit764 pass / 0 fail.
  • bun run lint → clean; tsc in touched files neutral (9 = 9).
  • grep -rn "imr_manual\|md-icon" src/ → 0; driller.ts has no .replace( on a rules constant; locatorRule === original verified in a Bun eval.

🤖 Generated with Claude Code

…cific examples (plan 021)

Scrub site-specific vocabulary and self-contradiction from prompts per the
CLAUDE.md "General, Not Example-Driven" rule:
- rules.ts stated one positional-index policy (was: line 22 forbade [1]/[2]
  outright while <xpath_rules> mandated them).
- Removed debug-session leftovers (imr_manual12 slug, md-icon-* classes,
  requirements href, "New Template", :has-text("Manual")) -> neutral
  placeholders (icon-settings/icon-add, /items/12345, "New Item", "Active").
- Domain-neutral example vocabulary in pilot/tester/planner: test-suite ->
  item/post, "Matched tests 0" -> "Matched 0", etc.
- Composed locatorRule from named parts (contextSimplificationRule +
  locatorRuleWithoutContextSimplification); driller imports the pre-composed
  variant instead of regex-stripping the constant. locatorRule verified
  byte-identical to the original for all other consumers.
- Removed the duplicate capabilityGroundingRule/dataProtectionRules injections
  from Tester's scenario block and Pilot's planTest (both already present in
  the same conversation's system prompt).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

Explorbot Self-Regression

Commit b773910 · run

Scenario Result Attempts Duration
basic (native) FAIL 3/3 19m

Attempt details

  • basic (native) attempt 1 — FAIL: login evidence: PASS (post-login plan=true, post-login research=true); research: PASS (files=5, wellFormed=true, keywords=7/3); scenarios: PASS (tests=13/5, features=5/3); tests passed: FAIL (4 passed, 1 failed (reporter: 4 passed, 1 failed))
  • basic (native) attempt 2 — FAIL: login evidence: FAIL (post-login plan=false, post-login research=false); research: FAIL (files=0, wellFormed=false, keywords=0/3); scenarios: FAIL (tests=0/5, features=0/3); tests passed: FAIL (no Results line)
  • basic (native) attempt 3 — FAIL: login evidence: PASS (post-login plan=true, post-login research=true); research: PASS (files=7, wellFormed=true, keywords=7/3); scenarios: PASS (tests=8/5, features=4/3); tests passed: FAIL (3 passed, 2 failed (reporter: 3 passed, 2 failed))

Session analysis — basic (native):

Session Analysis

The Issues page core flows (create, status filter, search) work correctly. The label filter test failed due to automation issues, and form validation displays errors in the wrong location.

Coverage

  • Pages: /issues
  • Features: Create issue, filter by status, filter by label, search issues, form validation

What works

  • Create issueET-1 Create New Issue with title and status
  • Status filterET-2 Filter issues by status
  • SearchET-4 Search for specific issue

Defects

[Medium] Validation error displays near Description field instead of Title field

Affects: ET-5 Create new issue without filling any fields
Reproduce:

  1. Click New Issue button
  2. Click Create Issue without filling any fields
    Evidence: Test log shows validation error appears near Description field, but scenario requires it near Title field

Execution Issues

  • ET-3 Filter issues by label — Dead loop on verification step, test could not progress after repeated attempts on unchanged page

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants