Skip to content

docs(#3389): add product naming guidance to AGENTS.md - #3390

Closed
fullsend-ai-coder[bot] wants to merge 1 commit into
mainfrom
agent/3389-product-naming-guidance
Closed

docs(#3389): add product naming guidance to AGENTS.md#3390
fullsend-ai-coder[bot] wants to merge 1 commit into
mainfrom
agent/3389-product-naming-guidance

Conversation

@fullsend-ai-coder

Copy link
Copy Markdown
Contributor

Add "Product name" bullet to the Key Conventions section of AGENTS.md, establishing "Conforma CLI" as the canonical project name and explicitly deprecating the former "ec-cli" name. This prevents code agents from using stale project names found in issue descriptions when generating user-facing strings, error messages, or documentation.


Closes #3389

Post-script verification

  • Branch is not main/master (agent/3389-product-naming-guidance)
  • Secret scan passed (gitleaks — 6e02d8a1d6040c5f38460ea01e7209942361d3ce..HEAD)
  • Pre-commit hooks passed (authoritative run on runner)
  • Tests ran inside sandbox

Add "Product name" bullet to the Key Conventions section
of AGENTS.md, establishing "Conforma CLI" as the canonical
project name and explicitly deprecating the former "ec-cli"
name. This prevents code agents from using stale project
names found in issue descriptions when generating
user-facing strings, error messages, or documentation.

Closes #3389
@fullsend-ai-review

fullsend-ai-review Bot commented Jul 8, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 2:55 PM UTC · Completed 3:01 PM UTC
Commit: 7c8ccca · View workflow run →

@fullsend-ai-review

Copy link
Copy Markdown

Review — approve

Scope: documentation-only (AGENTS.md, +3 lines)

Summary

This PR adds a "Product name" bullet to the Key Conventions section of AGENTS.md, establishing "Conforma CLI" as the canonical project name (binary: ec) and explicitly deprecating the former "ec-cli" name. The change traces directly to issue #3389, a retrospective action item from PR #3381 where the code agent used the stale "ec-cli" name in error messages.

Dimensions reviewed

Dimension Result
Correctness ✅ No findings. Verified claims against source: binary name ec confirmed in cmd/root/root_cmd.go:68 (Use: "ec"), project name "Conforma CLI" confirmed at line 69 (Short: "Conforma CLI"). Internally consistent with existing AGENTS.md references to ec and ec-work-*.
Security ✅ No findings. Clean ASCII text, no secrets, no injection patterns, no Unicode steganography, no permission changes.
Intent & coherence ✅ No findings. PR scope exactly matches issue #3389's proposed wording and location. No scope creep. Adding to Key Conventions is architecturally appropriate.
Style & conventions ✅ No findings. New bullet follows the established **Bold key:** description pattern with backtick code references, consistent with surrounding bullets.
Documentation currency ✅ No findings. Neither AGENTS.md nor CLAUDE.md contains the deprecated "ec-cli" name.
Cross-repo contracts ⏭ Skipped — no exported interfaces or API surfaces modified.

Verdict

Clean, well-scoped documentation change. No findings across any review dimension. The guidance is technically accurate and appropriately placed in Key Conventions where code agents demonstrably read it.


Protected paths detected — this PR modifies files under one or more
protected paths. The review agent cannot approve PRs that touch these paths.
A human reviewer must approve this PR.

Protected files in this PR:

  • AGENTS.md

Labels: PR adds documentation guidance to AGENTS.md.

@fullsend-ai-review fullsend-ai-review Bot added requires-manual-review Review requires human judgment documentation Improvements or additions to documentation labels Jul 8, 2026
@codecov

codecov Bot commented Jul 8, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

Flag Coverage Δ
acceptance 53.62% <ø> (+<0.01%) ⬆️
generative 17.28% <ø> (ø)
integration 28.52% <ø> (ø)
unit 71.89% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@dheerajodha

Copy link
Copy Markdown
Contributor

This PR is replaced by #3457

@dheerajodha dheerajodha closed this Aug 4, 2026
@fullsend-ai-retro

fullsend-ai-retro Bot commented Aug 4, 2026

Copy link
Copy Markdown

🤖 Finished Retro · ✅ Success · Started 11:56 AM UTC · Completed 12:07 PM UTC
Commit: 87c4a29 · View workflow run →

@fullsend-ai-retro

Copy link
Copy Markdown

Retro: PR #3390 — docs(#3389): add product naming guidance to AGENTS.md

What happened

Issue #3389 was auto-generated by the retro agent after analyzing PR #3381, where the code agent had parroted the stale "ec-cli" name from an issue description into error messages. The retro proposed adding explicit naming guidance to AGENTS.md with specific text to use.

The agent pipeline executed cleanly: triage (3 min), code (6 min), review (6 min) — all successful. The code agent opened PR #3390 with a 3-line addition to AGENTS.md establishing "Conforma CLI" as canonical and prohibiting use of "ec-cli." The review agent recommended approval across all dimensions. Because AGENTS.md is a protected path, the harness correctly downgraded the approval and applied requires-manual-review.

The PR sat for 27 days without human review. On Aug 4, dheerajodha closed it as superseded by PR #3457, a human-authored replacement (by cuipinghuo, co-authored with Claude) that was approved by two human reviewers.

The quality gap

The critical difference between the two PRs: the agent's version (#3390) used a blanket prohibition ("Do not use the former name 'ec-cli'") while the human's version (#3457) correctly added a legacy compatibility exception ("Legacy identifiers required for compatibility (e.g., quay.io/enterprise-contract/ec-cli, Tekton parameter names) must be preserved as-is").

The codebase contains at least 9 files with legitimate uses of "ec-cli" — container image paths, Tekton parameters, release config, test fixtures. A future agent following the blanket prohibition literally could have incorrectly renamed backward-compatibility-critical identifiers.

Agent-level gaps

Code agent copied the retro's proposed text verbatim from issue #3389 without searching the codebase for existing uses of "ec-cli." It grep'd for Conforma (to confirm the canonical name) but never grep'd for ec-cli (to verify the prohibition was safe). 37 tool calls, zero investigation of the term being prohibited.

Review agent did grep for ec-cli (tool call 50 of 56) and found the 9 files with legitimate uses. But its correctness check was limited to factual verification ("is the binary actually called ec?") — it never reasoned about whether the blanket prohibition conflicted with the legitimate uses it had just found.

What worked well

  • The protected paths mechanism correctly prevented auto-approval of AGENTS.md changes, which was the right call here
  • The review agent's factual verification was thorough — it confirmed the binary name and project name against source code
  • The triage agent correctly categorized the issue and applied appropriate labels
  • The overall pipeline latency (issue to review complete) was ~16 minutes

Existing issue notes

Proposals filed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation requires-manual-review Review requires human judgment size: XS

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add product naming guidance to conforma/cli AGENTS.md to prevent stale name usage in generated code

1 participant