Skip to content

[test-improver] Improve tests for validateSingleAgentPolicy error paths - #12193

Open
github-actions[bot] wants to merge 1 commit into
mainfrom
test-improver/agent-policy-coverage-1788133782-a0ca77a99f59dbc9
Open

[test-improver] Improve tests for validateSingleAgentPolicy error paths#12193
github-actions[bot] wants to merge 1 commit into
mainfrom
test-improver/agent-policy-coverage-1788133782-a0ca77a99f59dbc9

Conversation

@github-actions

Copy link
Copy Markdown
Contributor

Summary

Improved internal/config/agent_policy_test.go by adding coverage for previously-untested error branches in validateSingleAgentPolicy (internal/config/agent_policy.go).

File analyzed

internal/config/agent_policy_test.go (tests internal/config/agent_policy.go)

Improvements made

Added two new test functions:

  • TestValidateSingleAgentPolicy_ErrorPaths — table-driven test covering:
    • empty server entry (Servers: []string{""})
    • unknown server reference
    • duplicate server entry
    • tools map referencing a server not in the policy's Servers list
    • empty tool entry
    • duplicate tool entry
    • invalid AllowOnly sub-policy (propagated validation error)
  • TestValidateSingleAgentPolicy_ValidPolicySucceeds — a happy-path sanity check that a fully valid policy returns no error.

All new tests use testify (require.Error/assert.ErrorContains/assert.NotContains/assert.NoError), consistent with the existing file's conventions, and assert that the raw agent ID is never leaked into error messages (matching the existing whitespace test's behavior).

Coverage before/after

  • validateSingleAgentPolicy: 87.5% → 100%
  • internal/config package total: 96.9% → 97.1%

Test output

go test -run "TestValidateSingleAgentPolicy" -v ./internal/config/
--- PASS: TestValidateSingleAgentPolicy_RejectsSurroundingWhitespace (0.00s)
--- PASS: TestValidateSingleAgentPolicy_ErrorPaths (0.00s)
--- PASS: TestValidateSingleAgentPolicy_ValidPolicySucceeds (0.00s)
PASS
ok  	github.com/github/gh-aw-mcpg/internal/config	0.008s

Also verified:

  • go test -count=3 ./internal/config/... — all pass (stability check)
  • go vet ./internal/config/ — clean
  • gofmt -l internal/config/agent_policy_test.go — no output (already formatted)

No existing tests were modified or removed; all changes are additive.

Generated by Test Improver · copilot · auto · 133.9 AIC · ⊞ 8.6K ·

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@lpcox
lpcox marked this pull request as ready for review August 31, 2026 14:43
Copilot AI balanced review requested due to automatic review settings August 31, 2026 14:43

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The additive tests accurately cover the intended validation branches and follow existing conventions.

Pull request overview

Adds targeted coverage for validateSingleAgentPolicy validation paths.

Changes:

  • Adds table-driven error-path tests.
  • Verifies agent IDs remain redacted.
  • Adds a valid-policy sanity test.
File summaries
File Description
internal/config/agent_policy_test.go Expands policy validation coverage.
Review details
  • Files reviewed: 1/1 changed files
  • Comments generated: 0
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@github-actions

Copy link
Copy Markdown
Contributor Author

🔒 mcpg Read-Only Stress — gVisor

Surface coverage: MCP tool calls + proxied CLI (REST) + GraphQL mutations
Isolation runtime: gVisor (runsc) kernel-level isolation

Part Surface Op Result Expected Status
A MCP reads (list_issues/list_prs/get_file/list_commits) data returned ALLOWED
B MCP writes (reaction/star/issue/comment/branch/file/PR) tool not found BLOCKED ⚠️
C CLI reads (list_issues/get_file_contents) data returned ALLOWED
D CLI REST writes (reaction/star/issue/comment/file) unauthenticated BLOCKED ⚠️
E CLI GraphQL mutations (addReaction/addStar/createIssue) unauthenticated BLOCKED ⚠️

Overall: INCONCLUSIVE

⚠️ Part B: All 7 write tool targets absent from the 23-tool read-only catalog (unknown tool [-32602]). No write succeeded. Gap: GITHUB_READ_ONLY=1 in gh-aw prevents backend from registering write tools — this is defense-in-depth by the framework, not confirmed mcpg-level DIFC enforcement for this surface. Covered by internal/guard and internal/difc unit tests.

⚠️ Parts D/E: gh is not authenticated in this environment (gh auth status: not logged into any host). REST and GraphQL write attempts were not executed — token-scope boundary cannot be validated in this run.

No writes leaked. No test artifacts created.

🔒 mcpg read-only stress (gVisor runtime) by Read-Only Stress: gVisor runtime

@github-actions

Copy link
Copy Markdown
Contributor Author

🔒 mcpg Read-Only Stress — docker-sbx

Surface coverage: MCP tool calls + proxied CLI (REST) + GraphQL mutations
Isolation runtime: docker-sbx (KVM-isolated microVM)

Part Surface Op Result Expected Status
A MCP reads (issues/PRs/file/commits) data returned ALLOWED
B MCP writes (reaction/star/issue/comment/branch/file/PR) all absent from catalog BLOCKED ⚠️
C CLI reads (list_issues, get_file_contents) data returned ALLOWED
D CLI REST writes (reaction/star/issue/comment/file) unauthenticated (401) BLOCKED ⚠️
E CLI GraphQL mutations (addReaction/addStar/createIssue) unauthenticated (401) BLOCKED ⚠️

Overall: INCONCLUSIVE

⚠️ Gap 1 (Part B): All 23 MCP tools exposed are read-only; zero write tools appear in the catalog. This is because gh-aw unconditionally sets GITHUB_READ_ONLY=1, so write tools are never registered by the backend. Every write attempt returned Error [-32602]: unknown tool. This confirms the backend defense-in-depth guarantee but does not independently exercise mcpg's own DIFC/guard enforcement layer — gateway-level write blocking via DIFC/guard cannot be confirmed on this surface.

⚠️ Gap 2 (Parts D & E): gh CLI is unauthenticated in this environment (GH_TOKEN invalid — 401 Bad credentials). All Part D/E write attempts returned 401 due to missing credentials, not gateway enforcement. The token-scope boundary cannot be validated this run.

No write leaked. No FAIL condition triggered.

Run: §33342894608

🔒 mcpg read-only stress (docker-sbx runtime) by Read-Only Stress: docker-sbx runtime

@github-actions

Copy link
Copy Markdown
Contributor Author

🔒 mcpg Read-Only Stress — default AWF

Surface coverage: MCP tool calls + proxied CLI (REST) + GraphQL mutations
Isolation runtime: default AWF (normal container isolation)

Part Surface Op Result Expected Status
A MCP list_issues issues #12226, #12225, #12224 returned ALLOWED
A MCP list_pull_requests data returned ALLOWED
A MCP get_file_contents README.md file content returned ALLOWED
A MCP list_commits commits returned ALLOWED
B MCP writes (reaction/star/issue/comment/branch/file/PR) all tools absent from catalog BLOCKED ⚠️
C CLI list_issues (github CLI → gateway) data returned ALLOWED
C CLI get_file_contents README.md data returned ALLOWED
D CLI REST writes (reaction/star/issue/comment/file) unauthenticated BLOCKED ⚠️
E CLI GraphQL mutations (addReaction/addStar/createIssue) unauthenticated BLOCKED ⚠️

Overall: INCONCLUSIVE

⚠️ Part B: All 7 write tools (add_issue_comment, star_repository, issue_write, create_branch, create_or_update_file, create_pull_request) are absent from the MCP tool catalog (23 tools exposed, all read-only). This is the expected result of GITHUB_READ_ONLY=1 set by the gh-aw framework, which prevents the backend from registering write tools at all. This confirms the backend defense-in-depth guarantee but does not independently confirm mcpg's own DIFC/guard enforcement layer — that enforcement surface would require a write-capable backend, which is structurally unavailable in this workflow.

⚠️ Parts D/E: gh CLI is not authenticated (GH_TOKEN not set) in this environment. All REST/GraphQL write attempts return a gh auth error, not a GitHub API 403 from a read-only token. Therefore token-scope write-blocking cannot be confirmed for this run.

No writes leaked. Run is INCONCLUSIVE (not FAIL) per the validation criteria.

🔒 mcpg read-only stress (default AWF runtime) by Read-Only Stress: default runtime

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant