Repository navigation
git: clarify when NOT to call status/diff/log — cuts agent extra-call rate (measured +21.7 pts) - #4520
git: clarify when NOT to call status/diff/log — cuts agent extra-call rate (measured +21.7 pts)#4520sujugithub wants to merge 1 commit into
Conversation
Measured with an agent harness (12 scenarios x 5 runs each, live server): tool selection was already perfect (100% hit rate) but agents padded runs with ritual git_status/git_diff_staged/git_log verification calls. With only these description changes: strict success (right tool + right args + no extra calls) 75.0% -> 96.7%. No behavior changes.
|
The measurement harness is now public: https://github.com/sujugithub/hitrate — the git scenario suite is |
|
Thank you for this pull request, and apologies for the long wait for a response. These servers are reference implementations: they exist to show how each part of MCP is used, not to be general-purpose products, so new features are taken selectively, and this change isn't one the maintainers plan to take. This repository also now accepts issues, not pull requests, from anyone but its maintainers; the policy is in This is not a judgment on the idea or on you. A server in the MCP Registry may well offer it, and anyone is welcome to build on these servers to do so. If you think we've misread it, please open an issue describing the problem and the expected behavior. This is part of the backlog triage in #4875. |
What
Rewrites 7 tool description strings in
src/git/src/mcp_server_git/server.py. No code or behavior changes — only the text agents read when choosing tools.Why
I measured how a mid-tier agent actually uses this server (12 realistic scenarios × 5 runs each, live server, every tool call recorded). Tool selection is already excellent — 100% hit rate, the diff-trio naming works. The failure mode is ritual verification calls: after
git_add, agents callgit_statusorgit_diff_stagedto check it worked; beforegit_show HEAD, they callgit_logto find a hash they don't need. A quarter of all runs were padded this way.Measured effect of this diff (nothing else changed)
Method: agent = gpt-oss-120b (deliberately mid-tier — weaker agents are hurt most by underspecified descriptions), N=5 runs per scenario, descriptions applied via in-memory override so before/after used the identical server build. Happy to share the full per-run data and scenario suite, or re-run against any revision of the wording.
Notes
git_showwording impliedrevisionwas optional; sinceGitShow.revisionis required, this PR saysPass revision=\"HEAD\"instead — same intent, schema-accurate.🤖 Generated with Claude Code