Skip to content

Commit 6edab1b

Browse files
committed
test(copilot): catch a tool silently dropping its query from the chip
The "has an intentional display title for every visible catalog tool" assertion only checks that a title exists. A bare fallback label satisfies it exactly as well as the useful one, so a tool that DID show its query can lose that behavior with the suite still green — a coarse check contented by a degraded state. Adds a subset ratchet: any catalog tool that takes a `query` must show it, unless it is in a documented allowlist. The set may shrink, never grow, so removing a tool or teaching one to show its query needs no update here. Baselines the three tools that ignore their query today rather than changing their chips — making the rule true everywhere is a UX change and does not belong in a test-hardening commit. Verified by removing an entry from the allowlist: the assertion fails with the offending tool named.
1 parent f43b52c commit 6edab1b

1 file changed

Lines changed: 39 additions & 0 deletions

File tree

apps/sim/lib/copilot/tools/tool-display.test.ts

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -84,6 +84,45 @@ describe('getToolDisplayTitle natural-language coverage', () => {
8484
)
8585
})
8686

87+
/**
88+
* Tools that take a `query` but whose chip does not show it. This set may
89+
* SHRINK, never grow — the assertion below is a subset check, so removing a
90+
* tool or teaching one to show its query needs no update here.
91+
*
92+
* The point is to catch a specific silent regression: a tool that DID show
93+
* its query losing that behavior. The neighbouring "has an intentional
94+
* display title" test cannot catch it, because a bare fallback label like
95+
* "Searching docs" satisfies it just as well as the useful one — a coarse
96+
* check contented by a degraded state.
97+
*/
98+
const TITLES_IGNORING_QUERY = new Set([
99+
'search_documentation',
100+
'search_library_docs',
101+
// Has an argument-aware case, but reads toolTitle/title and never falls
102+
// back to query.
103+
'search_online',
104+
])
105+
106+
it('no tool silently stops showing its query in the chip', () => {
107+
const hiddenToolNames = getHiddenToolNames()
108+
const regressions = Object.entries(TOOL_CATALOG)
109+
.filter(([name, entry]) => !hiddenToolNames.has(name) && !entry.internal)
110+
.filter(([, entry]) => {
111+
const properties = (entry.parameters as { properties?: Record<string, unknown> })
112+
?.properties
113+
return properties !== undefined && 'query' in properties
114+
})
115+
.filter(([name]) => {
116+
const withoutArgs = getToolDisplayTitle(name)
117+
const withQuery = getToolDisplayTitle(name, { query: 'a distinctive query' })
118+
return withoutArgs === withQuery
119+
})
120+
.map(([name]) => name)
121+
.filter((name) => !TITLES_IGNORING_QUERY.has(name))
122+
123+
expect(regressions).toEqual([])
124+
})
125+
87126
it('has an intentional display title for every visible catalog tool', () => {
88127
const hiddenToolNames = getHiddenToolNames()
89128
const fallbackToolNames = Object.keys(TOOL_CATALOG).filter(

0 commit comments

Comments
 (0)