[rig-tasks] Daily rig evaluation — 2026-07-23 — 5/5 passed

[github-actions]

Summary

Task	Description	Typecheck	Key finding
1	API endpoint extractor (p.bash + classifier subagent + s.enum purpose)	✅ pass	Idiomatic; subagent wired correctly via agents: { classifier }
2	Package health checker (two-agent chain via agents:)	✅ pass	Sequential chaining idiom works cleanly; no issues
3	TODO/FIXME/HACK tracker (p.bash grep + p.write markdown report)	✅ pass	s.literal("todo-report.md") for reportPath is a nice strictness pattern
4	npm audit vulnerability reporter (p.bash + repair/steering + s.enum)	✅ pass	repair + steering() addons compose correctly; double-brace in bash string required care
5	OpenAPI spec validator (p.readOptional + defineTool + s.enum issue type)	❌ fail → fixed	defineTool requires explicit type parameter defineTool<{content:string}>(...); default T=unknown causes TS2322

All 5 programs passed typecheck after one fix.

---

Problems encountered

Task 5 — defineTool handler type inference failure

What the generated code tried to do:

const validateSchema = defineTool("validate_schema", {
  parameters: s.object({ content: s.string }),
  handler: async ({ content }) => { ... },
});

What went wrong:

error TS2322: Type '({ content }: { content: any; }) => Promise<string>'
  is not assignable to type 'ToolHandler<unknown>'.
  Types of parameters '__0' and 'args' are incompatible.
    Type 'unknown' is not assignable to type '{ content: any; }'.

defineTool defaults to T = unknown, so without an explicit type parameter the handler argument is unknown. TypeScript cannot destructure unknown. The fix requires:

const validateSchema = defineTool<{ content: string }>("validate_schema", { ... });

Why this is a usability issue: The parameters schema (s.object({ content: s.string })) already carries enough type information for the runtime. Users reasonably expect defineTool to infer T from the parameters field shape, removing the need for a manual type annotation that duplicates the schema. This is the single most confusing friction point in the defineTool API.

---

Improvement opportunities

Missing schema helpers (s.*)

• s.int vs s.number discoverability: When writing dependencyCount, it is natural to reach for s.number. SKILL.md does document s.int but it could be surfaced more prominently in the quick-reference table since counts and line numbers should always be integers.

Missing prompt helpers (p.*)

• p.bashRaw discoverability: p.bash("grep -rn 'app\\.get\\|app\\.post'...") requires double-escaping backslashes inside a TypeScript string. p.bashRaw\grep -rn 'app.get|app.post'...`avoids this entirely but is easy to miss. SKILL.md mentions it but the bash-with-regex pattern is a common enough use case that generated code (and docs examples) should default top.bashRaw`.
• p.write semantics reminder: p.write contributes a write-file instruction to the prompt; it does not return a path string. This is documented but commonly causes confusion when authors try to reference the written path in a subsequent expression.

Error message quality

• defineTool type error (TS2322): The error message Type 'unknown' is not assignable to type '{ content: any; }' does not mention defineTool or suggest the explicit type parameter fix. A developer seeing this cold would not immediately know the solution. A clearer message or runtime assertion would help.

API ergonomics

• defineTool type inference from parameters: The parameters field is a SchemaLike that carries full type information at compile time. defineTool could use conditional types to infer TArgs from the parameters field, eliminating the manual defineTool<{...}>(...) annotation. This is the highest-value ergonomics improvement for the tools API.
• p.bash string escaping burden: Complex grep patterns with pipes and backslashes are very common in agentic scripts. Making p.bashRaw the canonical recommendation (not a footnote) would reduce escape-related bugs.

Documentation gaps

• defineTool explicit type parameter requirement: SKILL.md's defineTool example does not show a case where the handler destructures the parameters object. An example of the form defineTool<{ issue: string }>("lookup_issue", { ..., handler: async ({ issue }) => ... }) would prevent the TS2322 pattern immediately.
• steering() requires repair prerequisite: SKILL.md states this but it's worth making it a callout box since using steering() alone silently does nothing useful.

---

Tasks run today

• (reused) Task 1 — API endpoint extractor (p.bash + classifier subagent + s.enum)
• (reused) Task 2 — Package health checker (two-agent sequential chain)
• (reused) Task 3 — TODO/FIXME/HACK comment tracker (p.bash + p.write)
• (new) Task 4 — npm audit vulnerability reporter (p.bash + repair/steering + structured s.enum)
• (new) Task 5 — OpenAPI spec validator (p.readOptional + defineTool + s.enum issue type)

Generated by Daily Rig Task Generator · sonnet46 62.7 AIC · ⌖ 4.56 AIC · ⊞ 5.5K · ◷