[rig-sampler] fix: extract balanced JSON arrays from prose responses

[github-actions]

Sample run: 11-release-notes.ts

The harness ran sample 11 (11-release-notes.ts) successfully in a single turn with no repair loops. The agent used s.enum, s.object, s.array, and s.string — all working correctly.

What the run revealed

While analyzing the JSON parsing fallback path (extractBalancedJsonObject), I found a gap: the function only scanned for {...} JSON objects. Agents whose output schema is s.array(...) could receive a response from the model where the JSON array is embedded in surrounding prose text (e.g., "Here are the results:\n["alpha","beta"]\nDone."). In that case:

1. JSON.parse(text) fails (not bare JSON)
2. extractFencedJson fails (not in a fenced block)
3. extractBalancedJsonObject fails to find { and returns undefined
4. The parse error "No JSON value found." triggers a repair turn — even though the array data was present

Change

Renamed extractBalancedJsonObject → extractBalancedJson and updated the logic to pick whichever of { or [ appears first in the response, then balance the corresponding close character (} or ]). This means prose-wrapped arrays are extracted correctly without a repair loop.

A new test verifies the fix:

it("parses a JSON array embedded in surrounding text", ...)

All 130 existing tests continue to pass.

Generated by Daily Rig Sampler · 77.9 AIC · ⌖ 8.15 AIC · ⊞ 5.2K · ◷

[github-actions]

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

[github-actions]

Skills-Based Review 🧠

Applied /diagnosing-bugs and /tdd — requesting changes on one correctness issue.

📋 Key Themes & Highlights

Key Themes

• Correctness risk in start-position heuristic: Math.min(objectStart, arrayStart) picks the first bracket character in the text. Prose responses that use [word] notation before the actual JSON (very common in LLM output: "Use tags [optional]: {...}") will cause extractBalancedJson to latch onto the bracket span, return malformed text, and trip the JSON.parse error path — a regression relative to the old behaviour.
• Missing regression test: no test locks in correct behaviour when prose contains a [-delimited span before a JSON object.

Positive Highlights

• ✅ Rename extractBalancedJsonObject → extractBalancedJson accurately reflects expanded scope
• ✅ Fix for array extraction is clean and minimal
• ✅ New test mirrors the existing object-in-prose test nicely

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · 33.4 AIC · ⌖ 4.23 AIC · ⊞ 6.3K
_{Comment /matt to run again}

[github-actions]

[/diagnosing-bugs] Picking the earlier of [ and { can silently break object extraction when prose contains a bracket-delimited span (e.g. "Use tags [optional]: {\"key\":1}") — [optional] would be extracted, fail JSON.parse, and return an error instead of finding the valid object.

💡 Safer approach: try both, return the one that parses

One robust fix is to try extracting from both candidate positions and return whichever one successfully parses:

function extractBalancedJson(text: string): string | undefined {
  const candidates = [text.indexOf("{"), text.indexOf("[")]
    .filter(i => i !== -1)
    .sort((a, b) => a - b);
  for (const start of candidates) {
    const result = extractBalancedJsonFrom(text, start);
    if (result !== undefined) {
      try { JSON.parse(result); return result; } catch { /* try next */ }
    }
  }
  return undefined;
}

Alternatively, always prefer the earlier candidate but fall through to the other on parse failure — the current structure in parseJson already catches parse errors, so the fallback could live there.

The current heuristic works for the new test case but introduces a new failure mode for any prose that uses [...] notation before a JSON object.

[github-actions]

[/tdd] The new test covers the happy path but misses the case where prose contains [markdown link] or [note] text before a JSON object — exactly the regression the Math.min heuristic can introduce.

💡 Suggested regression test

it("parses a JSON object when prose contains a bracket span before it", async () => {
  mocks.setSendAndWaitImpl(async () => "Tags [optional]: {\"key\":\"value\"}");

  const tagger = agent({
    name: "tagger",
    output: s.object({ key: s.string }),
  });

  await expect(tagger("go")).resolves.toEqual({ key: "value" });
});

This test would currently fail, making the regression detectable before merging. Adding it alongside the implementation fix would lock in correctness for both types.