Skip to content

Commit ba545fa

Browse files
committed
fix(copilot): keep prose a mispaired tag would swallow, and bail on dead JSON
Two more real cases from traces, both of which the earlier heuristics missed. 1. A MATCHED pair whose body fails to parse was silently dropped, cursor and all. When the model explains tag syntax and ends with a backticked example containing a real closing tag, that example closes an EARLIER opener and three paragraphs become the "body" — which is not valid JSON, so the whole span vanished and the render resumed mid-sentence (trace b095e080). Now emitted verbatim, but only when the body contains a tag-shaped marker, which is what shows the pairing was wrong. A marker-free body that merely fails validation is a genuinely malformed agent payload and keeps being dropped — an existing test asserts that deliberately, and showing the user raw JSON there would be a regression. 2. The JSON heuristic only checked the FIRST character, so `{"type":"file"}</workspac and then prose...` looked viable forever: it opens with `{`, and the truncated close is not a marker any rule can see (trace afbeefd0). Track depth instead — once the top-level value closes, any non-whitespace after it is fatal, so the stray character decides it immediately. String contents are blanked first so braces inside strings do not skew the count.
1 parent f0ff34b commit ba545fa

2 files changed

Lines changed: 122 additions & 4 deletions

File tree

apps/sim/app/workspace/[workspaceId]/home/components/message-content/components/special-tags/special-tags.test.ts

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -149,6 +149,53 @@ describe('parseSpecialTags with <question>', () => {
149149
expect(segments).toEqual([{ type: 'text', content: 'Thinking about it. ' }])
150150
})
151151

152+
it('keeps the text when a matched pair fails to parse', () => {
153+
// Verbatim from a real message (trace b095e080). The model explained the
154+
// tag and ended with a backticked example containing a REAL closing tag,
155+
// which closed the earlier opener and made everything between it the body.
156+
// That body is not valid JSON, so the segment was dropped and the render
157+
// resumed mid-sentence at ") is what actually produces the interactive
158+
// chip." — three paragraphs silently gone.
159+
const raw =
160+
'Here you go — with the ending tag intentionally malformed as `</workflow_resource>`:\n\n' +
161+
'<workspace_resource>{"type": "file", "path": "files/notes.md", "title": "notes.md"}</workflow_resource>\n\n' +
162+
"Since the closing tag doesn't match the opening `<workspace_resource>`, the chat won't " +
163+
'recognize it as a valid resource chip. A properly matched pair ' +
164+
'(`<workspace_resource>...</workspace_resource>`) is what actually produces the interactive chip.'
165+
166+
const rendered = parseSpecialTags(raw, false)
167+
.segments.map((segment) => ('content' in segment ? segment.content : ''))
168+
.join('')
169+
170+
expect(rendered).toContain("Since the closing tag doesn't match")
171+
expect(rendered).toContain('A properly matched pair')
172+
expect(rendered).toContain('"path": "files/notes.md"')
173+
// No segment renders as a resource chip — the body was never valid.
174+
expect(parseSpecialTags(raw, false).segments.some((s) => s.type === 'workspace_resource')).toBe(
175+
false
176+
)
177+
})
178+
179+
it('still drops a marker-free malformed payload rather than showing raw JSON', () => {
180+
// The complement of the case above: no tag markers in the body, so this is
181+
// a genuinely broken emission from the agent, not swallowed prose.
182+
const { segments } = parseSpecialTags(
183+
'Before. <question>{"type":"single_select"}</question> After.',
184+
false
185+
)
186+
expect(segments).toEqual([
187+
{ type: 'text', content: 'Before. ' },
188+
{ type: 'text', content: ' After.' },
189+
])
190+
})
191+
192+
it('still renders a matched pair whose body IS valid', () => {
193+
const raw =
194+
'see <workspace_resource>{"type":"file","path":"files/a.md","title":"a.md"}</workspace_resource> ok'
195+
const { segments } = parseSpecialTags(raw, false)
196+
expect(segments.some((s) => s.type === 'workspace_resource')).toBe(true)
197+
})
198+
152199
it('shows prose immediately mid-stream instead of blanking the rest', () => {
153200
// The failure this replaces: everything after the marker stayed invisible
154201
// for the remainder of the stream, then reappeared when it ended.
@@ -160,6 +207,24 @@ describe('parseSpecialTags with <question>', () => {
160207
)
161208
})
162209

210+
it('shows text once the JSON value has closed and stray content follows', () => {
211+
// Verbatim shape from a real message (trace afbeefd0): the close tag was
212+
// TRUNCATED to `</workspac`, so no marker rule can see it — but the JSON
213+
// value completes at the `}`, which makes everything after it fatal.
214+
const raw =
215+
'kicks off in <workspace_resource>{"type":"file","path":"files/notes.md"}</workspac and after that I brew a cup of coffee.'
216+
const { segments, hasPendingTag } = parseSpecialTags(raw, true)
217+
expect(hasPendingTag).toBe(false)
218+
expect(segments.map((s) => ('content' in s ? s.content : '')).join('')).toContain(
219+
'I brew a cup of coffee'
220+
)
221+
})
222+
223+
it('tolerates braces inside JSON strings when tracking depth', () => {
224+
const raw = 'x <workspace_resource>{"title":"a } b","path":"files/a.md"'
225+
expect(parseSpecialTags(raw, true).hasPendingTag).toBe(true)
226+
})
227+
163228
it('still suppresses a JSON-bodied tag that is genuinely mid-stream', () => {
164229
const { segments, hasPendingTag } = parseSpecialTags(
165230
'Here you go <workspace_resource>{"type":"file","id":"abc"',

apps/sim/app/workspace/[workspaceId]/home/components/message-content/components/special-tags/special-tags.tsx

Lines changed: 57 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -473,6 +473,12 @@ function parseSpecialTagData(
473473
* Trailing partial opening tags (e.g. `<opt`, `<usage_`) are also stripped
474474
* during streaming to prevent flashing raw markup.
475475
*/
476+
/**
477+
* Any tag-shaped marker, including names that are not special tags at all — the
478+
* model inventing `</workflow_resource>` is exactly the case that matters.
479+
*/
480+
const TAG_SHAPED_MARKER = /<\/?[a-zA-Z][\w-]*>/
481+
476482
/**
477483
* Tags whose body must be JSON. `thinking` is the exception — its body is prose
478484
* (see {@link parseTextTagBody}), so a non-JSON body there says nothing about
@@ -525,6 +531,41 @@ function blankJsonStringLiterals(body: string): string {
525531
return out
526532
}
527533

534+
/**
535+
* True while `body` could still grow into a single valid JSON value.
536+
*
537+
* Checking only the first character is not enough: a body like
538+
* `{"type":"file"}</workspac and then prose...` opens with `{` and looks fine,
539+
* but the value CLOSES at the `}` and everything after it is fatal. Tracking
540+
* depth catches that the moment the stray character arrives, instead of waiting
541+
* for a close tag that is never coming.
542+
*
543+
* String contents are blanked first, so braces and brackets inside JSON strings
544+
* do not affect the depth count.
545+
*/
546+
function isViableJsonPrefix(body: string): boolean {
547+
const scannable = blankJsonStringLiterals(body)
548+
if (scannable.trim() === '') return true
549+
550+
const firstChar = scannable.trimStart().charAt(0)
551+
if (firstChar !== '{' && firstChar !== '[') return false
552+
553+
let depth = 0
554+
for (let i = 0; i < scannable.length; i++) {
555+
const char = scannable[i]
556+
if (char === '{' || char === '[') {
557+
depth++
558+
} else if (char === '}' || char === ']') {
559+
depth--
560+
if (depth < 0) return false
561+
// The top-level value just closed: only trailing whitespace may follow.
562+
if (depth === 0) return scannable.slice(i + 1).trim() === ''
563+
}
564+
}
565+
566+
return true
567+
}
568+
528569
/**
529570
* True when an opening tag with no close yet can NEVER resolve, so the text
530571
* after it should be shown immediately instead of held back until the stream
@@ -558,10 +599,7 @@ function unclosedTagCannotResolve(
558599
if (scannable.includes(`</${name}>`) || scannable.includes(`<${name}>`)) return true
559600
}
560601

561-
if (JSON_BODY_TAG_NAMES.has(tagName)) {
562-
const firstChar = body.trimStart().charAt(0)
563-
if (firstChar !== '' && firstChar !== '{' && firstChar !== '[') return true
564-
}
602+
if (JSON_BODY_TAG_NAMES.has(tagName) && !isViableJsonPrefix(body)) return true
565603

566604
return false
567605
}
@@ -647,6 +685,21 @@ export function parseSpecialTags(content: string, isStreaming: boolean): ParsedS
647685
const parsedTag = parseSpecialTagData(nearestTagName, body)
648686
if (parsedTag) {
649687
segments.push(parsedTag)
688+
} else if (TAG_SHAPED_MARKER.test(body)) {
689+
// The body failed to parse AND contains tag markers, which means the
690+
// close we matched was not this opener's — the model was explaining tag
691+
// syntax and a later example like `<workspace_resource>...</workspace_resource>`
692+
// closed an earlier opener, making paragraphs of prose the "body".
693+
// Dropping that silently loses the text and resumes mid-sentence, so emit
694+
// it verbatim. Streamdown escapes the markers, so it reads as literal text.
695+
//
696+
// A marker-free body that merely fails validation is a genuinely
697+
// malformed payload from the agent; that keeps being dropped rather than
698+
// showing the user raw JSON.
699+
const literal = content.slice(nearestStart, closeIdx + closeTag.length)
700+
if (literal.trim()) {
701+
segments.push({ type: 'text', content: literal })
702+
}
650703
}
651704

652705
cursor = closeIdx + closeTag.length

0 commit comments

Comments
 (0)