Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@

### Fixed

- **`X-AIMock-Strict` is parsed case-insensitively and tolerates surrounding whitespace.** `resolveStrictMode` compared the raw header value against `"true"` / `"false"` / `"1"` / `"0"` literally, so the variants clients, proxies and shell snippets actually emit — `True`, `TRUE`, `" true "` — matched nothing and fell through to the server default. A caller on a `--strict false` server asking for `X-AIMock-Strict: True` kept getting `404` instead of `503`, and on a `--strict true` server `X-AIMock-Strict: False` stayed strict; the same header gates reasoning suppression, so the mismatch could also surface as a flaky reasoning assertion. The value is now trimmed and lower-cased before comparison. Unrecognised values still fall back to the server default, and trimming is end-only — `"tr ue"` is still not a match (#408)

- **The AG-UI drift collector no longer reports "clean" for a failure it could not read.** `collectAgUiDriftEntries` recognizes three failure-message shapes; any FAILED assertion matching none of them was dropped outright — no entry, no counter, no warning — and a FAILED assertion carrying no message at all was skipped before it was even looked at. Both paths produced zero entries, exit 0 and `conclusion: "clean"`, so a genuinely failing `agui-schema.drift.ts` assertion (`should parse aimock event types` fails with a bare `expect` message that matches no shape) certified AG-UI as drift-free. Every such failure now takes the collector's EXISTING quarantine lane — held for review at exit 5, the same "never silently swallowed" contract the HTTP leg already used — per-assertion and unconditional, so an unreadable failure survives a mixed run in which other failures DID parse. Separately, a ZERO-exit AG-UI run whose stdout is not vitest JSON now THROWS instead of returning an empty (`"no failures"`) result, matching what the HTTP twin `runDriftTests()` has always done on the same condition: the two legs no longer disagree about whether garbage output is clean. Drift-harness tooling, not runtime behavior — no published surface changes (#391)

- **The AG-UI drift suite's canonical-schema parser no longer drops the field declared after a trailing comment.** `extractExtendFields` stripped whole-line comments only (`/^\s*\/\/.*$/gm`). Upstream's `STATE_DELTA` declares `delta: z.array(z.any()), // JSON Patch (RFC 6902)`, and after the top-level comma split that trailing comment heads the NEXT entry, so the field-name match failed and the entry was discarded silently — which is why the drift report read canonical as declaring `subagentRunId` on 23 events rather than 24, and flagged `STATE_DELTA.subagentRunId` as aimock-only. Comments are now stripped wherever they appear; no canonical schema literal contains `//`, so the unconditional strip is safe. This is drift-harness tooling, not runtime behavior — no published surface changes (#391)
Expand Down
54 changes: 54 additions & 0 deletions src/__tests__/strict-header.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,60 @@ describe("resolveStrictMode", () => {
expect(resolveStrictMode(true, { "x-aimock-strict": "0" })).toBe(false);
});

// The header arrives from clients, proxies and shell snippets that do not
// agree on casing or padding. HTTP header VALUES are case-sensitive by spec,
// so this is a deliberate tolerance, not something the platform does for us —
// which is exactly why it needs pinning: before this, `True` silently fell
// through to the server default, and a caller asking for strict got whatever
// the server already was.
it("accepts any casing", () => {
for (const on of ["TRUE", "True", "TrUe"]) {
expect(resolveStrictMode(false, { "x-aimock-strict": on })).toBe(true);
}
for (const off of ["FALSE", "False", "FaLsE"]) {
expect(resolveStrictMode(true, { "x-aimock-strict": off })).toBe(false);
}
});

it("tolerates surrounding whitespace, including tabs and newlines", () => {
for (const on of [" true", "true ", " true ", "\ttrue\t", "true\r\n"]) {
expect(resolveStrictMode(false, { "x-aimock-strict": on })).toBe(true);
}
for (const off of [" 0", "0 ", " false ", "\tfalse"]) {
expect(resolveStrictMode(true, { "x-aimock-strict": off })).toBe(false);
}
});

it("accepts casing and whitespace together", () => {
expect(resolveStrictMode(false, { "x-aimock-strict": " TRUE " })).toBe(true);
expect(resolveStrictMode(true, { "x-aimock-strict": " False " })).toBe(false);
expect(resolveStrictMode(false, { "x-aimock-strict": " 1 " })).toBe(true);
});

// A repeated header arrives as an array; the first value is the one that
// decides, and it gets the same normalisation as the string form.
it("normalises the first value of a repeated header", () => {
expect(resolveStrictMode(false, { "x-aimock-strict": [" TRUE "] })).toBe(true);
expect(resolveStrictMode(true, { "x-aimock-strict": [" False ", "true"] })).toBe(false);
});

// NEGATIVE CONTROLS: trimming and lower-casing must not widen what counts as
// a valid value. Without these, the assertions above would also pass on an
// implementation that treated any non-empty string as truthy.
it("does not turn unrecognised values into a match after normalising", () => {
for (const bogus of ["", " ", "truthy", "true-ish", "t", "yes", "on", "2", "-1", "10"]) {
expect(resolveStrictMode(true, { "x-aimock-strict": bogus })).toBe(true);
expect(resolveStrictMode(false, { "x-aimock-strict": bogus })).toBe(false);
}
});

// Whitespace is trimmed from the ENDS only — an inner space is not noise, it
// makes the value a different token.
it("does not strip interior whitespace", () => {
expect(resolveStrictMode(false, { "x-aimock-strict": "tr ue" })).toBe(false);
expect(resolveStrictMode(true, { "x-aimock-strict": "fa lse" })).toBe(true);
});

it("ignores unrecognised header values and falls back to server default", () => {
expect(resolveStrictMode(true, { "x-aimock-strict": "yes" })).toBe(true);
expect(resolveStrictMode(false, { "x-aimock-strict": "maybe" })).toBe(false);
Expand Down
13 changes: 9 additions & 4 deletions src/helpers.ts
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,8 @@ import type {
* Header values override the server default — same precedence pattern as chaos
* config headers (see resolveChaosConfig in chaos.ts).
*
* Header: `X-AIMock-Strict` — "true"/"1" → strict on, "false"/"0" → strict off.
* Header: `X-AIMock-Strict` — "true"/"1" → strict on, "false"/"0" → strict off
* (case-insensitive, surrounding whitespace trimmed).
* When absent or unrecognised, falls back to the server-level default.
*/
export function resolveStrictMode(
Expand All @@ -42,9 +43,13 @@ export function resolveStrictMode(
): boolean {
if (rawHeaders) {
const header = rawHeaders["x-aimock-strict"];
const val = typeof header === "string" ? header : Array.isArray(header) ? header[0] : undefined;
if (val === "true" || val === "1") return true;
if (val === "false" || val === "0") return false;
const rawVal =
typeof header === "string" ? header : Array.isArray(header) ? header[0] : undefined;
if (typeof rawVal === "string") {
const val = rawVal.trim().toLowerCase();
if (val === "true" || val === "1") return true;
if (val === "false" || val === "0") return false;
}
}
return serverDefault ?? false;
}
Expand Down
Loading