Skip to content

fix(server): don't 404 non-websocket upgrade probes, and keep the body on every supported Node - #418

Merged
jpr5 merged 7 commits into
mainfrom
fix/h2c-upgrade-probe-node26
Sep 9, 2026
Merged

fix(server): don't 404 non-websocket upgrade probes, and keep the body on every supported Node#418
jpr5 merged 7 commits into
mainfrom
fix/h2c-upgrade-probe-node26

Conversation

@jpr5

@jpr5 jpr5 commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Carries @dennisoelkers' fix from #410 (their four commits, authorship intact) and adds the piece needed to make it correct on every Node this package supports. Closes #410.

Their fix

Node fires "upgrade" rather than "request" for any Connection: Upgrade header, whatever protocol is named. langchain4j's OkHttp client speculatively sends Upgrade: h2c on plain requests to probe for HTTP/2 cleartext, so GET /api/tags and POST /api/chat landed in the WebSocket-only handler and took an unconditional 404. The fix replays the request with Upgrade/Connection stripped and lets Node re-parse it as an ordinary request. Diagnosis and approach are both right.

I verified the parts that worried me, since this is raw socket work:

  • real WebSocket upgrades still get 101, including the mixed-case Upgrade: WebSocket form
  • the replayed request still passes the AIMOCK_API_KEYS boundary — an unauthenticated h2c probe gets 401, not a bypass
  • keep-alive reuse of the same socket after a replay works: two requests, two 200s
  • no listener growth on the server across 25 probes

What this adds: where the body lives is version-dependent

The original replay forwards head. That is only where the body is on some versions:

  • Node ≤ 24 detaches the parser at "upgrade", so req yields nothing and the body lands in head — or arrives on the socket afterwards if the client sent it later.
  • Node ≥ 26 parses the body onto req and leaves head empty. The bytes are on neither the socket nor head.

So on Node 26 a head-only replay left the re-parsed request waiting on a Content-Length worth of bytes that never arrived — no data, no end, no error. The connection hung open instead of answering, which is worse than the 404 it replaced: a 404 fails a test in milliseconds, a hang blocks the caller until its own timeout. engines is >=20.15.0, so Node 26 is declared supported.

Measured on a dependency-free reproduction of just the replay technique:

node 20/22/24   GET 200 | POST 200 | POST delayed 200 | POST 200KB 200
node 26         GET 200 | POST (NO RESPONSE) | POST delayed (NO RESPONSE) | POST 200KB (NO RESPONSE)

Instrumenting it showed the headers parse fine on 26 — right method, right URL, correct content-length — and pinpointed where the bytes went:

node 24   cl=7  head=7B  socketAfter=0B
node 26   cl=7  head=0B  socketAfter=0B     ← and req yields "{"a":1}"

The fix: drain req and concat it into the replay. The stream has already ended in both cases, so it never waits on the network — on Node ≤ 24 it yields zero bytes and head carries the body exactly as before. All four probe shapes now pass on 20, 22, 24 and 26.

Worth noting the earlier reformulations that did not help, since they look plausible: a single Buffer.concat([rebuilt, head]) instead of two unshifts, socket.resume() after the emit, deferring the emit to process.nextTick, and bridging through a fresh Duplex rather than re-emitting the socket. None move the needle, because the problem was never the socket — the bytes simply weren't in head.

Tests

Two added beside the original. The existing test asserts the body was parsed — but a validation error only needs enough of the body to see a field is missing, so it would still pass on a truncated replay. The new ones assert the body arrived intact (matching a fixture on its content, which truncation cannot fake) and that a 200KB body survives, which no single read can carry.

Mutation-tested: dropping parsedBody from the concat reds 3 of the 95 tests in the file; restored, 95 pass.

CI matrix

Adds Node 26 to the unit and pytest matrices. It sits inside the declared engines range and was untested — which is exactly why the body-loss case passed CI on the original PR.

Verification

typecheck (all three configs) exit 0 · full suite 183 files / 5694 tests · eslint and prettier clean.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Y445N6QQBeAdiLcEvEpqGe

dennisoelkers and others added 6 commits September 3, 2026 10:42
Node fires the "upgrade" event (not "request") for any request with a
Connection: Upgrade header, regardless of the target protocol. OkHttp
(used by langchain4j) speculatively sends Upgrade: h2c on plain
requests to probe for HTTP/2 cleartext, which made GET /api/tags land
in the WebSocket-only upgrade handler and get an unconditional 404.

Only treat the request as a WebSocket upgrade when Upgrade: websocket
is actually present; otherwise fall through to the normal HTTP
pipeline so routes like /api/tags still work.
Node fires the "upgrade" event (not "request") for any request with a
Connection: Upgrade header, regardless of the target protocol. OkHttp
(used by langchain4j) speculatively sends Upgrade: h2c on plain
requests to probe for HTTP/2 cleartext, which made GET /api/tags land
in the WebSocket-only upgrade handler and get an unconditional 404.

Only treat the request as a WebSocket upgrade when Upgrade: websocket
is actually present; otherwise fall through to the normal HTTP
pipeline so routes like /api/tags still work.
The earlier h2c-probe fix (rebuilding a ServerResponse via assignSocket)
only worked for bodyless requests like GET /api/tags. Node detaches its
HTTP parser from the socket the moment "upgrade" fires, so any body
bytes land in the `head` buffer instead of `req`'s stream; unshifting
them back onto the socket never reconnects them to `req`, so
readBody(req) resolved empty. A POST like /api/chat then failed JSON
parsing ("Unexpected end of JSON input") instead of surfacing the real
validation error for the request that was actually sent.

Rebuild the request line and headers with Upgrade/Connection dropped,
replay them plus `head` on the socket, and let Node re-parse the
connection from scratch via server.emit("connection", socket). This
reuses Node's own parser for the body (Content-Length or chunked)
instead of hand-rolling body buffering.
…-upgrade-probe

# Conflicts:
#	src/__tests__/ollama.test.ts
#	src/server.ts
Builds on @dennisoelkers' fix (#410), whose diagnosis and approach are right:
Node fires "upgrade" for any `Connection: Upgrade`, so langchain4j's h2c probe
took an unconditional 404, and replaying the request with `Upgrade`/`Connection`
stripped is the fix.

The gap is WHERE THE BODY IS. That differs by Node version, and the original
replay forwarded only `head`:

  - Node <= 24 detaches the parser at "upgrade": `req` yields nothing and the
    body lands in `head` (or arrives on the socket afterwards if sent later).
  - Node >= 26 parses the body onto `req` and leaves `head` EMPTY. The bytes
    are on neither the socket nor `head`.

So on Node 26 a `head`-only replay left the re-parsed request waiting on a
`Content-Length` worth of bytes that never arrived: no `data`, no `end`, no
error — the connection hung open instead of answering. That is worse than the
404 it replaced, because a hang blocks a caller until its own timeout, and
`engines` declares `>=20.15.0` so Node 26 is supported. Measured directly on a
dependency-free reproduction:

    node 20/22/24  GET 200 | POST 200 | POST delayed 200 | POST 200KB 200
    node 26        GET 200 | POST (NO RESPONSE) x3

Drain `req` and concat it into the replay. It is already ended in both cases,
so this never waits on the network: on Node <= 24 it yields zero bytes and
`head` carries the body exactly as before. All four probe shapes now pass on
20, 22, 24 and 26.

Two tests added beside the original. The existing one asserts the body was
PARSED — but a validation error only needs enough of the body to see a field
is missing, so it would still pass on a TRUNCATED replay. The new ones assert
the body arrived INTACT (matching a fixture on its content, which truncation
cannot fake) and that a 200KB body survives, which no single read can carry.

Mutation-tested: dropping `parsedBody` from the concat reds 3 of the 95 tests
in the file; restored, 95 pass.

Also adds Node 26 to the unit and pytest CI matrices. It sits inside the
declared `engines` range and was untested, which is exactly why the body-loss
case passed CI.

typecheck (all three configs) exit 0; full suite 183 files / 5694 tests;
eslint and prettier clean.
@pkg-pr-new

pkg-pr-new Bot commented Sep 9, 2026

Copy link
Copy Markdown

Open in StackBlitz

npm i https://pkg.pr.new/@copilotkit/aimock@418

commit: a6f66be

@jpr5
jpr5 merged commit b80cf4c into main Sep 9, 2026
31 checks passed
@jpr5
jpr5 deleted the fix/h2c-upgrade-probe-node26 branch September 9, 2026 00:24
@jpr5 jpr5 mentioned this pull request Sep 9, 2026
jpr5 added a commit that referenced this pull request Sep 9, 2026
Cuts the accumulated `[Unreleased]` work. **72 non-merge commits since
v1.39.0, 42 of them `feat:`/`fix:`** — minor, not patch: two additive
features, no breaking changes.

**Prepared, not merged.** `publish-release.yml` fires on push-to-main,
so merging this publishes to npm, tags, cuts the GitHub Release,
dispatches the Docker build, and posts to `#oss-alerts`. Merge when you
want it live.

## What ships

**Added** — `GET /__aimock/fixtures` fixture-count introspection (#407)
· recorded OpenAI/OpenRouter token usage incl. OpenRouter `usage.cost`
(#368) · AG-UI subagent lifecycle events + `subagentRunId` attribution
(#391)

**Fixed** — `X-AIMock-Strict` parsed case-insensitively with whitespace
tolerated (#408) · non-websocket upgrade probes no longer 404 and the
body survives on every supported Node (#410#418) · AG-UI drift
collector no longer reports clean for an unreadable failure · AG-UI
canonical parser no longer drops a field after a trailing comment ·
`openrouter` no longer logged as an unknown SSE provider when recording

**Changed** — the AG-UI drift CI lane runs every `agui-*.drift.ts` guard
rather than one hardcoded file

## Version surfaces — seven

Found by grepping the repo for `1.39.0` rather than working from a
checklist, because the checklist is what went wrong at v1.34.0 (four of
six surfaces shipped stale). **Zero occurrences of the old string
remain:**

| # | file | |
|---|---|---|
| 1 | `package.json` | `version` |
| 2 | `charts/aimock/Chart.yaml` | `appVersion` |
| 3 | `.claude-plugin/plugin.json` | `version` |
| 4 | `.claude-plugin/marketplace.json` | the `^` range under
`plugins[0].source` |
| 5 | `packages/aimock-pytest/.../_version.py` | `AIMOCK_VERSION` — the
npm pin the pytest harness downloads by default |
| 6 | `docs/index.html` | the version badge |
| 7 | `packages/aimock-pytest/README.md` | the documented
`--aimock-version` default |

Two of those (6, 7) aren't in the release SOP's list but carry the
version string and would have shipped stale.

`packages/aimock-pytest/pyproject.toml` stays at **0.5.3** — that's the
Python package's own version, on its own PyPI cadence.

## Two things I checked rather than assumed

**Bumping `_version.py` in the release commit is safe.**
`publish-pytest` runs `npm view @copilotkit/aimock@$AIMOCK_VERSION`,
which would fail against an unpublished version — but it's `needs:
[build, publish]` and gated on `needs.publish.result == 'success'`, so
1.40.0 is on npm by the time it reads the pin. No chicken-and-egg.

**`package.json.description` is deliberately not hand-synced.** It
diverges from the README subtitle in git, which looks like the drift the
SOP warns about. It isn't: `publish-release.yml` rewrites it from the
README subtitle *in the runner* before `npm publish`, without
committing. Editing it here would fight the workflow.

## README

Gains a **Recorded token usage and cost** bullet. The feature list
enumerates record/replay capabilities individually (timing-aware replay,
multi-turn, ordered blocks), so a user-visible one landing without an
entry is a real omission. The new control-API route needs no README line
— the README documents no `/__aimock/*` routes at all; that lives in
`docs/control-api`.

## Verification

`typecheck` (all three configs) exit 0 · `build` exit 0 ·
release/publish-pin/drift-script workflow tests 44 passing · prettier
clean · commit body wrapped ≤100 cols for commitlint.

Full-suite note: five tests failed on a first run and all five passed in
isolation — `cli.test.ts` SIGTERM, `multimedia.test.ts` transcription
frame scheduling, `proxy-buffer-cap.test.ts`, and both
`publish-pin-workflow.test.ts` real-pip cases. All wall-clock or
network-bound, and none reachable from a version-string change. CI is
the arbiter.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01Y445N6QQBeAdiLcEvEpqGe
pull Bot pushed a commit to TheTechOddBug/aimock that referenced this pull request Sep 9, 2026
Cuts the accumulated Unreleased work: 72 non-merge commits since v1.39.0, 42 of
them feat/fix. Minor, not patch — two additive features, no breaking changes.

### Added

- GET /__aimock/fixtures — read-only fixture-count introspection (CopilotKit#407)
- Recorded OpenAI/OpenRouter token usage, including OpenRouter usage.cost (CopilotKit#368)
- AG-UI subagent lifecycle events + subagentRunId attribution (CopilotKit#391)

### Changed

- The AG-UI drift CI lane runs every agui-*.drift.ts guard, not one file

### Fixed

- X-AIMock-Strict is parsed case-insensitively, whitespace tolerated (CopilotKit#408)
- Non-websocket upgrade probes no longer 404, and the body survives on every
  supported Node (CopilotKit#410, CopilotKit#418)
- The AG-UI drift collector no longer reports clean for an unreadable failure
- The AG-UI drift canonical parser no longer drops a field after a trailing
  comment
- openrouter is no longer logged as an unknown SSE provider when recording

Version surfaces bumped — seven, verified by grepping the repo for the old
string rather than working from a list (zero occurrences remain):
package.json, charts/aimock/Chart.yaml appVersion, .claude-plugin/plugin.json,
.claude-plugin/marketplace.json (the `^` range under plugins[0].source),
packages/aimock-pytest/src/aimock_pytest/_version.py (the npm pin the pytest
harness downloads by default), docs/index.html's version badge, and
packages/aimock-pytest/README.md's documented default.

packages/aimock-pytest/pyproject.toml stays at 0.5.3 — that is the Python
package's own version, released on its own PyPI cadence.

README gains a Recorded-token-usage-and-cost bullet: the feature list
enumerates record/replay capabilities individually, so a user-visible one
landing without an entry there is a real omission. The new control-API route
needs no README line — the README documents no /__aimock/* routes at all; it is
covered in docs/control-api.

package.json's description is deliberately NOT hand-synced: publish-release.yml
rewrites it from the README subtitle in the runner before npm publish, without
committing, which is why git and the npm page differ by design.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants