From 16f95e18a61b2ddb1ae68b66141e6abe042bf379 Mon Sep 17 00:00:00 2001 From: Vishnu Vardhan Date: Thu, 15 Oct 2026 15:11:58 +0530 Subject: [PATCH 01/17] fix(core): take one BiDi network row per request from the inspector selenium-webdriver's NetworkInspector hands every network event to every subscriber's callback, so the request handler also received responses and the response handler also received requests. Every request in a Selenium Chrome trace was listed twice, a pending row beside the completed one, and 57 of 135 rows in a Nightwatch trace. Each handler now ignores the other's events by their shape. The inspector also models neither a redirect hop, which re-sends the request under the same id, nor a failed fetch, for which it has no hook. Rows are kept by id for the session so a hop continues its row under the first hop's clock, and network.fetchError is read off the inspector's own connection so a request the browser gives up on closes as a failure instead of staying pending. A browser that refuses that subscription keeps the inspector and loses only the failure rows. Co-Authored-By: Claude Fable 5.1 --- .changeset/bidi-inspector-dispatch.md | 6 + CLAUDE.md | 1 + packages/core/src/bidi.ts | 144 ++++++++++++-- packages/core/tests/bidi.test.ts | 269 +++++++++++++++++++++++++- 4 files changed, 405 insertions(+), 15 deletions(-) create mode 100644 .changeset/bidi-inspector-dispatch.md diff --git a/.changeset/bidi-inspector-dispatch.md b/.changeset/bidi-inspector-dispatch.md new file mode 100644 index 00000000..4f2a35d0 --- /dev/null +++ b/.changeset/bidi-inspector-dispatch.md @@ -0,0 +1,6 @@ +--- +'@wdio/selenium-devtools': patch +'@wdio/nightwatch-devtools': patch +--- + +Selenium and Nightwatch traces on Chrome listed every BiDi request twice, a pending row beside the completed one, because selenium-webdriver's network inspector delivers every network event to every subscriber. Each request is now one row, a redirect continues that row rather than adding one, and a request the browser gives up on is closed as a failure instead of staying pending. diff --git a/CLAUDE.md b/CLAUDE.md index ca16c55b..61080bb5 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -269,6 +269,7 @@ Documented divergences from the conventions above. They exist today as debt to b - `replaceCommand` has two semantics — Selenium mutates in place (preserves `_id`/`id` for chained calls); Nightwatch splices and reissues. Both call the same `core/suite-helpers` factories; the storage strategy stays adapter-specific because runner integrations differ. Could be unified by parameterizing the policy if the divergence ever causes a real problem. - `patchNodeAssert` (via `core/assert-patcher`) is now wired in all three adapters, default-on behind each adapter's `captureAssertions` option (opt out with `captureAssertions: false`). Framework matcher libraries differ: Service taps expect-webdriverio's `beforeAssertion`/`afterAssertion` hooks so passing+failing matchers render as `expect.*` actions (mechanism in the assert-capture entry below); Nightwatch native `assert`/`verify` and Selenium's `node:assert` also surface passing+failing rows via their reconcile/patch paths. The remaining gap is Selenium's jest-style `expect()` (and chai): jest/vitest expose no pass+fail assertion hook, so only failing matchers surface there. - BiDi is auto-attached in Service and Selenium; Nightwatch is opt-in via `bidi: true` and requires `webSocketUrl: true` in capabilities. +- **selenium-webdriver's `NetworkInspector` hands every network event to every subscriber's callback** (`_on` adds one unfiltered socket listener per subscription), so `core/bidi.ts` `handleBidiRequestSent` also receives response events and `handleBidiResponseCompleted` request events. Unguarded, the request handler pushed a second, never-completed row per request and the response handler finalized rows on request events: measured, every request in a Selenium Chrome trace listed twice (a status-0 row beside the completed one) and 57 of 135 rows in a Nightwatch one. Each handler now ignores the other's event by shape (`response` present or absent). Two more shapes the inspector does not model: a redirect hop re-sends the request under the same id, so rows are kept by id for the session (bounded at 5000, oldest out) and a request for a known id reopens its row under the first hop's clock rather than pushing one; and the inspector has no `fetchError` hook at all, so core subscribes to `network.fetchError` on the inspector's own connection and closes the row as a failure (status 0, the browser's `errorText`), as the WDIO service's `handleNetworkFetchError` already did. Without that, every blocked analytics beacon sat in the Network tab as pending for the life of the session; a browser that refuses the subscription keeps the inspector and only that gap. - Retry-aware trace policies share one mechanism: adapters feed a per-attempt **outcome ledger** (`core/attempt-tracker.ts` `TestAttemptTracker.recordStart(uid, specFile)` + `recordOutcome(uid, state, attempt)`) keyed by the **retry-stable** uid, and the finalizer reads the scoped views (`all`/`forSpec`/`forTest`) so `trace-retention.ts` evaluates **group-by-test** — `retain-on-failure` keys on each test's *final* attempt (no over-retaining a fail-then-pass), `retain-on-first-failure` on *attempt 0*. `recordStart` on a second attempt stamps the prior attempt `failed` (a retry only follows a failure), which corrects runners that swallow the intermediate failure — e.g. Mocha via a `--require` plugin never surfaces the retried attempt's failure, so the ledger would otherwise see `[passed, passed]`. An empty scoped view falls back to `testMetadata` (never fail-open-retains). - **WDIO + Selenium: verified end-to-end** (manual fail-then-pass runs: retained under `retain-on-first-failure`, dropped under `retain-on-failure`). - **Nightwatch: `retain-on-failure` works; the other retry-aware policies degrade.** Its `--retries` re-runs the testcase *internally* without re-firing the plugin's per-test hooks, and the per-testcase results carry no attempt/retry field (retries live only in undocumented, version-varying Nightwatch internals — `suiteRetries.testRetriesCount` / `reporter.testResults.retryTest`), so the ledger sees only the final attempt for the `describe/it` and exports-object interfaces. Cucumber scenarios expose per-scenario hooks, so the feed captures their attempts. Not cleanly fixable without depending on those internals. diff --git a/packages/core/src/bidi.ts b/packages/core/src/bidi.ts index 831449f2..30a3b938 100644 --- a/packages/core/src/bidi.ts +++ b/packages/core/src/bidi.ts @@ -57,6 +57,14 @@ interface LogInspector { interface NetworkInspector { beforeRequestSent: (cb: (e: unknown) => void) => Promise responseCompleted: (cb: (e: unknown) => void) => Promise + /** The BiDi connection the inspector subscribed on; it offers no hook for + * the events it does not model, so those are read off the socket. */ + bidi?: { + subscribe: (events: string | string[]) => Promise + socket?: { + on: (event: 'message', listener: (raw: unknown) => void) => void + } + } } export function handleBidiConsoleEntry( @@ -188,29 +196,58 @@ function bidiResponseTiming( return { endTime, time: Math.max(0, endTime - startTime) } } +/** Rows a session has seen, by request id: a later event for the same request + * (a redirect hop, a late failure) continues the row rather than adding one. */ +const MAX_TRACKED_REQUESTS = 5000 + +function remember( + rows: Map, + id: string, + entry: NetworkRequest +): void { + rows.set(id, entry) + if (rows.size > MAX_TRACKED_REQUESTS) { + const oldest = rows.keys().next().value + if (oldest !== undefined) { + rows.delete(oldest) + } + } +} + export function handleBidiRequestSent( rawEvent: unknown, - pending: Map, + rows: Map, sinks: BidiHandlerSinks, log: BidiLogger ): void { const event = rawEvent as BeforeRequestSentEvent try { const requestId = String(event?.request?.request ?? event?.id ?? '') - if (!requestId) { + // selenium-webdriver's NetworkInspector hands every network event to every + // subscriber's callback, so response and failure events reach this + // handler too; taken as requests they pushed or reopened rows. + const shape = rawEvent as { response?: unknown; errorText?: unknown } + if (!requestId || shape?.response || shape?.errorText) { return } + const previous = rows.get(requestId) const entry: NetworkRequest = { id: requestId, url: event?.request?.url ?? '', method: event?.request?.method ?? 'GET', requestHeaders: arrayHeadersToObject(event?.request?.headers), - timestamp: Date.now(), - startTime: Number(event?.timestamp ?? Date.now()), + // A redirect hop re-sends the request under the same id: one row, which + // keeps the clock of the hop it follows and reopens for the next answer. + timestamp: previous?.timestamp ?? Date.now(), + startTime: previous?.startTime ?? Number(event?.timestamp ?? Date.now()), type: getRequestType(event?.request?.url ?? '') } - pending.set(requestId, entry) - sinks.pushNetworkRequest(entry) + remember(rows, requestId, entry) + if (previous) { + sinks.replaceNetworkRequest(requestId, entry) + } else { + sinks.pushNetworkRequest(entry) + } } catch (err) { log('warn', `beforeRequestSent threw: ${errorMessage(err)}`) } @@ -218,15 +255,17 @@ export function handleBidiRequestSent( export function handleBidiResponseCompleted( rawEvent: unknown, - pending: Map, + rows: Map, sinks: BidiHandlerSinks, log: BidiLogger ): void { const event = rawEvent as ResponseCompletedEvent try { const requestId = String(event?.request?.request ?? event?.id ?? '') - const previous = pending.get(requestId) - if (!previous) { + const previous = rows.get(requestId) + // A request event delivered here (see handleBidiRequestSent) carries no + // response; finalizing on it emptied the row. + if (!previous || !event?.response) { return } const { endTime, time } = bidiResponseTiming( @@ -244,14 +283,85 @@ export function handleBidiResponseCompleted( time, size: Number(event?.response?.bytesReceived) || undefined } - pending.delete(requestId) + remember(rows, requestId, finalized) sinks.replaceNetworkRequest(requestId, finalized) } catch (err) { log('warn', `responseCompleted threw: ${errorMessage(err)}`) } } -async function attachNetworkInspector( +interface FetchErrorEvent { + request?: { request?: string; timings?: FetchTimings } + id?: string + timestamp?: number + errorText?: string +} + +/** A request the browser gave up on: closed as a failure, as the WDIO service + * closes its own, so it never sits in the Network tab as pending. */ +export function handleBidiFetchError( + rawEvent: unknown, + rows: Map, + sinks: BidiHandlerSinks, + log: BidiLogger +): void { + const event = rawEvent as FetchErrorEvent + try { + const requestId = String(event?.request?.request ?? event?.id ?? '') + const previous = rows.get(requestId) + if (!previous) { + return + } + const { endTime, time } = bidiResponseTiming( + event?.request?.timings, + previous.startTime, + event?.timestamp + ) + const errorText = event?.errorText ?? 'Failed' + const finalized: NetworkRequest = { + ...previous, + status: 0, + statusText: errorText, + error: errorText, + endTime, + time + } + remember(rows, requestId, finalized) + sinks.replaceNetworkRequest(requestId, finalized) + } catch (err) { + log('warn', `fetchError threw: ${errorMessage(err)}`) + } +} + +/** The inspector models requests and responses only; a failed fetch is read + * off the socket it subscribed on, or the row would stay pending forever. */ +async function subscribeFetchErrors( + inspector: NetworkInspector, + rows: Map, + sinks: BidiHandlerSinks, + log: BidiLogger +): Promise { + const bidi = inspector.bidi + if (typeof bidi?.subscribe !== 'function' || !bidi.socket?.on) { + return + } + await bidi.subscribe('network.fetchError') + bidi.socket.on('message', (raw) => { + try { + const { method, params } = JSON.parse(String(raw)) as { + method?: string + params?: unknown + } + if (method === 'network.fetchError') { + handleBidiFetchError(params, rows, sinks, log) + } + } catch { + /* a frame that is not JSON is not an event */ + } + }) +} + +export async function attachNetworkInspector( driver: unknown, factory: InspectorFactory, sinks: BidiHandlerSinks, @@ -259,13 +369,19 @@ async function attachNetworkInspector( ): Promise { try { const inspector = (await factory(driver)) as NetworkInspector - const pending = new Map() + const rows = new Map() await inspector.beforeRequestSent((e) => - handleBidiRequestSent(e, pending, sinks, log) + handleBidiRequestSent(e, rows, sinks, log) ) await inspector.responseCompleted((e) => - handleBidiResponseCompleted(e, pending, sinks, log) + handleBidiResponseCompleted(e, rows, sinks, log) ) + try { + await subscribeFetchErrors(inspector, rows, sinks, log) + } catch (err) { + // The inspector is attached either way; only failed requests stay pending. + log('warn', `BiDi network.fetchError unavailable: ${errorMessage(err)}`) + } log('info', '✓ BiDi NetworkInspector attached (request + response)') return true } catch (err) { diff --git a/packages/core/tests/bidi.test.ts b/packages/core/tests/bidi.test.ts index cf17e4c2..61ab63fd 100644 --- a/packages/core/tests/bidi.test.ts +++ b/packages/core/tests/bidi.test.ts @@ -5,6 +5,8 @@ import { loadSeleniumSubmodule, handleBidiConsoleEntry, handleBidiJsException, + attachNetworkInspector, + handleBidiFetchError, handleBidiRequestSent, handleBidiResponseCompleted, type BidiHandlerSinks @@ -242,7 +244,8 @@ describe('handleBidiResponseCompleted', () => { expect(replacements[0].entry.statusText).toBe('Created') expect(replacements[0].entry.size).toBe(1024) // pending entry is consumed - expect(pending.has('req-2')).toBe(false) + // Kept: a redirect hop or a late failure for this id continues the row. + expect(pending.has('req-2')).toBe(true) // pushNetworkRequest fired during requestSent, replaceNetworkRequest on completion expect(networkRequests).toHaveLength(1) }) @@ -296,6 +299,270 @@ describe('handleBidiResponseCompleted', () => { }) }) +describe('handleBidi* under selenium-webdriver dispatch', () => { + // selenium-webdriver's NetworkInspector hands every network event to every + // subscriber's callback, so each handler also receives the other's events. + const requestEvent = { + request: { request: 'R1', url: 'https://x/', method: 'GET', headers: [] }, + timestamp: 1, + initiator: { type: 'other' } + } + const responseEvent = { + request: { request: 'R1', url: 'https://x/', method: 'GET', headers: [] }, + timestamp: 2, + response: { + status: 200, + headers: [{ name: 'content-type', value: 'text/html' }] + } + } + + it('ignores a response event delivered to the request handler', () => { + const { sinks, networkRequests } = makeSinks() + const pending = new Map() + handleBidiRequestSent(responseEvent, pending, sinks, silentLog) + expect(networkRequests).toEqual([]) + expect(pending.size).toBe(0) + }) + + it('ignores a request event delivered to the response handler', () => { + const { sinks, replacements } = makeSinks() + const pending = new Map() + handleBidiRequestSent(requestEvent, pending, sinks, silentLog) + handleBidiResponseCompleted(requestEvent, pending, sinks, silentLog) + expect(replacements).toEqual([]) + expect(pending.has('R1')).toBe(true) + }) + + it('leaves one row per request when both handlers see both events', () => { + // The sequence as the inspector delivers it: every event to every handler. + const { sinks, networkRequests, replacements } = makeSinks() + const pending = new Map() + for (const event of [requestEvent, responseEvent]) { + handleBidiRequestSent(event, pending, sinks, silentLog) + handleBidiResponseCompleted(event, pending, sinks, silentLog) + } + expect(networkRequests).toHaveLength(1) + expect(replacements).toHaveLength(1) + expect(replacements[0]).toMatchObject({ + id: 'R1', + entry: { id: 'R1', status: 200 } + }) + expect(pending.size).toBe(1) + }) + + it('ignores a failure event delivered to the request handler', () => { + const { sinks, networkRequests, replacements } = makeSinks() + const pending = new Map() + handleBidiRequestSent(requestEvent, pending, sinks, silentLog) + handleBidiRequestSent( + { + request: { request: 'R1' }, + timestamp: 3, + errorText: 'net::ERR_BLOCKED' + }, + pending, + sinks, + silentLog + ) + expect(networkRequests).toHaveLength(1) + expect(replacements).toEqual([]) + }) + + it('moves a redirected request’s row instead of adding one', () => { + const { sinks, networkRequests, replacements } = makeSinks() + const pending = new Map() + handleBidiRequestSent(requestEvent, pending, sinks, silentLog) + handleBidiRequestSent( + { + ...requestEvent, + request: { ...requestEvent.request, url: 'https://x/moved' }, + redirectCount: 1 + }, + pending, + sinks, + silentLog + ) + expect(networkRequests).toHaveLength(1) + expect(replacements).toEqual([ + { id: 'R1', entry: expect.objectContaining({ url: 'https://x/moved' }) } + ]) + }) + + it('moves a redirect hop onto the row its completed hop left', () => { + // Chrome completes the 302 before re-sending the request under the same + // id; the row left `pending` with it, and a fresh push listed the hop + // twice: once finalized, once pending for the rest of the session. + const { sinks, networkRequests, replacements } = makeSinks() + const pending = new Map() + handleBidiRequestSent(requestEvent, pending, sinks, silentLog) + handleBidiResponseCompleted( + { ...responseEvent, response: { status: 302, headers: [] } }, + pending, + sinks, + silentLog + ) + handleBidiRequestSent( + { + ...requestEvent, + request: { ...requestEvent.request, url: 'https://x/secure' }, + redirectCount: 1 + }, + pending, + sinks, + silentLog + ) + handleBidiResponseCompleted(responseEvent, pending, sinks, silentLog) + expect(networkRequests).toHaveLength(1) + // The row ends on the url the chain landed on, with the final status. + expect(replacements.map((r) => r.entry.url)).toEqual([ + 'https://x/', + 'https://x/secure', + 'https://x/secure' + ]) + expect(replacements.at(-1)?.entry.status).toBe(200) + // The chain's clock is the first hop's, so its duration spans the chain. + expect(replacements.at(-1)?.entry.startTime).toBe(1) + expect(pending.size).toBe(1) + }) + + it('closes a request the browser gave up on as a failure, not a pending row', () => { + const { sinks, replacements } = makeSinks() + const pending = new Map() + handleBidiRequestSent(requestEvent, pending, sinks, silentLog) + handleBidiFetchError( + { + request: { request: 'R1' }, + timestamp: 3, + errorText: 'net::ERR_BLOCKED' + }, + pending, + sinks, + silentLog + ) + expect(replacements).toHaveLength(1) + expect(replacements[0].entry).toMatchObject({ + id: 'R1', + status: 0, + statusText: 'net::ERR_BLOCKED', + error: 'net::ERR_BLOCKED' + }) + expect(pending.size).toBe(1) + }) + + it('ignores a failure for a request it never saw', () => { + const { sinks, replacements } = makeSinks() + handleBidiFetchError( + { request: { request: 'R9' }, errorText: 'net::ERR_BLOCKED' }, + new Map(), + sinks, + silentLog + ) + expect(replacements).toEqual([]) + }) +}) + +describe('attachNetworkInspector and the socket it reads failures from', () => { + /** A selenium-webdriver inspector: callbacks per event, and the BiDi + * connection it subscribed on, whose socket carries every frame. */ + function fakeInspector({ refuse = false } = {}) { + const callbacks: Record void> = {} + const listeners: ((raw: unknown) => void)[] = [] + const subscribed: string[] = [] + const inspector = { + beforeRequestSent: async (cb: (e: unknown) => void) => { + callbacks.request = cb + }, + responseCompleted: async (cb: (e: unknown) => void) => { + callbacks.response = cb + }, + bidi: { + subscribe: async (events: string | string[]) => { + if (refuse) { + throw new Error('unknown event') + } + subscribed.push(...(Array.isArray(events) ? events : [events])) + }, + socket: { + on: (_event: 'message', listener: (raw: unknown) => void) => { + listeners.push(listener) + } + } + } + } + const raw = (text: string) => + listeners.forEach((listener) => listener(Buffer.from(text))) + const frame = (method: string, params: unknown) => + raw(JSON.stringify({ method, params })) + return { inspector, callbacks, subscribed, frame, raw } + } + + it('closes a request the browser gave up on through the socket', async () => { + const { sinks, replacements } = makeSinks() + const { inspector, callbacks, subscribed, frame } = fakeInspector() + const attached = await attachNetworkInspector( + {}, + async () => inspector, + sinks, + silentLog + ) + expect(attached).toBe(true) + expect(subscribed).toEqual(['network.fetchError']) + callbacks.request({ + request: { request: 'R1', url: 'https://x/beacon', method: 'GET' }, + timestamp: 1, + initiator: { type: 'other' } + }) + frame('network.fetchError', { + request: { request: 'R1' }, + timestamp: 2, + errorText: 'net::ERR_BLOCKED_BY_CLIENT' + }) + expect(replacements).toHaveLength(1) + expect(replacements[0].entry).toMatchObject({ + status: 0, + error: 'net::ERR_BLOCKED_BY_CLIENT' + }) + }) + + it('reads only failure frames off the socket, and only JSON ones', async () => { + const { sinks, networkRequests, replacements } = makeSinks() + const { inspector, callbacks, frame, raw } = fakeInspector() + await attachNetworkInspector({}, async () => inspector, sinks, silentLog) + callbacks.request({ + request: { request: 'R1', url: 'https://x/', method: 'GET' }, + timestamp: 1, + initiator: { type: 'other' } + }) + frame('network.responseCompleted', { + request: { request: 'R1' }, + response: { status: 200, headers: [] } + }) + expect(() => raw('not json')).not.toThrow() + expect(replacements).toEqual([]) + expect(networkRequests).toHaveLength(1) + }) + + it('stays attached when the browser refuses the failure subscription', async () => { + const warnings: string[] = [] + const { sinks, networkRequests } = makeSinks() + const { inspector, callbacks } = fakeInspector({ refuse: true }) + const attached = await attachNetworkInspector( + {}, + async () => inspector, + sinks, + (level, message) => level === 'warn' && warnings.push(message) + ) + expect(attached).toBe(true) + expect(warnings.join('\n')).toContain('network.fetchError unavailable') + callbacks.request({ + request: { request: 'R1', url: 'https://x/', method: 'GET' }, + timestamp: 1, + initiator: { type: 'other' } + }) + expect(networkRequests).toHaveLength(1) + }) +}) + describe('attachBidiHandlers — graceful degradation', () => { // Two real-world failure modes the function must handle without crashing: // (a) submodules unresolvable → "not available" notice, returns false From 146e214b211c982b13a3f00d55cbdd9ce345fd16 Mon Sep 17 00:00:00 2001 From: Vishnu Vardhan Date: Thu, 15 Oct 2026 15:11:58 +0530 Subject: [PATCH 02/17] feat(script): capture network from Resource Timing where BiDi is absent Network capture rode on WebDriver BiDi, or on Chrome's performance log in Nightwatch, so Safari and every Appium session showed an empty Network tab. The injected page collector now also reads the document's Resource Timing buffer, which every browser keeps: each load with its timing phases and size, on Chromium its status, never headers or bodies. It reads the buffer at install so loads made before the collector existed are included, then observes. The collector's fetch/XHR patch claims each call it sees, so a request both saw is listed once with the patch's headers and body, and an entry that arrives before the call has answered is held until the patch records or drops that call. A driver-side source wins once it has delivered. The gate is evidence, not a subscription: on Appium 3.7's XCUITest driver the BiDi inspectors attach and then send nothing, so each adapter latches on the first BiDi network event it receives and gates the page's rows on that. Nightwatch's performance log, read after the drain, lays its headers and status over the page's row for the same load instead of listing it again, and a load the log listed first is dropped from the next drain. The Python adapter also stops treating safaridriver's echoed `webSocketUrl: true` as a BiDi endpoint, which crashed selenium's connection thread. What Resource Timing cannot see is stated rather than left blank: the row carries its source, the dashboard colours a status-less row as unknown rather than pending and names the limit on the detail card, and the HAR entry carries `_source` and `_resourceType` so a reopened trace renders the same row rather than a failed request. Verified on desktop Safari 26.5 and on iOS 26.5 Safari through Appium 3.7 with all four adapters; Chrome sessions are unchanged. Co-Authored-By: Claude Fable 5.1 --- .changeset/network-resource-timing.md | 12 + CLAUDE.md | 4 +- README.md | 8 +- examples/MOBILE.md | 17 +- .../app/src/components/workbench/network.ts | 14 +- .../workbench/network/request-detail.ts | 34 +- .../components/workbench/network/styles.ts | 3 + packages/app/src/utils/network-constants.ts | 10 +- packages/app/src/utils/network-helpers.ts | 28 +- .../test-ui/workbench/panels/network.test.ts | 70 +++- packages/app/tests/network-helpers.test.ts | 27 ++ packages/backend/src/trace-reader-types.ts | 8 + packages/backend/src/trace-reader-utils.ts | 60 ++- .../trace-reader-resource-timing.test.ts | 126 ++++++ packages/core/src/request-type.ts | 57 +-- packages/core/src/session-capturer.ts | 61 ++- packages/core/tests/request-type.test.ts | 93 +---- .../core/tests/session-capturer-base.test.ts | 58 ++- packages/nightwatch-devtools/README.md | 2 +- packages/nightwatch-devtools/src/bidi.ts | 2 + .../src/helpers/perfLogs.ts | 133 +++++- .../nightwatch-devtools/src/session-init.ts | 3 +- packages/nightwatch-devtools/src/session.ts | 59 ++- .../tests/perfLogs.test.ts | 245 +++++++++++ .../nightwatch-devtools/tests/session.test.ts | 178 ++++++++ packages/script/src/collector.ts | 13 +- .../script/src/collectors/network-helpers.ts | 51 +++ .../script/src/collectors/networkRequests.ts | 154 +++++-- .../script/src/collectors/resourceTiming.ts | 221 ++++++++++ packages/script/tests/collector.test.ts | 64 ++- packages/script/tests/networkRequests.test.ts | 155 +++++++ packages/script/tests/resourceTiming.test.ts | 383 ++++++++++++++++++ packages/selenium-devtools-py/README.md | 8 +- .../changes/network-without-bidi.md | 7 + .../src/selenium_devtools/_wire_types.py | 14 +- .../src/selenium_devtools/bidi.py | 26 +- .../src/selenium_devtools/bidi_preload.py | 4 +- .../src/selenium_devtools/capturer.py | 7 + .../src/selenium_devtools/instrumentation.py | 30 +- .../src/selenium_devtools/snapshot.py | 45 +- .../selenium-devtools-py/tests/test_bidi.py | 60 +++ .../tests/test_bidi_preload.py | 14 +- .../tests/test_capturer.py | 15 + .../tests/test_instrumentation.py | 65 +++ .../tests/test_snapshot.py | 52 ++- packages/selenium-devtools/README.md | 2 +- packages/selenium-devtools/src/bidi.ts | 3 + .../src/session-lifecycle.ts | 2 +- packages/selenium-devtools/src/session.ts | 11 +- .../selenium-devtools/tests/session.test.ts | 82 ++++ packages/service/src/session.ts | 9 +- packages/service/tests/session.test.ts | 52 +++ packages/shared/src/index.ts | 1 + packages/shared/src/request-type.ts | 54 +++ packages/shared/src/types.ts | 48 ++- packages/shared/tests/request-type.test.ts | 86 ++++ packages/shared/wire-schema.json | 37 ++ packages/trace/src/trace-har.ts | 54 ++- packages/trace/tests/trace-har.test.ts | 72 ++++ 59 files changed, 2900 insertions(+), 313 deletions(-) create mode 100644 .changeset/network-resource-timing.md create mode 100644 packages/backend/tests/trace-reader-resource-timing.test.ts create mode 100644 packages/script/src/collectors/network-helpers.ts create mode 100644 packages/script/src/collectors/resourceTiming.ts create mode 100644 packages/script/tests/resourceTiming.test.ts create mode 100644 packages/selenium-devtools-py/changes/network-without-bidi.md create mode 100644 packages/shared/src/request-type.ts create mode 100644 packages/shared/tests/request-type.test.ts create mode 100644 packages/trace/tests/trace-har.test.ts diff --git a/.changeset/network-resource-timing.md b/.changeset/network-resource-timing.md new file mode 100644 index 00000000..9df52ce0 --- /dev/null +++ b/.changeset/network-resource-timing.md @@ -0,0 +1,12 @@ +--- +'@wdio/devtools-script': minor +'@wdio/devtools-service': minor +'@wdio/selenium-devtools': minor +'@wdio/nightwatch-devtools': minor +'@wdio/devtools-backend': patch +'@wdio/devtools-app': patch +--- + +Capture network requests on browsers without WebDriver BiDi: Safari, and every session Appium serves. Network capture rode on BiDi (or, in Nightwatch, Chrome's performance log), so those sessions showed an empty Network tab. The injected page collector now also reads the document's Resource Timing buffer, which every browser keeps: the document, scripts, styles, images and fonts, each with its timing phases and size, and on Chromium its status. It reads the buffer at install, so loads made before the collector existed are included, then observes. The collector's existing `fetch`/`XMLHttpRequest` patch keeps supplying headers, status and bodies for the calls the page makes, and a request both saw is listed once. + +What Resource Timing cannot see is stated rather than left blank: the detail card of such a request names that this browser exposes no headers or body for it (and on Safari no status), a request with no status is coloured as unknown rather than as still in flight, the method column is a dash where the browser names none, and the trace's `trace.network` entry carries `_source: "resource-timing"` and the captured `_resourceType` so a reopened trace renders the same row rather than a failed request. When BiDi is serving a session, the Resource Timing rows are dropped at ingest, so Chrome sessions are unchanged. Serving means delivering, not subscribed: against Appium 3.7's XCUITest driver the BiDi inspectors attach and then send no event, so each adapter now counts BiDi as a source only once a network event has arrived (the Selenium adapter applies the same rule to its console gate). Nightwatch's performance log, which is read after the page is drained, now upgrades the page's row for each load in place with the headers and status it has, so a Chrome session lists each request once and a Safari or Appium session whose capabilities still name a performance log is not left with an empty tab. A native app session has no document and still has no network capture, because the driver exposes none. diff --git a/CLAUDE.md b/CLAUDE.md index 61080bb5..86be3284 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -269,7 +269,9 @@ Documented divergences from the conventions above. They exist today as debt to b - `replaceCommand` has two semantics — Selenium mutates in place (preserves `_id`/`id` for chained calls); Nightwatch splices and reissues. Both call the same `core/suite-helpers` factories; the storage strategy stays adapter-specific because runner integrations differ. Could be unified by parameterizing the policy if the divergence ever causes a real problem. - `patchNodeAssert` (via `core/assert-patcher`) is now wired in all three adapters, default-on behind each adapter's `captureAssertions` option (opt out with `captureAssertions: false`). Framework matcher libraries differ: Service taps expect-webdriverio's `beforeAssertion`/`afterAssertion` hooks so passing+failing matchers render as `expect.*` actions (mechanism in the assert-capture entry below); Nightwatch native `assert`/`verify` and Selenium's `node:assert` also surface passing+failing rows via their reconcile/patch paths. The remaining gap is Selenium's jest-style `expect()` (and chai): jest/vitest expose no pass+fail assertion hook, so only failing matchers surface there. - BiDi is auto-attached in Service and Selenium; Nightwatch is opt-in via `bidi: true` and requires `webSocketUrl: true` in capabilities. -- **selenium-webdriver's `NetworkInspector` hands every network event to every subscriber's callback** (`_on` adds one unfiltered socket listener per subscription), so `core/bidi.ts` `handleBidiRequestSent` also receives response events and `handleBidiResponseCompleted` request events. Unguarded, the request handler pushed a second, never-completed row per request and the response handler finalized rows on request events: measured, every request in a Selenium Chrome trace listed twice (a status-0 row beside the completed one) and 57 of 135 rows in a Nightwatch one. Each handler now ignores the other's event by shape (`response` present or absent). Two more shapes the inspector does not model: a redirect hop re-sends the request under the same id, so rows are kept by id for the session (bounded at 5000, oldest out) and a request for a known id reopens its row under the first hop's clock rather than pushing one; and the inspector has no `fetchError` hook at all, so core subscribes to `network.fetchError` on the inspector's own connection and closes the row as a failure (status 0, the browser's `errorText`), as the WDIO service's `handleNetworkFetchError` already did. Without that, every blocked analytics beacon sat in the Network tab as pending for the life of the session; a browser that refuses the subscription keeps the inspector and only that gap. +- **Network has three sources, and the page's Resource Timing is the floor.** BiDi (all four adapters) and Nightwatch's Chrome performance log are driver-side and carry headers and status. The page collector carries two: its patched `fetch`/`XMLHttpRequest` (the only source with bodies) and the document's Resource Timing buffer (`script/src/collectors/resourceTiming.ts`): every load with timing phases and size, a status on Chromium only, never headers. Resource Timing is the only network source a browser without BiDi has, which is Safari and every Appium session (Appium's BiDi reference lists `session.*` alone). The collector reads the buffer at construction, which is what covers loads made before a `