fix: capture network on Safari and Appium sessions without BiDi

.changeset/bidi-inspector-dispatch.md

@@ -0,0 +1,6 @@
+---
+'@wdio/selenium-devtools': patch
+'@wdio/nightwatch-devtools': patch
+---
+
+Selenium and Nightwatch traces on Chrome listed every BiDi request twice, a pending row beside the completed one, because selenium-webdriver's network inspector delivers every network event to every subscriber. Each request is now one row, a redirect continues that row rather than adding one, and a request the browser gives up on is closed as a failure instead of staying pending.

.changeset/network-resource-timing.md

@@ -0,0 +1,12 @@
+---
+'@wdio/devtools-script': minor
+'@wdio/devtools-service': minor
+'@wdio/selenium-devtools': minor
+'@wdio/nightwatch-devtools': minor
+'@wdio/devtools-backend': patch
+'@wdio/devtools-app': patch
+---
+
+Capture network requests on browsers without WebDriver BiDi: Safari, and every session Appium serves. Network capture rode on BiDi (or, in Nightwatch, Chrome's performance log), so those sessions showed an empty Network tab. The injected page collector now also reads the document's Resource Timing buffer, which every browser keeps: the document, scripts, styles, images and fonts, each with its timing phases and size, and on Chromium its status. It reads the buffer at install, so loads made before the collector existed are included, then observes. The collector's existing `fetch`/`XMLHttpRequest` patch keeps supplying headers, status and bodies for the calls the page makes, and a request both saw is listed once.
+
+What Resource Timing cannot see is stated rather than left blank: the detail card of such a request names that this browser exposes no headers or body for it (and on Safari no status), a request with no status is coloured as unknown rather than as still in flight, the method column is a dash where the browser names none, and the trace's `trace.network` entry carries `_source: "resource-timing"` and the captured `_resourceType` so a reopened trace renders the same row rather than a failed request. When BiDi is serving a session, the Resource Timing rows are dropped at ingest, so Chrome sessions are unchanged. Serving means delivering, not subscribed: against Appium 3.7's XCUITest driver the BiDi inspectors attach and then send no event, so each adapter now counts BiDi as a source only once a network event has arrived (the Selenium adapter applies the same rule to its console gate). Nightwatch's performance log, which is read after the page is drained, now upgrades the page's row for each load in place with the headers and status it has, so a Chrome session lists each request once and a Safari or Appium session whose capabilities still name a performance log is not left with an empty tab. A native app session has no document and still has no network capture, because the driver exposes none. A mobile Chrome session, which opens on the browser's own new-tab page before the first navigation, no longer lists that page, and a second injection of the collector into a document keeps the first one instead of replacing it and dropping what it had buffered. A `driver.wait()` in a Selenium test now shows as the one command the test made rather than a row per poll. Safari delivers synthesized clicks only to its frontmost window and reports success either way, so once the live dashboard took the front every click reached nothing. The Selenium adapter now brings the session's own window forward before each element click, key input, clear or submit on Safari, and opens its dashboard behind it; the WDIO service, Nightwatch and Python bring Safari forward once after the session starts. The collector injected into a loaded page no longer waits for an animation frame before it starts listening, which on Safari under automation could be never, so a live run on Safari no longer replays an unchanging page with empty fields. On a browser without document-start injection (Safari, Appium), the Selenium adapter's navigation now resolves only once the collector is in the destination, so typing that follows a navigation is captured instead of racing the injection, and the collector's DOM anchor carries each form control's current value and checked state, so a form typed into before the collector arrived no longer replays empty.

README.md

@@ -51,7 +51,7 @@ Want a portable artifact instead of a live UI (CI / agent diffing)? Switch to **
 - **Screenshot Capture**: Automatic screenshots after each command for visual debugging
 - **Source Code Mapping**: View the exact line of code that triggered each command
 - **Console Logs**: Capture and display application console output with timestamps and log levels
-- **Network Logs**: Monitor and inspect HTTP requests/responses including headers, payloads, timing, and status codes
+- **Network Logs**: Monitor and inspect HTTP requests/responses including headers, payloads, timing, and status codes. Where the driver exposes no BiDi (Safari, Appium), the page's own Resource Timing still lists every request with its timing and size
 - **Error Tracking**: Full error messages and stack traces for failed tests
 
 ### 🎮 Execution Controls
@@ -91,6 +91,8 @@ Real-time capture of browser-side events through the WebDriver BiDi protocol —
 
 When BiDi is active in Selenium or Nightwatch, the per-command Chrome performance-log network-capture path is gated off so requests don't appear twice in the dashboard. The attach + sink logic lives in `@wdio/devtools-core`'s `bidi.ts` — same module both adapters consume.
 
+**Without BiDi: Safari, and every session Appium serves.** Neither driver exposes BiDi, so there the Network tab is filled by the page itself, through the same injected collector that records the DOM. It patches `fetch` / `XMLHttpRequest` (method, headers, status and body of each call the page makes) and reads the document's **Resource Timing** buffer: every load the browser made (the document, scripts, styles, images, fonts) with its timing phases and size, plus a status on Chromium. What Resource Timing cannot see are headers and bodies, and on Safari the status; each such row says so in its detail card, and its `trace.network` entry carries `_source: "resource-timing"`. When BiDi is serving the session, meaning it has delivered a network event and not merely accepted a subscription (Appium's BiDi shim accepts one and sends nothing), the Resource Timing rows are dropped at ingest so a request is listed once; Nightwatch's performance log, read after the page is drained, upgrades those rows in place with the headers and status it has. A native app session has no document, so it has no network capture at all. Only a device- or proxy-level capture outside the adapters can see that traffic.
+
 ### 📦 Trace mode (trace.zip)
 
 The dashboard runs in one of **two modes**, set per adapter via the shared `mode` option:
@@ -206,7 +208,7 @@ Appium sessions are captured in both live and trace mode. What gets captured dep
 
 | Session | Has a document | What is captured |
 |---|---|---|
-| **Mobile browser** (Chrome on Android, Safari on iOS) | always | the same as a desktop run — DOM, console, network, replay |
+| **Mobile browser** (Chrome on Android, Safari on iOS) | always | DOM, console and replay as on desktop; network from the page's own record, since Appium serves no BiDi (see [BiDi capture](#-bidi-capture-browser-console--js-exceptions--network)) |
 | **Native app** | never | commands, screenshots and element data from the platform's XML tree; no DOM |
 | **Hybrid app** | only while in a webview context | native halves as an app, webview halves as a page |
 
@@ -218,7 +220,7 @@ A hybrid app is the case capabilities alone cannot answer, because the answer ch
 
 **In the dashboard**, a phone capture is framed as the device — a full-height device column with the action list and dock beside it — in both live mode and the trace player. The trace's `context-options` records the platform, model and OS version, read back so the player labels frames without guessing.
 
-**What a mobile trace does not contain.** Per-action snapshots are issued from inside the command hook, and Appium serialises a probe behind the command it is observing — so where a session has a document to probe, the per-action snapshot is skipped. In practice that means **a hybrid app's webview actions carry no per-action element data, accessibility tree or settle screenshot**. Native sessions, mobile browsers and every desktop session are unaffected. Command rows and their screenshots, console, network and the archive itself are always captured. These absences are by design; they are not a capture failure.
+**What a mobile trace does not contain.** Per-action snapshots are issued from inside the command hook, and Appium serialises a probe behind the command it is observing — so where a session has a document to probe, the per-action snapshot is skipped. In practice that means **a hybrid app's webview actions carry no per-action element data, accessibility tree or settle screenshot**. Native sessions, mobile browsers and every desktop session are unaffected. Command rows and their screenshots, console and the archive itself are always captured; a mobile browser's network comes from the page's Resource Timing and fetch/XHR patch rather than from BiDi, and a native session has no network capture, because the driver exposes none. These absences are by design; they are not a capture failure.
 
 On a native session, element data and the A11y panel come from the platform's XML tree rather than the DOM, so locators read as `android=new UiSelector()…` / `-ios predicate string:…` rather than CSS or XPath over HTML.
 

examples/MOBILE.md

@@ -318,11 +318,19 @@ instrumentation process is not running`, so the trace ends up with fewer
 
 Not failures, and not worth chasing:
 
-- **Selenium logs three BiDi warnings per run** (`BiDi LogInspector attach
-failed`, `BiDi preload unavailable`, `BiDi NetworkInspector attach failed`).
-  The adapter requests `webSocketUrl` unconditionally and Appium serves no
-  BiDi, so the attach fails and capture falls back to per-document injection —
-  which is the correct path for a native session anyway.
+- **Selenium logs BiDi lines that describe nothing real.** The adapter
+  requests `webSocketUrl` unconditionally. Against UiAutomator2 the attach
+  fails and logs three warnings (`BiDi LogInspector attach failed`, `BiDi
+  preload unavailable`, `BiDi NetworkInspector attach failed`); against
+  Appium 3.7's XCUITest driver the inspectors report `attached` and then
+  deliver no event at all, which is why the adapter counts BiDi as a source
+  only once an event has arrived. Either way capture runs on per-document
+  injection, which is the correct path for a native session anyway. On the
+  `web` target that injected collector is also what fills the Network tab,
+  from two sources of the page's own: a `fetch` / `XMLHttpRequest` call goes
+  through its patch and carries headers, status and body, and every other
+  load comes from the page's Resource Timing buffer with its timing and size,
+  no headers or bodies, and on Safari no status.
 - **An inherited `DEVTOOLS_MODE=live` changes what a run produces** — a
   dashboard window and no zip, instead of a zip and no window. That is correct
   behaviour for live mode and surprising when the variable is left over from an

examples/wdio/mocha/wdio.conf.ts

@@ -27,7 +27,9 @@ export const config: WebdriverIO.Config = {
           '--headless',
           '--disable-gpu',
           '--remote-allow-origins=*',
-          '--window-size=1600,900'
+          '--window-size=1600,900',
+          // Chrome's password-leak check drops all WebDriver input after the demo credential is submitted; see CLAUDE.md.
+          '--host-resolver-rules=MAP passwordsleakcheck-pa.googleapis.com 127.0.0.1'
         ]
       }
     }

packages/app/src/components/workbench/network.ts

@@ -7,13 +7,13 @@ import { consume } from '@lit/context'
 import { networkRequestContext } from '../../controller/context.js'
 import {
   FAILED_STATUS_LABEL,
+  NO_VALUE,
   RESOURCE_TYPES,
   TYPE_DOT_CLASS,
   type ResourceFilter
 } from '../../utils/network-constants.js'
 import {
   formatTime,
-  statusKind,
   getResourceType,
   getFileName,
   contentType
@@ -27,7 +27,8 @@ import {
 import {
   formatTransferSize,
   renderNetworkRequestDetail,
-  requestFailed
+  requestFailed,
+  requestStatusKind
 } from './network/request-detail.js'
 
 import '../placeholder.js'
@@ -143,7 +144,7 @@ export class DevtoolsNetwork extends Element {
   }
 
   #renderRequestRow(request: NetworkRequest, range: WaterfallScale) {
-    const kind = statusKind(request.status, requestFailed(request))
+    const kind = requestStatusKind(request)
     const dotClass = TYPE_DOT_CLASS[getResourceType(request)]
     // A zero duration is a measurement (a cached or same-tick response), so the
     // cell reports it; only the bar is held back, because a zero-width bar draws
@@ -155,7 +156,8 @@ export class DevtoolsNetwork extends Element {
     // as ERR so a dead request is never mistaken for one still in flight, which
     // is what the dash means.
     const statusLabel =
-      request.status || (requestFailed(request) ? FAILED_STATUS_LABEL : '—')
+      request.status ||
+      (requestFailed(request) ? FAILED_STATUS_LABEL : NO_VALUE)
     return html`
       <div
         class="grid request-row ${
@@ -169,7 +171,7 @@ export class DevtoolsNetwork extends Element {
             >${getFileName(request.url)}</span
           >
         </span>
-        <span class="req-method">${request.method}</span>
+        <span class="req-method">${request.method || NO_VALUE}</span>
         <span class="req-status kind-${kind}">
           <i class="status-dot"></i>
           ${statusLabel}
@@ -190,7 +192,7 @@ export class DevtoolsNetwork extends Element {
           </span>
         </span>
         <span class="req-dur ${duration === undefined ? 'req-dur-empty' : ''}"
-          >${duration === undefined ? '—' : formatTime(duration)}</span
+          >${duration === undefined ? NO_VALUE : formatTime(duration)}</span
         >
         <span class="req-size">${formatTransferSize(request.size)}</span>
       </div>

packages/app/src/components/workbench/network/request-detail.ts

@@ -3,23 +3,27 @@
 // network drawer (TraceTimeline). The returned markup relies on `networkStyles`
 // being present in the host component's shadow root.
 
-import type { NetworkRequest } from '@wdio/devtools-shared'
+import {
+  NETWORK_CAPTURE_SOURCE,
+  type NetworkRequest
+} from '@wdio/devtools-shared'
 import { html, nothing, type TemplateResult } from 'lit'
 import {
   formatBytes,
   formatTime,
   statusKind,
   contentType
 } from '../../../utils/network-helpers.js'
-import { FAILED_STATUS_LABEL } from '../../../utils/network-constants.js'
+import {
+  FAILED_STATUS_LABEL,
+  NO_VALUE
+} from '../../../utils/network-constants.js'
 
 // The capture side reports a transport failure as status 0 carrying the failure
 // text in `statusText` (service's `handleNetworkFetchError`), so 0 is a failure
 // that happened — never a status still on its way.
 const TRANSPORT_FAILURE_STATUS = 0
 
-const NO_VALUE = '—'
-
 const ZERO_BYTES = '0B'
 
 /** Whether a request failed rather than completing — a producer-reported error,
@@ -28,6 +32,11 @@ export function requestFailed(req: NetworkRequest): boolean {
   return Boolean(req.error) || req.status === TRANSPORT_FAILURE_STATUS
 }
 
+/** The status bucket the list and the card colour a request by. */
+export function requestStatusKind(req: NetworkRequest) {
+  return statusKind(req.status, requestFailed(req), req.source)
+}
+
 /** Bytes a request transferred. A captured 0 (a 204, a HEAD, a body the
  *  collector could not read) is a measured size, where `formatBytes` renders it
  *  as the same dash it gives a size that was never captured at all. */
@@ -44,6 +53,17 @@ function statusCode(req: NetworkRequest): string {
   return requestFailed(req) ? FAILED_STATUS_LABEL : NO_VALUE
 }
 
+/** What a Resource Timing capture could not see, so an empty headers section
+ *  and a dash for the status read as the browser's limit rather than ours. */
+function captureLimit(req: NetworkRequest): string | undefined {
+  if (req.source !== NETWORK_CAPTURE_SOURCE.RESOURCE_TIMING) {
+    return undefined
+  }
+  return req.status
+    ? 'Resource Timing: this browser exposes no headers or body for the request'
+    : 'Resource Timing: this browser exposes no status, headers or body for the request'
+}
+
 function formatBody(body: string): string {
   try {
     return JSON.stringify(JSON.parse(body), null, 2)
@@ -95,18 +115,20 @@ function bodySection(title: string, body: string | undefined) {
 }
 
 function generalSection(req: NetworkRequest) {
-  const kind = statusKind(req.status, requestFailed(req))
+  const kind = requestStatusKind(req)
+  const limit = captureLimit(req)
   return html`
     <div class="detail-section">
       <div class="detail-title">General</div>
       <div class="kv-card">
-        ${kv('Request URL', req.url)} ${kv('Method', req.method)}
+        ${kv('Request URL', req.url)} ${kv('Method', req.method || NO_VALUE)}
         ${kv(
           'Status',
           html`${statusCode(req)} ${req.statusText || ''}`,
           `kind-${kind}`
         )}
         ${kv('Type', contentType(req))}
+        ${limit ? kv('Captured from', limit) : nothing}
         ${
           typeof req.time === 'number'
             ? kv('Time', formatTime(req.time))

packages/app/src/components/workbench/network/styles.ts

@@ -201,6 +201,9 @@ export const networkStyles = css`
   .kind-pending {
     color: var(--vscode-descriptionForeground);
   }
+  .kind-unknown {
+    color: var(--vscode-disabledForeground);
+  }
 
   .req-type,
   .req-dur,

packages/app/src/utils/network-constants.ts

@@ -94,12 +94,18 @@ export const HTTP_STATUS = {
  *  flight shows. */
 export const FAILED_STATUS_LABEL = 'ERR'
 
-/** Coarse status buckets used to colour the status dot/number in the list. */
+/** The list's and the detail card's glyph for a value the capture never had. */
+export const NO_VALUE = '—'
+
+/** Coarse status buckets used to colour the status dot/number in the list.
+ *  `unknown` is a request that completed on a source that exposes no status
+ *  (Resource Timing outside Chromium); `pending` is one still in flight. */
 export const STATUS_KIND = {
   OK: 'ok',
   REDIRECT: 'redirect',
   ERROR: 'error',
-  PENDING: 'pending'
+  PENDING: 'pending',
+  UNKNOWN: 'unknown'
 } as const
 
 export type StatusKind = (typeof STATUS_KIND)[keyof typeof STATUS_KIND]

packages/app/src/utils/network-helpers.ts

@@ -1,4 +1,9 @@
-import { isRequestType, type NetworkRequest } from '@wdio/devtools-shared'
+import {
+  isRequestType,
+  NETWORK_CAPTURE_SOURCE,
+  type NetworkCaptureSource,
+  type NetworkRequest
+} from '@wdio/devtools-shared'
 import {
   RESOURCE_TYPE_BY_REQUEST_TYPE,
   RESOURCE_TYPE_PATTERNS,
@@ -44,14 +49,21 @@ export function formatTime(ms?: number): string {
 /**
  * Bucket an HTTP status into a coarse {@link StatusKind}. The single source of
  * truth for status ranges — both the dot colour and the text colour derive
- * from it.
+ * from it. A request with no status is in flight, unless its source is one
+ * that never exposes a status: that request completed, and reads as unknown.
  */
-export function statusKind(status?: number, hasError?: boolean): StatusKind {
+export function statusKind(
+  status?: number,
+  hasError?: boolean,
+  source?: NetworkCaptureSource
+): StatusKind {
   if (hasError) {
     return STATUS_KIND.ERROR
   }
   if (typeof status !== 'number') {
-    return STATUS_KIND.PENDING
+    return source === NETWORK_CAPTURE_SOURCE.RESOURCE_TIMING
+      ? STATUS_KIND.UNKNOWN
+      : STATUS_KIND.PENDING
   }
   if (status >= HTTP_STATUS.SUCCESS_MIN && status < HTTP_STATUS.REDIRECT_MIN) {
     return STATUS_KIND.OK
@@ -72,7 +84,8 @@ const STATUS_TEXT_CLASS: Record<StatusKind, string> = {
   [STATUS_KIND.OK]: 'text-green-500',
   [STATUS_KIND.REDIRECT]: 'text-yellow-500',
   [STATUS_KIND.ERROR]: 'text-red-500',
-  [STATUS_KIND.PENDING]: 'text-gray-500'
+  [STATUS_KIND.PENDING]: 'text-gray-500',
+  [STATUS_KIND.UNKNOWN]: 'text-gray-500'
 }
 
 /**
@@ -110,8 +123,9 @@ export function getResourceType(request: NetworkRequest): ResourceType {
   if (sniffed) {
     return sniffed
   }
-  // Still unknown, but a body-carrying method is a data request either way.
-  if (request.method !== 'GET') {
+  // Still unknown, but a body-carrying method is a data request either way. An
+  // empty method is a source that names none, which is no evidence at all.
+  if (request.method && request.method !== 'GET') {
     return 'Fetch'
   }
   return OTHER_RESOURCE_TYPE