(CI checks for #160)

.github/workflows/test.yml

@@ -52,9 +52,11 @@ jobs:
         working-directory: extension-repo
         run: npm run test:coverage
 
-  e2e-smoke:
+  e2e:
     runs-on: ${{ matrix.os }}
     strategy:
+      # Don't cancel the other OS's e2e run when one fails — we want both reports.
+      fail-fast: false
       matrix:
         os: [ubuntu-latest, windows-latest]
     steps:
@@ -112,15 +114,16 @@ jobs:
         working-directory: paranext-core
         run: npm run build:dll
 
-      - name: Run e2e smoke tests (Linux)
+      - name: Run e2e tests (Linux)
         if: matrix.os == 'ubuntu-latest'
         working-directory: extension-repo
-        run: xvfb-run --auto-servernum --server-args="-screen 0 1280x960x24" npm run test:e2e:smoke
+        # Linux needs an xvfb display
+        run: xvfb-run --auto-servernum --server-args="-screen 0 1280x960x24" npm run test:e2e
 
-      - name: Run e2e smoke tests (Windows)
+      - name: Run e2e tests (Windows)
         if: matrix.os == 'windows-latest'
         working-directory: extension-repo
-        run: npm run test:e2e:smoke
+        run: npm run test:e2e
 
       - name: Upload Playwright test results
         uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
@@ -129,6 +132,7 @@ jobs:
           name: playwright-results-${{ matrix.os }}
           retention-days: 7
           path: |
+            extension-repo/e2e-tests/.cdp-app-startup.log
             extension-repo/e2e-tests/playwright-report/
             extension-repo/e2e-tests/test-results/
           if-no-files-found: warn

.gitignore

@@ -36,6 +36,8 @@ temp-build
 # AI files
 .claude
 
-# Playwright test output
+# Playwright output
+e2e-tests/.cdp-*
+e2e-tests/.dev-*
 e2e-tests/playwright-report
 e2e-tests/test-results

cspell.json

@@ -20,6 +20,7 @@
     "BBCCCVVV",
     "believ",
     "clickability",
+    "cmdk",
     "cullable",
     "deconflict",
     "deconfliction",
@@ -65,6 +66,7 @@
     "Stylesheet",
     "typedefs",
     "unanalyzed",
+    "unglossed",
     "unhover",
     "unobserves",
     "unphrased",

e2e-tests/README.md

@@ -1,6 +1,15 @@
 # e2e-tests
 
-End-to-end tests for the interlinearizer extension using Playwright + Electron. The suite launches a real Platform.Bible instance with the extension loaded via `--extensions` and verifies the extension starts up correctly. Currently contains one smoke test confirming the extension activates and registers its PAPI command.
+End-to-end tests for the interlinearizer extension using Playwright + Electron. The suite has two tiers:
+
+- **Smoke tests** (`tests/smoke/`, `app.fixture`) launch a fresh Platform.Bible instance with the extension loaded via `--extensions` and verify the extension starts up correctly.
+- **Feature tests** (`tests/features/`, `cdp.fixture`) connect over CDP to a running Platform.Bible instance and exercise interlinearizer UI flows (glossing, draft persistence, project modals).
+
+Run everything with `npm run test:e2e` (smoke tier then CDP tier). Each tier can be run alone with `npm run test:e2e:smoke` and `npm run test:e2e:cdp`.
+
+Both tiers are self-launching: the CDP tier's `globalSetup` launches its own Platform.Bible instance (with `--remote-debugging-port=9223`) in an isolated user-data dir and tears it down afterward, so `npm run test:e2e:cdp` needs no manual `npm run start:cdp` first. To iterate against a warm instance instead, run `npm run start:cdp` in one terminal, then run the CDP config directly with `npx playwright test --config e2e-tests/playwright-cdp.config.ts`: the setup detects the in-use CDP port, reuses that instance, and leaves it running.
+
+In CI (`.github/workflows/test.yml`, `e2e` job) the full suite runs on both Linux and Windows. Each tier writes its Playwright HTML report to its own subfolder (`playwright-report/smoke`, `playwright-report/cdp`) so a combined run keeps both.
 
 **Contents:**
 
@@ -17,3 +26,17 @@ These tests are adapted from `paranext-core`'s e2e suite with changes to support
 - **Extension launch helper** — `fixtures/helpers.ts` uses `launchElectronWithExtension()` instead of `launchElectronApp()`. It passes `--extensions <dist>` to the Electron process, resolves the Electron binary from paranext-core's `node_modules`, and polls `rpc.discover` for the extension's PAPI method to confirm activation.
 - **Window finding** — `fixtures/app.fixture.ts` manually polls `electronApp.windows()` by URL instead of calling `electronApp.firstWindow()`, because the extension injects content into an existing window rather than being the sole owner of the renderer.
 - **Renderer readiness** — `global-setup.ts` adds an HTTP GET probe after the TCP port check to wait for webpack compilation to finish, rather than assuming the port being open means the bundle is ready.
+
+## Writing feature tests
+
+Feature tests run with `npm run test:e2e:cdp`. That command launches a fresh, isolated instance, but the tests are also run against a shared, long-lived `npm run start:cdp` instance during local iteration (see above), so they must assume nothing about the instance's state and must leave nothing behind that could poison the next run. The protocol:
+
+- **Import from `cdp.fixture`, never `app.fixture`.** The CDP config already serializes execution (`workers: 1`), so tests never race each other on the shared instance.
+- **The instance is only ever used with the WEB project.** This is an operating assumption, not something tests verify: `ensureInterlinearizerOpenOnWeb()` trusts an existing Interlinearizer tab and only picks WEB when opening fresh. Don't point `start:cdp` at other projects.
+- **Mutating tests operate on the dedicated "E2E Test Project", never on a developer's own projects.** `ensureE2eProjectActive()` opens it (creating it on first use) at the start of each mutating test. Because the draft is the single per-source working buffer, replacing it could destroy unsaved developer work — so when the draft is dirty and the active project is _not_ the e2e project, the helper first saves the draft into a new `e2e-rescued-work-<timestamp>` project. Rescue projects are backups, not junk: delete them manually once recovered. Dirty state left while the e2e project is active is treated as leftover test data and discarded.
+- **Self-establish every precondition.** Each mutating test starts with `ensureInterlinearizerOpenOnWeb()` → `ensureE2eProjectActive()` → `navigateToScriptureRef()` (the scroll-group reference could be anywhere) → `wipeDraft()`.
+- **Reset at the start, tidy at the end.** The start-of-test sequence is what guarantees correctness — it self-heals whatever a failed run left behind. Mutating tests additionally end with `ensureE2eProjectActive(page, { rescueDirtyDraft: false })` to discard their own leftovers; this is a courtesy so the next run doesn't misread test junk as rescuable developer work, not something correctness depends on.
+- **Use unique per-run values** (e.g. `` `e2e-gloss-${Date.now()}` ``) for anything written into the draft, so a stale leftover can never satisfy an assertion.
+- **Drive only the visible UI.** No JSON-RPC/WebSocket calls to set up or assert state (the rpc.discover readiness polls in the shared helpers are the one sanctioned exception).
+- **Prefer existing accessible selectors** (roles, aria-labels like `Gloss for {word}`, ModalShell title ids) over adding new `data-testid`s to production code.
+- **Mutating tests must not overwrite or delete projects, and must not create any beyond what `ensureE2eProjectActive()` creates** (the e2e project itself, plus rescue projects). The current modal coverage is a read-only cancel tour; a create/delete lifecycle test needs its own self-healing cleanup (e.g. deleting leftover `e2e-*` projects at start) before it's safe on a shared instance.

e2e-tests/fixtures/cdp.fixture.ts

@@ -7,10 +7,12 @@
  *
  * - No app restart needed (no port 8876 conflict)
  * - Tests run against the same app instance used during development
- * - No teardown/shutdown of the app on completion
+ * - No teardown/shutdown of the app by the fixture on completion
  *
- * Prerequisite: Platform.Bible running with --remote-debugging-port=9223 and the interlinearizer
- * extension loaded.
+ * The app is provided by the CDP config's `globalSetup` (which self-launches one for `npm run
+ * test:e2e:cdp`) or by a developer's own `npm run start:cdp`; either way it must be running with
+ * `--remote-debugging-port=9223` and the interlinearizer extension loaded by the time this fixture
+ * connects.
  */
 import { test as base, chromium, Page } from '@playwright/test';