test(daemon): seeded concurrency torture lane for session/lease/lock invariants

.github/workflows/ci.yml

@@ -303,6 +303,14 @@ jobs:
           pnpm clean:daemon
           pnpm test:integration:node
 
+      - name: Run seeded concurrency torture lane (fast PR sweep)
+        # #1416's nightly torture lane lives under test/integration/nightly/, out
+        # of the test:integration:node glob, so this is a *deliberate* fast PR
+        # sweep (TORTURE_RUNS default 128 seeds, ~sub-second) — not an accidental
+        # glob inclusion. The Concurrency Torture Nightly workflow sweeps a much
+        # larger seed range on schedule.
+        run: pnpm test:concurrency-torture
+
       - name: Run provider-backed integration tests
         run: pnpm test:integration:provider
 

.github/workflows/concurrency-torture-nightly.yml

@@ -0,0 +1,65 @@
+name: Concurrency Torture Nightly
+
+# Seeded concurrency torture lane for session/lease/lock invariants (#1416,
+# umbrella #1412 Track A). N concurrent clients drive randomized-but-seeded
+# interleavings of open/mutate/close/takeover/kill against the real SessionStore
+# + LeaseRegistry, with all concurrency routed through a deterministic scheduler
+# so a seed fully determines execution order. Deterministic and offline — no
+# devices, simulators, or wall-clock stress (those are out of scope for #1416).
+#
+# Scheduled + manual only: the PR gate already runs a fast default sweep via the
+# Node integration lane; this nightly sweeps a much larger seed range to keep
+# mining for ordering bugs. A failure prints the seed and the exact
+# `TORTURE_SEED=<n> pnpm test:concurrency-torture` replay command.
+
+on:
+  schedule:
+    - cron: '0 5 * * *'
+  workflow_dispatch:
+    inputs:
+      runs:
+        description: 'Number of seeded interleavings to sweep'
+        required: false
+        default: '5000'
+      seed-start:
+        description: 'First seed in the sweep'
+        required: false
+        default: '0'
+
+permissions:
+  contents: read
+
+concurrency:
+  group: ci-${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  torture:
+    name: Session/lease/lock torture sweep
+    runs-on: ubuntu-latest
+    timeout-minutes: 20
+    env:
+      TORTURE_RUNS: ${{ github.event.inputs.runs || '5000' }}
+      TORTURE_SEED_START: ${{ github.event.inputs.seed-start || '0' }}
+      # Standard scheduled-lane artifact envelope (#1430) built via the shared
+      # scripts/lib/lane-envelope.ts (commit, tool/configHash, seed range, result);
+      # the seed sweep rides in the typed data payload. Emitted by the test and
+      # uploaded below so the freshness watcher can detect a lane going dark.
+      TORTURE_ENVELOPE: ${{ github.workspace }}/torture-results/envelope.json
+    steps:
+      - name: Checkout
+        uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+
+      - name: Setup toolchain
+        uses: ./.github/actions/setup-node-pnpm
+
+      - name: Run concurrency torture sweep
+        run: pnpm test:concurrency-torture
+
+      - name: Upload torture lane envelope
+        if: always()
+        uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
+        with:
+          name: concurrency-torture-envelope
+          path: torture-results/
+          if-no-files-found: warn

docs/agents/testing.md

@@ -189,6 +189,69 @@ AGENT_DEVICE_WEB_E2E=1 pnpm test:smoke:web
 
 The test is skipped unless `AGENT_DEVICE_WEB_E2E=1` is set. The test runs `agent-device web setup` and `agent-device web doctor` with an isolated state directory before opening the fixture URL, so it verifies the public managed-backend setup path instead of relying on a global `agent-browser`. CI runs the lane on Node 24 because the managed backend requires Node >= 24. Failure artifacts, daemon state, and browser config are written under `test/artifacts/web/`.
 
+## Concurrency torture lane
+
+`test/integration/nightly/concurrency-torture.test.ts` (#1416, umbrella #1412 Track A) runs N concurrent
+clients through randomized-but-**seeded** interleavings of open/mutate/close/takeover/kill against
+the real `SessionStore` + `LeaseRegistry` (plus an in-memory device-claim model). After every run it
+asserts: no leaked leases or claims, no cross-session state bleed, every lock released after owner
+death, the session store stays consistent, and same-device critical sections never overlap (this
+pins the router's same-device open serialization under 100+ interleavings).
+
+A seed alone cannot reproduce Promise/event-loop interleavings, so **all** concurrency is routed
+through a deterministic scheduler (`nightly/concurrency-torture/deterministic-scheduler.ts`) — an
+instrumented dispatcher that is the sole source of ordering (which fiber steps next, and which waiter
+wins a contended lock). A seed therefore fully determines execution order.
+
+Each operation's lock plan is **not** hand-written: it is built exactly as the daemon builds it in
+`createRequestExecutionScope` — gate on the production decision `shouldLockSessionExecution(command)`
+(`src/daemon/daemon-command-registry.ts`), and only then resolve keys via the production router
+primitive `resolveRequestExecutionLockKeys` (`src/daemon/request-binding.ts`), driven with a fake
+device inventory through the production `withDeviceInventoryProvider` seam
+(`nightly/concurrency-torture/bindings.ts`). Only the mutex *grant* is modeled by the scheduler, because
+`withKeyedLock`'s native microtask hand-off cannot be reproduced from a seed. Consequently reverting
+*either* production decision — exempting a command from execution locking, or dropping the `device:`
+key — changes the derived plan and trips the overlap invariant, so the lane is genuinely coupled to
+production lock resolution, not a duplicate of it. **Real:**
+`SessionStore` and `LeaseRegistry`. **Modeled:** the advisory device claim (`InMemoryClaimRegistry`)
+and process "kill" — the production claim is a filesystem/OS lock and real process death, both out of
+scope for this scheduling lane and covered by their own unit tests. The full real-vs-modeled boundary
+is documented at the top of `nightly/concurrency-torture/harness.ts`.
+
+Because the seeded sweep *models* the mutex grant, a separate **real-scope guard**
+(`nightly/concurrency-torture/real-scope-serialization.ts`) drives concurrent same-device opens through the
+actual `createRequestExecutionScope().runLocked()` → `withRequestExecutionLocks` → `withKeyedLock`
+and asserts the critical sections never overlap. This is intentionally not seeded (it exercises real
+event-loop scheduling); its job is to fail if the production lock *application* path regresses, which
+the modeled sweep alone could not catch.
+
+```bash
+pnpm test:concurrency-torture                    # default sweep (TORTURE_RUNS=128 seeds from 0)
+TORTURE_SEED=1234 pnpm test:concurrency-torture  # replay ONE seed's exact interleaving (seed-replay flag)
+TORTURE_RUNS=5000 TORTURE_SEED_START=0 pnpm test:concurrency-torture   # widen the sweep
+```
+
+Replay is exact: a given seed reproduces the whole scheduler trace (`traceSignature`), the terminal
+invariant outcome, and the contention profile — equality on all three is asserted not just under
+`TORTURE_SEED` but for **every seed in the normal sweep** (each seed is re-run and compared), so
+non-determinism is caught on the ordinary CI/nightly path. The sweep also asserts real same-device
+lock *contention* occurred (two clients parked on one `device:` lock), and a dedicated forced
+two-client same-device test pins both clients to one device via `pinnedDevice` so they cannot land on
+different devices, driving that contention deterministically.
+
+Every failure prints the offending seed and the exact `TORTURE_SEED=<n> pnpm test:concurrency-torture`
+replay command. The lane lives under `test/integration/nightly/`, deliberately **out** of the
+`test:integration:node` glob so it is not an accidental PR-time run: the PR gate runs a fast default
+sweep via an explicit `Run seeded concurrency torture lane` step in the Integration job, and the
+`Concurrency Torture Nightly` workflow sweeps a much larger seed range on schedule. The nightly run
+emits the shared scheduled-lane envelope (`scripts/lib/lane-envelope.ts`, #1430 — commit,
+tool/`configHash` from the lane source hash, `seed` range, duration, result, with the seed sweep in
+the typed `data` payload) via `TORTURE_ENVELOPE=<path>`, uploaded as the `concurrency-torture-envelope`
+artifact. The
+envelope is written once, after **all** lane tests settle, and reports `fail` if any of them (sweep,
+replay self-check, or forced-contention guardrail) failed — a later-failing guardrail can never be
+published as a passing envelope. Optional knobs: `TORTURE_CLIENTS`, `TORTURE_OPS`.
+
 ## Speed rules (experiment-backed, 2026-07-04)
 
 Measured on the full unit suite (340 files, 3,210 tests, 48s wall at ~7x parallelism):

package.json

@@ -161,6 +161,7 @@
     "test:smoke": "node --test test/integration/smoke-*.test.ts",
     "test:integration:node": "node --test test/integration/*.test.ts",
     "test:integration": "pnpm test:integration:node && pnpm test:integration:provider",
+    "test:concurrency-torture": "node --test test/integration/nightly/concurrency-torture.test.ts",
     "test:replay:ios": "node --experimental-strip-types src/bin.ts test test/integration/replays/ios/simulator",
     "test:replay:ios-device": "node --experimental-strip-types src/bin.ts test test/integration/replays/ios/device",
     "test:replay:android": "node --experimental-strip-types src/bin.ts test test/integration/replays/android",