Studio: Start a site from its path in cli.json instead of listing every site first by chubes4 · Pull Request #5351 · Automattic/studio

chubes4 · 2026-10-23T18:45:31Z

Related issues

Follows the discussion on Studio: Keep the site list from CLI events instead of asking the CLI each time #5317: get the site-start gain without a new cache layer.

How AI was used in this PR

Claude (via the Pi coding agent in a Roadie session) traced the CLI processes behind a site start, wrote the change, and ran the local checks below. I reviewed the code.

Proposed Changes

startServer ran site list (a full CLI process, about 0.35 s) to look up the site before running site start. The only thing a start needs is the site's path, and readSite() reads it from cli.json without a CLI process. stopServer already looks the site up that way.

startServer reads the site from cli.json with readSite().
A site that cli.json doesn't have now fails the start with "Site not found" instead of doing nothing, which used to end in a false "Site started" toast. The same applies when cli.json can't be parsed; before, site list failed on that.
It runs getSite() for the CLI's full site details only after a failed start. Those details feed PHP error recovery (port, PHP version) and the Sentry context.

There is no new state or cache: the start reads the same cli.json that site list reads.

Testing Instructions

Locally (macOS arm64, packaged build), I timed every CLI process during the tools/metrics site-startup test. In each of the 5 restarts, site start now begins about 0.1 s after the click, with no site list ahead of it. On trunk, a site list ran first.
The CI Performance Test Results comment gives the siteStartup number.

Pre-merge Checklist

Have you checked for TypeScript, React or other console errors?

…ry site first

…hange

wpmobilebot · 2026-10-23T19:18:20Z

📊 Performance Test Results

Comparing 5aacf8f vs trunk

app-size

Metric	trunk	`5aacf8f`	Diff	Change
App Size (Mac)	954.29 MB	954.29 MB	+0.00 MB	⚪ 0.0%

site-editor

Metric	trunk	`5aacf8f`	Diff	Change
load	1177 ms	1221 ms	+44 ms	⚪ 0.0%

site-startup

Metric	trunk	`5aacf8f`	Diff	Change
siteCreation	4730 ms	4903 ms	+173 ms	🔴 3.7%
siteStartup	2237 ms	2011 ms	226 ms	🟢 -10.1%

Results are median values from multiple test runs.

Legend: 🟢 Improvement (faster) | 🔴 Regression (slower) | ⚪ No change (<50ms diff)

chubes4 · 2026-10-23T19:46:50Z

Closing. This removes the site list that ran before every site start, about 0.3 s, but the siteStartup benchmark can't show a clear gain from that:

CI: 2873 → 2843 ms.
Locally, interleaved, 25 restarts per side on the same trunk base: median 2837 → 2335 ms, mean 3120 → 2964 ms. The difference isn't significant (permutation test p ≈ 0.15 for the median, 0.29 for the mean).
The benchmark's restart times land in steps of about 0.5 s (1.82, 2.33, 2.83 s…), so a 0.3 s saving only shows when it moves a restart into a lower step.

Most of the wait between a start and the site showing as running is the renderer's site list refetches after site start finishes.

…5505) ## Related issues - Explains why #5351 (removing a ~0.3 s `site list` before each start) didn't show in `siteStartup`, and why `siteStartup` is bimodal on trunk (about 2.87 s or 3.39 s). ## How AI was used in this PR Claude (via the Pi coding agent in a Roadie session) traced a site start through a packaged build, found the poll interval in Playwright's source, wrote the change, and ran the local measurements below. I reviewed the design and the code. ## Proposed Changes The site-startup metrics stopped the clock when `expect( statusButton ).toHaveAttribute( 'data-state', 'running' )` passed. Playwright retries that assertion after 20, 50, 100 and 100 ms, then every 500 ms (`retryWithProgressAndBackoff` in Playwright 1.64.1). So each time landed on the next 500 ms poll after the site showed as running. Restart times came out in steps of about 0.5 s, and a change smaller than that only shows when it moves a restart down a step. - `recordRunningTimes( page )` puts a `MutationObserver` in the page. It records the time each element's `data-state` most recently became `running`. - `siteCreation` and `siteStartup` use that recorded time for the status button. The assertion still checks that the site is running; it just no longer sets the time. - Both the PR build and the trunk build run the PR's tests, so PR comparisons stay like for like. The trunk history will drop once when this lands, by the average poll delay (about 0.25 s). ## Testing Instructions Locally, on macOS arm64 with a packaged build, I logged both times for 15 restarts of `tools/metrics/tests/site-startup.test.ts`: - The old times sat on the poll steps: 13 of 15 were 2,335–2,350 ms, and the others were 2,837 and 3,361 ms. - The new times spread across 2,048–2,881 ms, each 54–480 ms below the old one. - In a second run with the app's CLI processes timestamped, the new time was 240–340 ms after the CLI reported the site running in all 8 restarts. That is one `site list` refetch plus a render. - `npx eslint tools/metrics` passes. To try it, run `npm run package`, then `npm run test:metrics`. ## Pre-merge Checklist - [x] Have you checked for TypeScript, React or other console errors?

chubes4 · 2026-10-28T00:18:30Z

Reopening. The siteStartup benchmark couldn't show this change when I closed it, and #5505 fixed the reason.

The benchmark stopped the clock when Playwright's toHaveAttribute( 'data-state', 'running' ) assertion passed. After its first few tries, that assertion retries every 500 ms, so restart times landed in steps of about 0.5 s. A 0.3 s saving only showed when it moved a restart down a step. #5505 records the moment the status button turns to running in the page instead.

With trunk merged (5aacf8f73), the Performance Test Results now show the saving: siteStartup 2,237 → 2,011 ms (−10.1%). siteCreation (+3.7%) doesn't run startServer, and it is one sample per round.

Claude (via the Pi coding agent in a Roadie session) wrote this comment from the CI results and a local trace of a site start. I reviewed it.

…5506) ## Related issues - Follows #5351 and #5505, which trimmed and measured the start path. ## How AI was used Claude (via the Pi coding agent in a Roadie session) traced the start path, wrote the change and its tests, checked the tests by reverting each piece, and counted the `site list` calls in packaged builds. I reviewed the diff and the evidence. ## Proposed Changes Every site event the renderer received made it invalidate the site list. Each refetch spawns a CLI `site list`, which takes about 300 ms. A start or stop sends several events and also awaited one more refetch of its own. Yet each `site-updated` and `site-operations-changed` event already carries the site's full record. - **Renderer (`use-sites.ts`):** writes the event's site into the TanStack Query cache with `setQueryData`, as `useUpdateSitesSortOrder` already does. It keeps the two fields only Studio stores (`sortOrder`, `autoStart`). `satisfies` ties those to `SiteMetadata`, so a new metadata field won't compile until it's carried over. A `site-operations-changed` event keeps the cached `running`, because it doesn't know whether the site runs. - It still refetches for creates, deletes, sites the list doesn't have yet, and events with no site. - It also still refetches when it joins an in-flight fetch, as before. - Start and stop refetch only if the list doesn't already show the result. - **Main process:** a site serving the PHP error page is shown at a URL only `getSites` knows. Its events now go out without the CLI's details, so the renderer refetches for them as before. - **CLI:** `site list` and the events now build the site from one `getSiteDetails`. Until now the events left out `emailPath`, so an applied event would have dropped it. Applying events directly, rather than refetching on each one, exposed two gaps that the refetches had been covering up: - **Auto-start** waited for `isFetchedAfterMount`. That flag also turns true when an event is applied on top of the list persisted from the last launch, which predates the `autoStart` flags the quit set. Auto-start now reads the list itself with `fetchQuery`, which joins the launch fetch already in flight. - **`_events`** read every site's status before subscribing to the process manager. A site that finished starting in between never got an event. It now subscribes first, so the last event for a site reflects its current state. No new cache layer: TanStack Query stays the only one. In packaged builds, I counted `site list` calls through a probe in the CLI entry. A run of the startup metric (one creation plus five starts and stops) made 33 calls on trunk and 10–16 with this change, across three runs each. Calls only remain where a fallback refetch is needed, for example when an operation's release lands after the start has resolved. Local timings were unusable because the machine's load average was 20–30. The Performance Metrics job is the timing check. ## Testing Instructions - `npx vitest run apps/ui/src/data/queries/use-sites.test.tsx apps/studio/src/modules/cli/lib/tests/cli-events-subscriber.test.ts apps/cli/commands/site/tests/list.test.ts` - The new start/stop test fails on trunk. - The auto-start test fails with the old `isFetchedAfterMount` gate. - It also fails if any of these is reverted: the cached `running` for operation events, the kept metadata, or the skipped refetches. - Start and stop a site in the app. Its status, URL and settings update as before. - Break a site's `wp-config.php` and start it. It shows the PHP error page at its URL, as before. ## Pre-merge Checklist - [x] Have you checked for TypeScript, React or other console errors?

Studio: Start a site from its path in cli.json instead of listing eve…

e980b11

…ry site first

github-actions Bot assigned chubes4 Oct 23, 2026

chubes4 added 2 commits October 23, 2026 14:52

Drop the startServer unit tests: the startup benchmark measures the c…

02023d5

…hange

Fail a start for a site cli.json doesn't have instead of doing nothing

cbe0e8b

chubes4 closed this Oct 23, 2026

chubes4 deleted the app/start-without-lookup branch October 23, 2026 19:46

chubes4 mentioned this pull request Oct 27, 2026

Metrics: Time site starts from the moment the site shows as running #5505

Merged

1 task done

chubes4 restored the app/start-without-lookup branch October 27, 2026 23:56

chubes4 reopened this Oct 27, 2026

Merge branch 'trunk' into app/start-without-lookup

5aacf8f

chubes4 merged commit 6e9103c into trunk Oct 28, 2026
12 checks passed

chubes4 deleted the app/start-without-lookup branch October 28, 2026 00:31

chubes4 mentioned this pull request Oct 28, 2026

Studio: Apply site events to the site list instead of refetching it #5506

Merged

1 task done

Navigation Menu

Repository navigation

Studio: Start a site from its path in cli.json instead of listing every site first - #5351

Studio: Start a site from its path in cli.json instead of listing every site first#5351
chubes4 merged 4 commits into
trunkfrom
app/start-without-lookup

chubes4 commented Oct 23, 2026 •

edited

Loading

Uh oh!

wpmobilebot commented Oct 23, 2026 •

edited

Loading

Uh oh!

chubes4 commented Oct 23, 2026 •

edited

Loading

Uh oh!

chubes4 commented Oct 28, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

chubes4 commented Oct 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Related issues

How AI was used in this PR

Proposed Changes

Testing Instructions

Pre-merge Checklist

Uh oh!

wpmobilebot commented Oct 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📊 Performance Test Results

app-size

site-editor

site-startup

Uh oh!

chubes4 commented Oct 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chubes4 commented Oct 28, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

chubes4 commented Oct 23, 2026 •

edited

Loading

wpmobilebot commented Oct 23, 2026 •

edited

Loading

chubes4 commented Oct 23, 2026 •

edited

Loading