Repository navigation
Build/Test Tools: Measure wp-admin in the browser and check performance budgets - #15957
Open
rene-hermenau wants to merge 4 commits into
Open
rene-hermenau wants to merge 4 commits into
rene-hermenau wants to merge 4 commits into
Conversation
… performance tests. The admin performance spec only recorded Server-Timing and time to first byte for the Dashboard, so changes that add scripts or styles to every admin screen were not visible in the performance results. Enqueueing the Command Palette on all admin screens in [61022] added about 40 scripts and 5 MB of JavaScript to every classic admin screen without any change in the reported numbers. The spec now also records `domContentLoaded` (from `getLoadingDurations()`) and the number and size of the scripts and stylesheets in the page markup, as `scriptCount`, `scriptBytes`, `stylesheetCount` and `stylesheetBytes`, and it measures the General Settings screen (`options-general.php`) next to the Dashboard. The new suites are added to the results sent to codevitals, and `formatValue()` shows counts as plain numbers and byte sizes in KiB. The sizes use the Resource Timing `encodedBodySize`, not `transferSize`. The `/?clear_cache` request before each measurement loads the front end as a logged-in user, so the toolbar assets are already in the HTTP cache when the admin screen loads, and their `transferSize` is 0, or only the header size when they are revalidated. Which of the two happens depends on the files' Last-Modified date, so the numbers could change between builds without any code change. `encodedBodySize` is the same for cached and fetched responses. Each suite now starts from an empty results object, so the de_DE suites no longer include the Server-Timing values collected for en_US. See #66642, #63845.
The performance workflow measures pull requests and their target commit, but it only shows the numbers in a collapsed job summary table. Nothing points out a regression and nothing fails, so a change that doubles the load time of every admin screen passes all checks. `check-regressions.js` compares the medians of this change with the target commit on pull requests, and with the last release on trunk pushes, metric by metric. A metric regresses when it grows by more than an absolute and a relative budget; timing metrics must also grow by more than three times the combined robust spread (MAD) of both samples, so CI noise does not trip the check. A metric whose target value is 0 can regress as well. Asset counts and sizes (`scriptBytes`, `stylesheetBytes`), database queries and memory usage are deterministic for a build. Exceeding their budget fails the Compare job on pull requests. Timings only warn. The comparison with the last release is reported but never fails, which gives trunk commits a release-to-release check as well. A crash of the check itself does not fail trunk. The `Accepted Performance Regression` label marks a regression as reviewed, so the job reports it without failing. The label is read from the pull request event, so it takes effect with the next push; re-running a job keeps the labels of the original event. The results are written at the top of the job summary and as workflow annotations. See #66642, #63845.
|
The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. Core Committers: Use this line as a base for the props when committing in SVN: To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook. Props Bot commandsYou can trigger Props Bot actions by commenting on this PR:
|
… report. With the `Accepted Performance Regression` label, the job summary still showed the red cross and `error` in the severity column. Only one sentence below the table told it apart from a failing check, so reviewers could not see at a glance that the regression had been accepted. An accepted report now uses the warning sign, says in the same line that the label accepts the regression, and lists error-severity metrics as `error (accepted)`. The separate sentence below the table is gone. Reports without the label and the exit codes are unchanged. See #66685.
The check failed a pull request as soon as a screen loaded more scripts or bytes than a small budget allowed. Code grows with every feature, so growth on its own is no reason to block a change. What should not happen is a screen getting slower. A timing regression (DOMContentLoaded, LCP, TTFB, server time) now fails the check when the same screen also loads more scripts or stylesheets, runs more database queries or uses more memory. The growth confirms that the slowdown comes from the change and not from runner noise. Growth without a slowdown is still listed in the summary and as a warning annotation, so reviewers see right away when a change adds, for example, 100 KiB of JavaScript and can judge it. A slowdown without any growth is reported as a warning as well, since timings on shared CI runners vary. On the Command Palette change, the check reports 8 errors for the slower screens and 16 warnings for the added scripts and stylesheets. See #66685.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR changes only the performance tests and their workflow. No file under
src/is touched. It lets the tests see what a change costs in the browser on wp-admin, and adds a budget check to the Compare job that fails a pull request when it makes wp-admin measurably slower.What the new check reports for the Command Palette change from [61022]. Rendered locally from the output of
check-regressions.js.tests/performance/specs/admin.test.js,tests/performance/utils.js,tests/performance/log-results.jstests/performance/check-regressions.js(new),.github/workflows/performance.yml,.github/workflows/reusable-performance-report-v2.ymlThe metrics and the budget check can land separately if the budget check needs more discussion.
Why
Today the admin performance spec records only Server-Timing and time to first byte on the Dashboard, and the Compare job never fails. That is how [61022] (#63845) added about 40 scripts and 5.4 MiB of JavaScript to every classic admin screen without a visible change in the results. The fix for that regression is in #66642; this PR is about catching the next one.
Video: what that regression feels like on Settings › General (background only, not part of this PR)
Trunk on the left, the same build without the eager palette enqueue on the right. Cold cache, Chromium with 4× CPU throttling, 20 Mbit/s and 40 ms latency, real time. From 450 ms on, the Help button is clicked every 150 ms (red circle) until the Help panel opens: after about 1,740 ms and 6 clicks on trunk, after about 880 ms and 2 clicks without the palette.
4-settings-general-side-by-side.mp4
1. Browser metrics on admin screens
admin.test.jsmeasures General Settings (options-general.php) next to the Dashboard, as the new suitesadmin-general-settingsandadmin-general-settings-l10n. The Dashboard keeps its suite name, so its history on codevitals.run continues.domContentLoadedfrommetrics.getLoadingDurations(), andscriptCount,scriptBytes,stylesheetCount,stylesheetBytesfor the scripts and stylesheets in the page markup. Sizes use Resource Timing'sencodedBodySize;transferSizewould count the toolbar assets as 0 bytes, because the/?clear_cacherequest before each measurement has already cached them.formatValue()shows counts as plain numbers and sizes in KiB.2. Budget check
The goal is that wp-admin does not get slower. Code grows with every feature, so growth on its own does not block a change, but reviewers should see it.
tests/performance/check-regressions.js, run in the Compare job, compares the medians of this change with the target commit and with the last release.Accepted Performance Regressionlabel marks a regression as reviewed, so the job reports it without failing.Results
Local run on trunk, with and without
remove_action( 'admin_enqueue_scripts', 'wp_enqueue_command_palette_assets' ). CI-like environment (build, debug constants off, cron and external HTTP off),TEST_RUNS=10, 2 repetitions, medians.Accepted Performance RegressionOn the palette change, the 8 errors are
domContentLoadedon all four suites, plustimeToFirstByteandwpTotalon both de_DE suites (TTFB +16 % on the Dashboard, +23 % on General Settings). The 16 warnings are the added scripts and stylesheets. The two synthetic cases take the same-build data and add 100 KiB toscriptBytesor 50 % todomContentLoaded.The other cases
A change that adds 100 KiB of JavaScript but is not slower passes, and the growth is listed for the reviewer (synthetic case):
The same build measured twice passes, so runner noise does not trip it:
With the
Accepted Performance Regressionlabel, the palette change is reported but does not fail:Notes for reviewers
Accepted Performance Regressionlabel has to be created in the repository. It only takes effect when it is set before the Compare job starts.workflow_runworkflow; left for a follow-up.Trac ticket: https://core.trac.wordpress.org/ticket/66685
Use of AI Tools
AI assistance: Yes
Tool(s): Claude Code
Model(s): Claude Opus 5.5
Used for: Implementation, tests, measurements and the ticket text, under my direction; I reviewed the change and take responsibility for it.
This Pull Request is for code review only. Please keep all other discussion in the Trac ticket. Do not merge this Pull Request. See GitHub Pull Requests for Code Review in the Core Handbook for more details.