Speed up the default PHPUnit suite

[sirreal]

Summary

This PR speeds up the default PHPUnit suite by more than 70% without removing tests or reducing coverage.

Measured default-suite wall time:

Run	Wall time	Delta vs baseline
Baseline	275.951s	-
Final candidate	77.960s	-197.991s

That is a 71.75% wall-time reduction. The 70% target ceiling was 82.785s, so the final run is 4.825s under target.

JUnit body time also dropped from 154.880826s to 57.582364s, a 62.82% reduction. The final full run still executed 29,639 tests with 105,737 assertions.

Commit Structure

This branch is intentionally split into logical commits:

1. 6e7a4dc77e Tests: Optimize shared PHPUnit setup
2. fb964f6bc4 Tests: Run export tests in process
3. eadb4dab9a Tests: Avoid process isolation in sitemap 404 tests
4. 1dca902b0e Tests: Reduce isolated PHP processes
5. 9b32b13f63 Compat: Decode UTF-8 hot paths directly
6. 0f2273fdb1 HTML API: Use byte helpers in token maps
7. f6115ff315 Tests: Reduce media fixture cost
8. dbdc75f3a7 Tests: Add reduced REST route setup helpers
9. bbd6926c13 Tests: Reduce REST route setup in controller suites

What Changed

Shared PHPUnit Harness

The largest non-REST foundation is in WP_UnitTestCase_Base:

• Cache parsed PHPUnit expected-deprecation annotations per test method instead of reparsing them every setup.
• Keep common test-context hooks registered persistently and route them through the currently running test instance.
• Reuse stable core post type, taxonomy, and post status registration snapshots when no hook/filter state makes that unsafe.
• Avoid repeating SET autocommit = 0 once it is already disabled for the test connection.
• Apply low-cost test password hashing to class fixtures as well as per-test setup.
• Add guard tests for cached registration snapshots and filter-bypass behavior.

The auth tests were updated to preserve algorithm coverage while avoiding repeated expensive hash generation where a stable fixture hash is equivalent.

Export Tests

export_wp() declared WXR helper functions inside the function body. That forced the test class to run in separate PHP processes to avoid redeclaration and header issues.

This PR:

• Defines the WXR helpers only once per PHP process.
• Guards export download headers when output has already started.
• Removes class-level process isolation from Tests_Admin_ExportWp.

The export coverage remains the same; the tests now run in process.

Process Isolation Reductions

Several tests were isolated only because they used constants or historical global assumptions. This PR removes unnecessary isolation while keeping explicit coverage where constants are the behavior under test.

Changed areas:

• Sitemaps 404 tests now initialize the sitemap server explicitly and run in process.
• WP_Filesystem_Direct::mkdir() tests pass explicit chmod values where FS_CHMOD_DIR is not the behavior under test.
• A focused isolated mkdir test remains for omitted-chmod FS_CHMOD_DIR behavior.
• Selected WP_Upgrader::install_package() tests no longer define chmod constants and now assert the expected filesystem calls directly.
• Independent wp_unique_prefixed_id() datasets now run in process.
• The default-theme bundled-files test no longer needs process isolation.

UTF-8 and Token Map Hot Paths

The compat layer now avoids generic scanner work in common hot paths:

• _mb_ord() decodes the first UTF-8 code point directly after validating byte ranges.
• _wp_utf8_decode_fallback() directly handles valid UTF-8 spans and invalid subpart lengths while preserving the previous replacement behavior.
• Exhaustive Unicode tests still cover all valid code points and invalid edge cases, but reduce per-code-point assertion overhead where the whole corpus can be checked in aggregate.
• WP_Token_Map uses chr() and ord() for single-byte lengths instead of pack() and unpack() in token-map hot paths.

Media Test Fixture Cost

The media changes keep the same behavior under test but reduce fixture weight:

• URL crop failure uses an unreachable loopback URL instead of an external WordPress.org URL.
• Some image generation tests use smaller deterministic fixture images.
• Cross-origin isolation rewrite tests run in process, while one isolated header assertion remains for Document-Isolation-Policy emission.
• The production output-buffer callback avoids sending headers after output has already started, which matches normal PHP constraints and lets rewrite-only tests run in process.

REST Route Setup

REST setup was the largest remaining cost. Most controller-specific test classes were paying the cost of registering the entire core REST route catalog for every test, even when a class only dispatched one controller's routes.

This PR adds shared helpers to:

• Fire rest_api_init while temporarily suppressing create_initial_rest_routes.
• Preserve default REST filters, settings registration, and other non-initial-route hooks.
• Restore create_initial_rest_routes in a finally block.
• Register only the post type, taxonomy, or controller routes needed by a specific test class.
• Mirror core post-type route ordering for main, revisions, autosaves, and late route registration.

Individual REST test classes opt into minimal route sets only where focused validation proved the related routes they dispatch, link, embed, or assert. Examples include:

• Posts, pages, comments, attachments, post meta, term meta.
• Users and application passwords.
• Themes, plugins, global styles, templates, template revisions, and autosaves.
• Widgets, sidebars, widget types.
• Menus, menu items, menu locations, and navigation fallback.
• Search with related post/page and category/tag routes for embeddable self links.
• Block, pattern, and font REST controllers.
• Tests_REST_API setup paths that do not need the full route catalog.
• URL details tests that directly exercise parsing helpers or no-op abstract placeholders without registering routes.

REST-ish JUnit class time dropped from about 63.661s to 19.385s in the final full-suite artifact.

Coverage and Equivalence Notes

This PR does not remove tests and does not narrow assertions. The speedups come from:

• Avoiding separate PHP processes where the isolated global/constant behavior was not under test.
• Reusing stable test harness state when current hooks and filters make it safe.
• Replacing heavyweight fixtures with smaller deterministic fixtures that exercise the same code paths.
• Avoiding unrelated REST route registration in controller-specific tests.
• Keeping focused isolated tests where isolation is the behavior boundary, such as header emission and constant-backed defaults.

For REST controller tests, the default behavior remains full initial route creation unless a class explicitly opts into reduced setup. The reduced setup still runs rest_api_init; it only suppresses create_initial_rest_routes and then registers the routes relevant to that class.

Validation

Full Suite

Command:

WP_TESTS_SKIP_INSTALL=1 vendor/bin/phpunit --log-junit artifacts/phpunit-after-expanded-rest-route-reductions-junit.xml

Result:

• Wall time: 77.960s
• JUnit body time: 57.582364s
• Tests: 29,639
• Assertions: 105,737
• Warnings: 86
• Skipped: 78
• Errors: 1
• Failures: 4

The error/failures are known local/environmental issues observed during the speedup work:

1. Tests_WP_Customize_Manager::test_wp_customize_publish_changeset - local sidebar state warning/error.
2. Tests_Abilities_API_WpRegisterCoreAbilities::test_core_get_site_info_executes - generated REST fixture state where description is not the expected string.
3. Tests_DB::test_mysqli_flush_sync - missing local DB procedure privileges.
4. WP_Test_REST_Attachments_Controller::test_sideload_scaled_unique_filename - local canola-scaled-1.jpg filename collision.
5. Tests_Theme_ThemeDir::test_broken_themes - local extra broken html theme fixture.

The generated QUnit REST fixture drift produced by the full run was restored after validation.

Focused and Group Validation

The following focused/group validations were run during development:

• Changed and affected class batches for export, auth, sitemaps, XML-RPC auth side effects, image functions, UTF-8 compat, and cross-origin isolation.
• REST route-reduction focused batches for posts, comments, attachments, post meta, URL details, Tests_REST_API, widgets/sidebars, singleton REST controllers, term/global-style/pattern controllers, menu/navigation controllers, block controllers, pages, and font controllers.
• --group restapi with the route-reduction candidate. The route-specific coverage held; the run exposed the known local attachment collision and an oEmbed classic-provider issue that also reproduced when run alone, so it was not caused by route reduction.
• git diff --check is clean.

Adversarial Review

An adversarial expert review subagent reviewed the expanded REST route-reduction batch and explicitly approved it with no required fixes.

Key approval points:

• Shared helpers preserve did_action( 'rest_api_init' ), default REST filters, settings registration, and oEmbed route registration while suppressing only create_initial_rest_routes.
• Base REST controller testcase behavior remains full route creation by default.
• Reduced route setup is explicit per subclass and still uses a fresh Spy_REST_Server.
• Singleton controller reductions preserve the tested behavior.
• Related-route overlays for posts/pages/comments/attachments, widgets/sidebars, search, menus/menu-items/navigation fallback, fonts, application passwords, templates, revisions, and autosaves include the routes needed by dispatch and embedding behavior.
• Post-type and taxonomy helpers mirror core route registration ordering.
• Term-meta manual server setup reduction is safe.

Residual review note: controller-specific tests no longer incidentally cover coexistence with every unrelated core REST controller on each test method. Full route catalog coverage remains in schema/setup style tests, and these controller classes continue to assert their own behavioral surface.

Per-Commit Impact Report

After the initial PR write-up, I ran a second full-suite boundary sweep to estimate the marginal impact of each logical commit in branch order.

Method:

• Checked out the merge-base baseline and then each commit boundary in order.
• Ran WP_TESTS_SKIP_INSTALL=1 vendor/bin/phpunit --log-junit artifacts/impact/<boundary>.xml once per boundary.
• Restored the generated QUnit REST fixture after each run.
• Recorded both wall time and PHPUnit's accumulated JUnit testcase body time.
• All rows exited 2 with the same known local/environmental failures listed above, so the timing comparison is still useful.

The wall-time column is intentionally shown raw. It includes process startup, external waits, filesystem/cache effects, and local outliers. For smaller changes, the JUnit body-time delta is the more stable signal. Negative deltas below mean the suite got faster compared with the previous boundary.

Boundary	Commit	Wall time	Wall delta	JUnit body	Body delta	Assertions
Baseline	a1314bbebc	269.94s	-	148.48s	-	4,558,011
Shared PHPUnit setup	6e7a4dc77e	439.25s	+169.31s	112.33s	-36.15s	4,558,016
Export tests in process	fb964f6bc4	144.28s	-294.97s	108.28s	-4.06s	4,558,016
Sitemaps in process	eadb4dab9a	171.32s	+27.04s	107.73s	-0.54s	4,558,016
Reduce isolated processes	1dca902b0e	1087.49s	+916.17s	109.13s	+1.39s	4,558,018
UTF-8 direct decode	9b32b13f63	132.29s	-955.20s	106.51s	-2.62s	105,737
Token-map byte helpers	0f2273fdb1	129.99s	-2.30s	104.65s	-1.86s	105,737
Media fixture cost	f6115ff315	155.68s	+25.69s	130.03s	+25.38s	105,737
REST route helpers	dbdc75f3a7	211.82s	+56.14s	163.15s	+33.13s	105,737
REST route opt-ins	bbd6926c13	89.88s	-121.94s	65.97s	-97.18s	105,737

This second sweep measured the final branch at 89.88s wall time and 65.97s JUnit body time, versus 269.94s and 148.48s at baseline. That is 180.06s wall time saved (66.7%) and 82.51s JUnit body time saved (55.6%). The earlier final validation run in this PR body measured faster absolute numbers (77.960s wall, 57.582364s JUnit body), which is consistent with the amount of run-to-run wall-time variance visible in the boundary sweep.

Interpretation by Logical Change

1. REST route reduction is the strongest isolated win, but only as a bundle.

   • Helper-only commit dbdc75f3a7 costs time by itself because it adds the reduced setup machinery without using it broadly.
   • Opt-in commit bbd6926c13 pays that back heavily: -121.94s wall and -97.18s JUnit body against the helper-only boundary.
   • Evaluated as the intended two-commit REST bundle, from f6115ff315 to bbd6926c13, the suite saves 65.80s wall and 64.05s JUnit body.
   • Merge guidance: keep dbdc75f3a7 and bbd6926c13 together. Do not merge the helper commit alone for performance.

2. Shared PHPUnit setup is valuable, but should not ship alone.

   • 6e7a4dc77e cuts JUnit body time by 36.15s, which confirms the harness work reduces work inside tests.
   • Its wall time is +169.31s slower in isolation, likely because remaining process-isolated tests amplify the shared setup cost until the follow-up isolation changes land.
   • fb964f6bc4 then recovers the wall-time regression and leaves the suite much faster overall. Baseline to export-in-process is 269.94s -> 144.28s wall and 148.48s -> 108.28s JUnit body.
   • Merge guidance: treat the shared harness and export in-process commits as a performance bundle. Do not merge 6e7a4dc77e without at least fb964f6bc4.

3. UTF-8 direct decode has a modest full-suite time win and a major assertion-overhead win.

   • 9b32b13f63 reduces JUnit body time by 2.62s in this sweep.
   • The assertion count drops from about 4.56M to 105.7k because exhaustive Unicode coverage now validates large corpora in aggregate instead of emitting per-codepoint assertions.
   • Merge guidance: prioritize after behavioral review of the production UTF-8 fallback changes. It is not the largest wall-time lever, but it removes a huge amount of test assertion overhead and improves hot-path code.

4. Token-map byte helpers are a small, clean speedup.

   • 0f2273fdb1 saves 2.30s wall and 1.86s JUnit body in the full suite.
   • Merge guidance: lower priority than REST and harness work for suite speed, but it is a focused production hot-path cleanup.

5. Sitemaps process-isolation cleanup is small in full-suite terms.

   • eadb4dab9a saves only 0.54s JUnit body in this sweep, while wall time moves the wrong way by 27.04s.
   • That wall movement is within the noisy/outlier pattern seen in the sweep, so I would not use it as evidence that the commit is harmful.
   • Merge guidance: low speed priority on its own. Keep if the review value of removing unnecessary process isolation is worthwhile.

6. The broader process-isolation cleanup did not show a stable isolated speedup in this sweep.

   • 1dca902b0e produced a severe wall-time outlier (1087.49s) while JUnit body time stayed near the surrounding boundaries (109.13s).
   • I would not interpret the +916.17s wall delta as the commit's actual cost; it looks like one non-JUnit wait or local timeout in a single run.
   • Merge guidance: do not prioritize this commit based on measured suite speed. Review it for correctness and for reducing process-isolation dependency, or rerun targeted affected classes if it needs to stand alone.

7. Media fixture cost did not prove itself as a full-suite speedup in this run.

   • f6115ff315 was slower by 25.69s wall and 25.38s JUnit body in the boundary sweep.
   • The change may still be useful for removing external URL dependency, reducing specific fixture sizes, and allowing some media tests to run in process, but this measurement does not support treating it as a top suite-speed win.
   • Merge guidance: lowest priority if the only goal is default-suite wall time. Keep it only if the reliability/fixture-weight benefits justify it, or validate with targeted media-class timing before merging independently.

Prioritized Merge Guidance

1. Highest priority: merge the REST helper and opt-in commits together: dbdc75f3a7 + bbd6926c13. This is the clearest large JUnit-body reduction in the boundary sweep.
2. High priority: merge the shared harness with the export in-process follow-up: 6e7a4dc77e + fb964f6bc4. This pair gives a large net win, but the first commit is misleadingly bad on wall time when isolated.
3. Medium priority: merge UTF-8 direct decode and token-map byte helpers after normal production-code review: 9b32b13f63 + 0f2273fdb1. These are smaller full-suite wins but clean up hot paths and remove assertion overhead.
4. Lower priority: merge small process-isolation removals such as eadb4dab9a and 1dca902b0e for correctness/maintenance reasons, not because this sweep proves large suite-time wins.
5. Lowest speed priority: f6115ff315 media fixture cost. The full-suite boundary result was slower, so it should not be used as a headline performance justification without targeted follow-up data.

[github-actions]

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

Core Committers: Use this line as a base for the props when committing in SVN:

Props jonsurrell.

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.