From 9bba212858d92f56498d45c401aaaf798e1befba Mon Sep 17 00:00:00 2001 From: Adrian Moldovan <3854374+adimoldovan@users.noreply.github.com> Date: Thu, 12 Nov 2026 16:48:33 +0200 Subject: [PATCH 1/2] Build/Test Tools: Run tests in one E2E spec file in parallel. Turn on `fullyParallel` for the `parallel` project, so Playwright can run tests from the same file at the same time. Scope the classic menus to each test. Lock the tests that write the theme's menu locations or the privacy policy page setting, as this data cannot be scoped. See #66392. --- tests/e2e/README.md | 2 +- tests/e2e/playwright.config.js | 2 +- tests/e2e/specs/classic-menus.test.js | 75 ++++++++++++++++++++------- tests/e2e/specs/privacy.test.js | 3 +- tests/e2e/utils/locks.js | 2 + 5 files changed, 63 insertions(+), 21 deletions(-) diff --git a/tests/e2e/README.md b/tests/e2e/README.md index b9124d088d727..133f1b69a8dae 100644 --- a/tests/e2e/README.md +++ b/tests/e2e/README.md @@ -1 +1 @@ -# E2E Tests End-To-End (E2E) tests for WordPress. ## Running the tests The e2e tests require a production-like environment to run. By default, they will assume an environment is available at `http://localhost:8889`, with username `admin` and password `password`. If you don't already have an environment ready, you can set one up by following [these instructions](https://github.com/WordPress/wordpress-develop/blob/master/README.md). Then you can launch the tests by running: ``` npm run test:e2e ``` which will run the test suite using a headless browser. If your environment has a different url, username or password to the default, you can provide the base URL, username and password like this: ``` WP_BASE_URL=http://mycustomurl WP_USERNAME=username WP_PASSWORD=password npm run test:e2e ``` **DO NOT run these tests in an actual production environment, as they will delete all your content.** For debugging purposes, you might want to follow the test visually. You can do so by running the tests in an interactive mode: ``` npm run test:e2e -- --ui ``` [UI Mode](https://playwright.dev/docs/test-ui-mode) let's you explore, run and debug tests with a time travel experience complete with watch mode. All test files are loaded into the testing sidebar where you can expand each file and describe block to individually run, view, watch and debug each test. You can also run a single test file separately: ``` npm run test:e2e tests/e2e/specs/hello.test.js ``` ## Projects The suite has three Playwright projects, defined in `playwright.config.js`: * `all`: every spec, one at a time on one worker. `npm run test:e2e` runs this project. * `parallel`: specs that can share the site with other tests. Spec files run at the same time on several workers, one for every two CPU cores. Tests inside one file still run in order. Run it with `npm run test:e2e -- --project=parallel`. * `serial`: specs that break the site for other tests. They run one at a time on one worker. Run it with `npm run test:e2e -- --project=serial`. CI runs `parallel` and `serial` in separate jobs. New specs go in `parallel` by default. Add a spec to `SERIAL_SPECS` if it makes the site unusable for other tests. If it changes content or settings that other tests use, for example with `requestUtils.deleteAllPosts()`, follow [Parallel safety](#parallel-safety). Before you open a pull request with a new spec, run its project a few times with `--shuffle`. Playwright then runs the spec files in a random order, so a spec that needs another spec to run first, or that breaks a spec that runs after it, fails: ``` npm run test:e2e -- --project=parallel --shuffle ``` Playwright prints the seed of each run, for example `shuffle seed 317753389`. To run the same order again, give the seed: `--shuffle 317753389`. ## Parallel safety Specs in `parallel` share one site. A test that changes shared data can break a test in another file that runs at the same time. To make a spec safe, use the first of these that works: 1. Scope the data. Let the test create its own data, for example a post with a unique title, and read only that data. 2. If the test must change shared data, such as a site setting, give it a lock. Do this only if every spec that writes the same data holds the same lock. 3. If you cannot lock every spec that writes the data, add the spec to `SERIAL_SPECS`. A lock is a name. Playwright never runs two tests that hold the same lock name at the same time. A lock only schedules tests. It does not isolate data: a test without the lock can still change the data while a locked test runs. The lock names are in `utils/locks.js`. Import the constant, so that all specs use the same name. Playwright gives no error for a lock name that no other test uses, or for an import name that is not in `utils/locks.js`: ``` import { GENERAL_SETTINGS } from '../utils/locks'; test.describe( 'Settings -> General', { lock: GENERAL_SETTINGS }, () => {} ); ``` A form save writes every field on the page, not only the field that the test changed. A spec that saves the General Settings form (`options-general.php`) must hold `GENERAL_SETTINGS`. A spec in `parallel` that changes a General Settings option in a different way, for example with `siteSettings` or `siteOptions`, must also hold it. ## Logged-out pages Most specs run logged in as the admin user. There are two ways to run a test logged out: * To make the `page` of a whole `describe` group logged out, add `test.use( { storageState: { cookies: [], origins: [] } } )` to the group. See `specs/dashboard.test.js`. * To use a logged-out page next to the logged-in `page` in one test, import `test` and `expect` from `../fixtures` and use the `loggedOutPage` fixture. Its browser context has no cookies, and it closes when the test ends. Unlike `page`, it resolves a URL that starts with `/` against the host only, not the full `WP_BASE_URL`, so it does not work with a site in a subdirectory. See `specs/cache-control-headers-directives.test.js`. ## Fixtures To create posts, pages or users, to upload attachments, or to change a setting, an option, or a must-use plugin, import `test` and `expect` from `../fixtures`, and use one of the fixtures below. Each fixture cleans up when the test ends, also if the test fails. If the cleanup fails, the test fails. * `siteSettings`: reads and changes the settings of `/wp/v2/settings` through REST, with `get()`, `remember( keys )` and `set( patch )`. It restores only the keys passed to `remember` or `set`. See `specs/privacy.test.js`. * `siteOptions`: reads and changes options through WP-CLI, with `get( names )`, `remember( names )` and `set( values )`. It reads and restores the stored `gmt_offset`, not the value calculated for a named timezone. It restores only the keys passed to `remember` or `set`. See `specs/general-settings-invalid-timezone.test.js`. * `muPlugin`: writes a must-use plugin with `write( fileName, php )`, and deletes the file when the test ends. Call `write` before the test visits a page. See `specs/fatal-error-handler.test.js`. * `postFactory`: creates posts with `create( data )` and pages with `createPage( data )`, and force-deletes them when the test ends. `uniqueTitle( prefix )` adds a UUID to the prefix. See `specs/edit-posts.test.js`. * `attachments`: uploads a file with `upload( absolutePath )`, records an attachment that the browser uploaded with `track( id )`, and deletes them when the test ends. See `specs/media-new-client-side-upload.test.js`. * `userFactory`: creates users with `create( { roles, ...data } )`, which returns `{ user, username, password }`. For a user that the test creates through a form, call `adopt( username )` before the form is submitted. When the test ends, it deletes each user and the user's auto-drafts. To log in as the user, import `signIn( page, username, password )` from `../fixtures/user`. See `specs/users.test.js`. If the test changes a setting or an option through a form, first call `remember` with an array of the keys that the form changes. The fixture then reads the current value, without a write, and restores it when the test ends. `remember` returns the current values. A fixture goes in `fixtures/` only when 2 or more spec files use it. A fixture that only one spec uses stays in that spec. ## Documentation * Block Editor Handbook end to end testing overview: https://developer.wordpress.org/block-editor/contributors/code/testing-overview/#end-to-end-testing * Gutenberg e2e-test-utils-playwright package API docs: https://github.com/WordPress/gutenberg/tree/trunk/packages/e2e-test-utils-playwright * Playwright API docs: https://playwright.dev/docs (the version we are using is indicated in the `@wordpress/scripts` package: https://github.com/WordPress/gutenberg/blob/trunk/packages/scripts/package.json) \ No newline at end of file +# E2E Tests End-To-End (E2E) tests for WordPress. ## Running the tests The e2e tests require a production-like environment to run. By default, they will assume an environment is available at `http://localhost:8889`, with username `admin` and password `password`. If you don't already have an environment ready, you can set one up by following [these instructions](https://github.com/WordPress/wordpress-develop/blob/master/README.md). Then you can launch the tests by running: ``` npm run test:e2e ``` which will run the test suite using a headless browser. If your environment has a different url, username or password to the default, you can provide the base URL, username and password like this: ``` WP_BASE_URL=http://mycustomurl WP_USERNAME=username WP_PASSWORD=password npm run test:e2e ``` **DO NOT run these tests in an actual production environment, as they will delete all your content.** For debugging purposes, you might want to follow the test visually. You can do so by running the tests in an interactive mode: ``` npm run test:e2e -- --ui ``` [UI Mode](https://playwright.dev/docs/test-ui-mode) let's you explore, run and debug tests with a time travel experience complete with watch mode. All test files are loaded into the testing sidebar where you can expand each file and describe block to individually run, view, watch and debug each test. You can also run a single test file separately: ``` npm run test:e2e tests/e2e/specs/hello.test.js ``` ## Projects The suite has three Playwright projects, defined in `playwright.config.js`: * `all`: every spec, one at a time on one worker. `npm run test:e2e` runs this project. * `parallel`: specs that can share the site with other tests. Spec files run at the same time on several workers, one for every two CPU cores. Tests inside one file still run in order. Run it with `npm run test:e2e -- --project=parallel`. * `serial`: specs that break the site for other tests. They run one at a time on one worker. Run it with `npm run test:e2e -- --project=serial`. CI runs `parallel` and `serial` in separate jobs. New specs go in `parallel` by default. Add a spec to `SERIAL_SPECS` if it makes the site unusable for other tests. If it changes content or settings that other tests use, for example with `requestUtils.deleteAllPosts()`, follow [Parallel safety](#parallel-safety). Before you open a pull request with a new spec, run its project a few times with `--shuffle`. Playwright then runs the spec files in a random order, so a spec that needs another spec to run first, or that breaks a spec that runs after it, fails: ``` npm run test:e2e -- --project=parallel --shuffle ``` Playwright prints the seed of each run, for example `shuffle seed 317753389`. To run the same order again, give the seed: `--shuffle 317753389`. ## Parallel safety Specs in `parallel` share one site. A test that changes shared data can break another test that runs at the same time. The `parallel` project sets `fullyParallel: true`, so this can be a test in another file or a test in the same file. To make a spec safe, use the first of these that works: 1. Scope the data. Let the test create its own data, for example a post with a unique title, and read only that data. 2. If the test must change shared data, such as a site setting, give it a lock. Do this only if every spec that writes the same data holds the same lock. 3. If you cannot lock every spec that writes the data, add the spec to `SERIAL_SPECS`. A lock is a name. Playwright never runs two tests that hold the same lock name at the same time. A lock on a `describe` group applies to each test in the group, not to the group as a whole. A lock only schedules tests. It does not isolate data: a test without the lock can still change the data while a locked test runs. The lock names are in `utils/locks.js`. Import the constant, so that all specs use the same name. Playwright gives no error for a lock name that no other test uses, or for an import name that is not in `utils/locks.js`: ``` import { GENERAL_SETTINGS } from '../utils/locks'; test.describe( 'Settings -> General', { lock: GENERAL_SETTINGS }, () => {} ); ``` A form save writes every field on the page, not only the field that the test changed. A spec that saves the General Settings form (`options-general.php`) must hold `GENERAL_SETTINGS`. A spec in `parallel` that changes a General Settings option in a different way, for example with `siteSettings` or `siteOptions`, must also hold it. ## Logged-out pages Most specs run logged in as the admin user. There are two ways to run a test logged out: * To make the `page` of a whole `describe` group logged out, add `test.use( { storageState: { cookies: [], origins: [] } } )` to the group. See `specs/dashboard.test.js`. * To use a logged-out page next to the logged-in `page` in one test, import `test` and `expect` from `../fixtures` and use the `loggedOutPage` fixture. Its browser context has no cookies, and it closes when the test ends. Unlike `page`, it resolves a URL that starts with `/` against the host only, not the full `WP_BASE_URL`, so it does not work with a site in a subdirectory. See `specs/cache-control-headers-directives.test.js`. ## Fixtures To create posts, pages or users, to upload attachments, or to change a setting, an option, or a must-use plugin, import `test` and `expect` from `../fixtures`, and use one of the fixtures below. Each fixture cleans up when the test ends, also if the test fails. If the cleanup fails, the test fails. * `siteSettings`: reads and changes the settings of `/wp/v2/settings` through REST, with `get()`, `remember( keys )` and `set( patch )`. It restores only the keys passed to `remember` or `set`. See `specs/privacy.test.js`. * `siteOptions`: reads and changes options through WP-CLI, with `get( names )`, `remember( names )` and `set( values )`. It reads and restores the stored `gmt_offset`, not the value calculated for a named timezone. It restores only the keys passed to `remember` or `set`. See `specs/general-settings-invalid-timezone.test.js`. * `muPlugin`: writes a must-use plugin with `write( fileName, php )`, and deletes the file when the test ends. Call `write` before the test visits a page. See `specs/fatal-error-handler.test.js`. * `postFactory`: creates posts with `create( data )` and pages with `createPage( data )`, and force-deletes them when the test ends. `uniqueTitle( prefix )` adds a UUID to the prefix. See `specs/edit-posts.test.js`. * `attachments`: uploads a file with `upload( absolutePath )`, records an attachment that the browser uploaded with `track( id )`, and deletes them when the test ends. See `specs/media-new-client-side-upload.test.js`. * `userFactory`: creates users with `create( { roles, ...data } )`, which returns `{ user, username, password }`. For a user that the test creates through a form, call `adopt( username )` before the form is submitted. When the test ends, it deletes each user and the user's auto-drafts. To log in as the user, import `signIn( page, username, password )` from `../fixtures/user`. See `specs/users.test.js`. If the test changes a setting or an option through a form, first call `remember` with an array of the keys that the form changes. The fixture then reads the current value, without a write, and restores it when the test ends. `remember` returns the current values. A fixture goes in `fixtures/` only when 2 or more spec files use it. A fixture that only one spec uses stays in that spec. ## Documentation * Block Editor Handbook end to end testing overview: https://developer.wordpress.org/block-editor/contributors/code/testing-overview/#end-to-end-testing * Gutenberg e2e-test-utils-playwright package API docs: https://github.com/WordPress/gutenberg/tree/trunk/packages/e2e-test-utils-playwright * Playwright API docs: https://playwright.dev/docs (the version we are using is indicated in the `@wordpress/scripts` package: https://github.com/WordPress/gutenberg/blob/trunk/packages/scripts/package.json) \ No newline at end of file diff --git a/tests/e2e/playwright.config.js b/tests/e2e/playwright.config.js index 43c8bd10c3ffc..d6fad8f51c4a3 100644 --- a/tests/e2e/playwright.config.js +++ b/tests/e2e/playwright.config.js @@ -47,7 +47,7 @@ const config = defineConfig( { name: 'parallel', default: false, testIgnore: SERIAL_SPECS, - fullyParallel: false, + fullyParallel: true, use: { ...devices[ 'Desktop Chrome' ] }, }, { diff --git a/tests/e2e/specs/classic-menus.test.js b/tests/e2e/specs/classic-menus.test.js index 102094f14dd65..df02d659141fa 100644 --- a/tests/e2e/specs/classic-menus.test.js +++ b/tests/e2e/specs/classic-menus.test.js @@ -1,16 +1,48 @@ +/** + * External dependencies + */ +import { randomUUID } from 'node:crypto'; + /** * WordPress dependencies */ import { test, expect } from '@wordpress/e2e-test-utils-playwright'; +/** + * Internal dependencies + */ +import { NAV_MENU_LOCATIONS } from '../utils/locks'; + /** * The classic menus admin screen (nav-menus.php). Relies on Twenty * Twenty-One being the active theme, which registers the "Primary menu" * and "Secondary menu" locations. + * + * Assigning a menu to a location and deleting any menu both write the + * theme's menu locations, so every test holds the lock. */ -test.describe( 'Classic Menus', () => { - test.beforeEach( async ( { requestUtils } ) => { - await requestUtils.deleteAllMenus(); +test.describe( 'Classic Menus', { lock: NAV_MENU_LOCATIONS }, () => { + let prefix; + + test.beforeEach( async () => { + prefix = `e2e-menu-${ randomUUID() }`; + } ); + + // Deleting a menu also removes it from the theme's menu locations. + test.afterEach( async ( { requestUtils } ) => { + const menus = await requestUtils.rest( { + path: '/wp/v2/menus', + params: { search: prefix, per_page: 100 }, + } ); + for ( const menu of menus ) { + if ( menu.name.startsWith( `${ prefix }-` ) ) { + await requestUtils.rest( { + method: 'DELETE', + path: `/wp/v2/menus/${ menu.id }`, + params: { force: true }, + } ); + } + } } ); test( 'creates a menu, adds a custom link, and assigns it to a location', async ( { @@ -18,20 +50,22 @@ test.describe( 'Classic Menus', () => { page, requestUtils, } ) => { - await admin.visitAdminPage( '/nav-menus.php' ); + const menuName = `${ prefix }-menu`; + const linkText = `${ prefix }-link`; + + await admin.visitAdminPage( '/nav-menus.php', 'action=edit&menu=0' ); - // With no menus, the screen opens directly on the create form. const menuNameField = page.getByRole( 'textbox', { name: 'Menu Name', } ); - await menuNameField.fill( 'E2E Menu' ); + await menuNameField.fill( menuName ); await page.locator( '#save_menu_footer' ).click(); // The screen reloads into the edit state for the new menu. await expect( page ).toHaveURL( /[?&]menu=[1-9]\d*/ ); await expect( page.getByRole( 'textbox', { name: 'Menu Name' } ) - ).toHaveValue( 'E2E Menu' ); + ).toHaveValue( menuName ); // Add a custom link from the accordion. await page.getByRole( 'button', { name: 'Custom Links' } ).click(); @@ -40,7 +74,7 @@ test.describe( 'Classic Menus', () => { .fill( 'https://wordpress.org/' ); await page .getByRole( 'textbox', { name: 'Link Text' } ) - .fill( 'E2E Custom Link' ); + .fill( linkText ); // Each accordion panel has its own submit; scope to Custom Links. await page .locator( '#customlinkdiv' ) @@ -51,7 +85,7 @@ test.describe( 'Classic Menus', () => { await expect( page .locator( '#menu-to-edit' ) - .getByText( 'E2E Custom Link', { exact: true } ) + .getByText( linkText, { exact: true } ) ).toBeVisible(); // Assign the menu to the theme's primary location and save. @@ -69,7 +103,7 @@ test.describe( 'Classic Menus', () => { await page.locator( '#save_menu_footer' ).focus(); await page.keyboard.press( 'Enter' ); await expect( page.locator( '#message' ) ).toContainText( - 'E2E Menu has been updated.' + `${ menuName } has been updated.` ); // The location assignment persists. @@ -78,9 +112,9 @@ test.describe( 'Classic Menus', () => { async () => { const menus = await requestUtils.rest( { path: '/wp/v2/menus', - params: { context: 'edit' }, + params: { context: 'edit', search: prefix }, } ); - return menus.find( ( menu ) => menu.name === 'E2E Menu' ) + return menus.find( ( menu ) => menu.name === menuName ) ?.locations; }, { @@ -95,14 +129,17 @@ test.describe( 'Classic Menus', () => { await expect( page .getByRole( 'navigation' ) - .getByRole( 'link', { name: 'E2E Custom Link' } ) + .getByRole( 'link', { name: linkText } ) ).toBeVisible(); } ); test( 'deletes a menu', async ( { admin, page, requestUtils } ) => { - await requestUtils.createClassicMenu( 'Doomed Menu' ); + const menu = await requestUtils.createClassicMenu( `${ prefix }-menu` ); - await admin.visitAdminPage( '/nav-menus.php' ); + await admin.visitAdminPage( + '/nav-menus.php', + `action=edit&menu=${ menu.id }` + ); // Deleting asks for confirmation. page.on( 'dialog', ( dialog ) => dialog.accept() ); @@ -113,19 +150,21 @@ test.describe( 'Classic Menus', () => { await page.getByRole( 'link', { name: 'Delete Menu' } ).focus(); await page.keyboard.press( 'Enter' ); - // With the only menu gone, the screen returns to the create form. + // The screen opens any other menu on the site or, with no menus left, + // the create form, so check only that it no longer shows this menu. await expect( page.getByRole( 'textbox', { name: 'Menu Name' } ) - ).toHaveValue( '' ); + ).not.toHaveValue( menu.name ); await expect .poll( async () => { const menus = await requestUtils.rest( { path: '/wp/v2/menus', + params: { search: prefix }, } ); return menus.length; }, - { message: 'no menus should remain' } + { message: 'the menu should be gone' } ) .toBe( 0 ); } ); diff --git a/tests/e2e/specs/privacy.test.js b/tests/e2e/specs/privacy.test.js index bdf9d102b25e5..ccf1728942e14 100644 --- a/tests/e2e/specs/privacy.test.js +++ b/tests/e2e/specs/privacy.test.js @@ -2,12 +2,13 @@ * Internal dependencies */ import { test, expect } from '../fixtures'; +import { PRIVACY_POLICY_PAGE } from '../utils/locks'; function normalizeText( text ) { return text.replace( /\s+/g, ' ' ).trim(); } -test.describe( 'Privacy page creation', () => { +test.describe( 'Privacy page creation', { lock: PRIVACY_POLICY_PAGE }, () => { let createdPageId; test.beforeEach( () => { diff --git a/tests/e2e/utils/locks.js b/tests/e2e/utils/locks.js index 6c2afafe7223c..b118484eb05ea 100644 --- a/tests/e2e/utils/locks.js +++ b/tests/e2e/utils/locks.js @@ -1 +1,3 @@ export const GENERAL_SETTINGS = 'general-settings'; +export const NAV_MENU_LOCATIONS = 'nav-menu-locations'; +export const PRIVACY_POLICY_PAGE = 'privacy-policy-page'; From 121d2ff7b11a507a3fd8c427e5007c0266d76220 Mon Sep 17 00:00:00 2001 From: Adrian Moldovan <3854374+adimoldovan@users.noreply.github.com> Date: Thu, 12 Nov 2026 16:53:39 +0200 Subject: [PATCH 2/2] Build/Test Tools: Drop the E2E README note that tests in one file run in order. See #66392. --- tests/e2e/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/e2e/README.md b/tests/e2e/README.md index 133f1b69a8dae..b4dece7e23667 100644 --- a/tests/e2e/README.md +++ b/tests/e2e/README.md @@ -1 +1 @@ -# E2E Tests End-To-End (E2E) tests for WordPress. ## Running the tests The e2e tests require a production-like environment to run. By default, they will assume an environment is available at `http://localhost:8889`, with username `admin` and password `password`. If you don't already have an environment ready, you can set one up by following [these instructions](https://github.com/WordPress/wordpress-develop/blob/master/README.md). Then you can launch the tests by running: ``` npm run test:e2e ``` which will run the test suite using a headless browser. If your environment has a different url, username or password to the default, you can provide the base URL, username and password like this: ``` WP_BASE_URL=http://mycustomurl WP_USERNAME=username WP_PASSWORD=password npm run test:e2e ``` **DO NOT run these tests in an actual production environment, as they will delete all your content.** For debugging purposes, you might want to follow the test visually. You can do so by running the tests in an interactive mode: ``` npm run test:e2e -- --ui ``` [UI Mode](https://playwright.dev/docs/test-ui-mode) let's you explore, run and debug tests with a time travel experience complete with watch mode. All test files are loaded into the testing sidebar where you can expand each file and describe block to individually run, view, watch and debug each test. You can also run a single test file separately: ``` npm run test:e2e tests/e2e/specs/hello.test.js ``` ## Projects The suite has three Playwright projects, defined in `playwright.config.js`: * `all`: every spec, one at a time on one worker. `npm run test:e2e` runs this project. * `parallel`: specs that can share the site with other tests. Spec files run at the same time on several workers, one for every two CPU cores. Tests inside one file still run in order. Run it with `npm run test:e2e -- --project=parallel`. * `serial`: specs that break the site for other tests. They run one at a time on one worker. Run it with `npm run test:e2e -- --project=serial`. CI runs `parallel` and `serial` in separate jobs. New specs go in `parallel` by default. Add a spec to `SERIAL_SPECS` if it makes the site unusable for other tests. If it changes content or settings that other tests use, for example with `requestUtils.deleteAllPosts()`, follow [Parallel safety](#parallel-safety). Before you open a pull request with a new spec, run its project a few times with `--shuffle`. Playwright then runs the spec files in a random order, so a spec that needs another spec to run first, or that breaks a spec that runs after it, fails: ``` npm run test:e2e -- --project=parallel --shuffle ``` Playwright prints the seed of each run, for example `shuffle seed 317753389`. To run the same order again, give the seed: `--shuffle 317753389`. ## Parallel safety Specs in `parallel` share one site. A test that changes shared data can break another test that runs at the same time. The `parallel` project sets `fullyParallel: true`, so this can be a test in another file or a test in the same file. To make a spec safe, use the first of these that works: 1. Scope the data. Let the test create its own data, for example a post with a unique title, and read only that data. 2. If the test must change shared data, such as a site setting, give it a lock. Do this only if every spec that writes the same data holds the same lock. 3. If you cannot lock every spec that writes the data, add the spec to `SERIAL_SPECS`. A lock is a name. Playwright never runs two tests that hold the same lock name at the same time. A lock on a `describe` group applies to each test in the group, not to the group as a whole. A lock only schedules tests. It does not isolate data: a test without the lock can still change the data while a locked test runs. The lock names are in `utils/locks.js`. Import the constant, so that all specs use the same name. Playwright gives no error for a lock name that no other test uses, or for an import name that is not in `utils/locks.js`: ``` import { GENERAL_SETTINGS } from '../utils/locks'; test.describe( 'Settings -> General', { lock: GENERAL_SETTINGS }, () => {} ); ``` A form save writes every field on the page, not only the field that the test changed. A spec that saves the General Settings form (`options-general.php`) must hold `GENERAL_SETTINGS`. A spec in `parallel` that changes a General Settings option in a different way, for example with `siteSettings` or `siteOptions`, must also hold it. ## Logged-out pages Most specs run logged in as the admin user. There are two ways to run a test logged out: * To make the `page` of a whole `describe` group logged out, add `test.use( { storageState: { cookies: [], origins: [] } } )` to the group. See `specs/dashboard.test.js`. * To use a logged-out page next to the logged-in `page` in one test, import `test` and `expect` from `../fixtures` and use the `loggedOutPage` fixture. Its browser context has no cookies, and it closes when the test ends. Unlike `page`, it resolves a URL that starts with `/` against the host only, not the full `WP_BASE_URL`, so it does not work with a site in a subdirectory. See `specs/cache-control-headers-directives.test.js`. ## Fixtures To create posts, pages or users, to upload attachments, or to change a setting, an option, or a must-use plugin, import `test` and `expect` from `../fixtures`, and use one of the fixtures below. Each fixture cleans up when the test ends, also if the test fails. If the cleanup fails, the test fails. * `siteSettings`: reads and changes the settings of `/wp/v2/settings` through REST, with `get()`, `remember( keys )` and `set( patch )`. It restores only the keys passed to `remember` or `set`. See `specs/privacy.test.js`. * `siteOptions`: reads and changes options through WP-CLI, with `get( names )`, `remember( names )` and `set( values )`. It reads and restores the stored `gmt_offset`, not the value calculated for a named timezone. It restores only the keys passed to `remember` or `set`. See `specs/general-settings-invalid-timezone.test.js`. * `muPlugin`: writes a must-use plugin with `write( fileName, php )`, and deletes the file when the test ends. Call `write` before the test visits a page. See `specs/fatal-error-handler.test.js`. * `postFactory`: creates posts with `create( data )` and pages with `createPage( data )`, and force-deletes them when the test ends. `uniqueTitle( prefix )` adds a UUID to the prefix. See `specs/edit-posts.test.js`. * `attachments`: uploads a file with `upload( absolutePath )`, records an attachment that the browser uploaded with `track( id )`, and deletes them when the test ends. See `specs/media-new-client-side-upload.test.js`. * `userFactory`: creates users with `create( { roles, ...data } )`, which returns `{ user, username, password }`. For a user that the test creates through a form, call `adopt( username )` before the form is submitted. When the test ends, it deletes each user and the user's auto-drafts. To log in as the user, import `signIn( page, username, password )` from `../fixtures/user`. See `specs/users.test.js`. If the test changes a setting or an option through a form, first call `remember` with an array of the keys that the form changes. The fixture then reads the current value, without a write, and restores it when the test ends. `remember` returns the current values. A fixture goes in `fixtures/` only when 2 or more spec files use it. A fixture that only one spec uses stays in that spec. ## Documentation * Block Editor Handbook end to end testing overview: https://developer.wordpress.org/block-editor/contributors/code/testing-overview/#end-to-end-testing * Gutenberg e2e-test-utils-playwright package API docs: https://github.com/WordPress/gutenberg/tree/trunk/packages/e2e-test-utils-playwright * Playwright API docs: https://playwright.dev/docs (the version we are using is indicated in the `@wordpress/scripts` package: https://github.com/WordPress/gutenberg/blob/trunk/packages/scripts/package.json) \ No newline at end of file +# E2E Tests End-To-End (E2E) tests for WordPress. ## Running the tests The e2e tests require a production-like environment to run. By default, they will assume an environment is available at `http://localhost:8889`, with username `admin` and password `password`. If you don't already have an environment ready, you can set one up by following [these instructions](https://github.com/WordPress/wordpress-develop/blob/master/README.md). Then you can launch the tests by running: ``` npm run test:e2e ``` which will run the test suite using a headless browser. If your environment has a different url, username or password to the default, you can provide the base URL, username and password like this: ``` WP_BASE_URL=http://mycustomurl WP_USERNAME=username WP_PASSWORD=password npm run test:e2e ``` **DO NOT run these tests in an actual production environment, as they will delete all your content.** For debugging purposes, you might want to follow the test visually. You can do so by running the tests in an interactive mode: ``` npm run test:e2e -- --ui ``` [UI Mode](https://playwright.dev/docs/test-ui-mode) let's you explore, run and debug tests with a time travel experience complete with watch mode. All test files are loaded into the testing sidebar where you can expand each file and describe block to individually run, view, watch and debug each test. You can also run a single test file separately: ``` npm run test:e2e tests/e2e/specs/hello.test.js ``` ## Projects The suite has three Playwright projects, defined in `playwright.config.js`: * `all`: every spec, one at a time on one worker. `npm run test:e2e` runs this project. * `parallel`: specs that can share the site with other tests. Tests run at the same time on several workers, one for every two CPU cores. Run it with `npm run test:e2e -- --project=parallel`. * `serial`: specs that break the site for other tests. They run one at a time on one worker. Run it with `npm run test:e2e -- --project=serial`. CI runs `parallel` and `serial` in separate jobs. New specs go in `parallel` by default. Add a spec to `SERIAL_SPECS` if it makes the site unusable for other tests. If it changes content or settings that other tests use, for example with `requestUtils.deleteAllPosts()`, follow [Parallel safety](#parallel-safety). Before you open a pull request with a new spec, run its project a few times with `--shuffle`. Playwright then runs the spec files in a random order, so a spec that needs another spec to run first, or that breaks a spec that runs after it, fails: ``` npm run test:e2e -- --project=parallel --shuffle ``` Playwright prints the seed of each run, for example `shuffle seed 317753389`. To run the same order again, give the seed: `--shuffle 317753389`. ## Parallel safety Specs in `parallel` share one site. A test that changes shared data can break another test that runs at the same time. The `parallel` project sets `fullyParallel: true`, so this can be a test in another file or a test in the same file. To make a spec safe, use the first of these that works: 1. Scope the data. Let the test create its own data, for example a post with a unique title, and read only that data. 2. If the test must change shared data, such as a site setting, give it a lock. Do this only if every spec that writes the same data holds the same lock. 3. If you cannot lock every spec that writes the data, add the spec to `SERIAL_SPECS`. A lock is a name. Playwright never runs two tests that hold the same lock name at the same time. A lock on a `describe` group applies to each test in the group, not to the group as a whole. A lock only schedules tests. It does not isolate data: a test without the lock can still change the data while a locked test runs. The lock names are in `utils/locks.js`. Import the constant, so that all specs use the same name. Playwright gives no error for a lock name that no other test uses, or for an import name that is not in `utils/locks.js`: ``` import { GENERAL_SETTINGS } from '../utils/locks'; test.describe( 'Settings -> General', { lock: GENERAL_SETTINGS }, () => {} ); ``` A form save writes every field on the page, not only the field that the test changed. A spec that saves the General Settings form (`options-general.php`) must hold `GENERAL_SETTINGS`. A spec in `parallel` that changes a General Settings option in a different way, for example with `siteSettings` or `siteOptions`, must also hold it. ## Logged-out pages Most specs run logged in as the admin user. There are two ways to run a test logged out: * To make the `page` of a whole `describe` group logged out, add `test.use( { storageState: { cookies: [], origins: [] } } )` to the group. See `specs/dashboard.test.js`. * To use a logged-out page next to the logged-in `page` in one test, import `test` and `expect` from `../fixtures` and use the `loggedOutPage` fixture. Its browser context has no cookies, and it closes when the test ends. Unlike `page`, it resolves a URL that starts with `/` against the host only, not the full `WP_BASE_URL`, so it does not work with a site in a subdirectory. See `specs/cache-control-headers-directives.test.js`. ## Fixtures To create posts, pages or users, to upload attachments, or to change a setting, an option, or a must-use plugin, import `test` and `expect` from `../fixtures`, and use one of the fixtures below. Each fixture cleans up when the test ends, also if the test fails. If the cleanup fails, the test fails. * `siteSettings`: reads and changes the settings of `/wp/v2/settings` through REST, with `get()`, `remember( keys )` and `set( patch )`. It restores only the keys passed to `remember` or `set`. See `specs/privacy.test.js`. * `siteOptions`: reads and changes options through WP-CLI, with `get( names )`, `remember( names )` and `set( values )`. It reads and restores the stored `gmt_offset`, not the value calculated for a named timezone. It restores only the keys passed to `remember` or `set`. See `specs/general-settings-invalid-timezone.test.js`. * `muPlugin`: writes a must-use plugin with `write( fileName, php )`, and deletes the file when the test ends. Call `write` before the test visits a page. See `specs/fatal-error-handler.test.js`. * `postFactory`: creates posts with `create( data )` and pages with `createPage( data )`, and force-deletes them when the test ends. `uniqueTitle( prefix )` adds a UUID to the prefix. See `specs/edit-posts.test.js`. * `attachments`: uploads a file with `upload( absolutePath )`, records an attachment that the browser uploaded with `track( id )`, and deletes them when the test ends. See `specs/media-new-client-side-upload.test.js`. * `userFactory`: creates users with `create( { roles, ...data } )`, which returns `{ user, username, password }`. For a user that the test creates through a form, call `adopt( username )` before the form is submitted. When the test ends, it deletes each user and the user's auto-drafts. To log in as the user, import `signIn( page, username, password )` from `../fixtures/user`. See `specs/users.test.js`. If the test changes a setting or an option through a form, first call `remember` with an array of the keys that the form changes. The fixture then reads the current value, without a write, and restores it when the test ends. `remember` returns the current values. A fixture goes in `fixtures/` only when 2 or more spec files use it. A fixture that only one spec uses stays in that spec. ## Documentation * Block Editor Handbook end to end testing overview: https://developer.wordpress.org/block-editor/contributors/code/testing-overview/#end-to-end-testing * Gutenberg e2e-test-utils-playwright package API docs: https://github.com/WordPress/gutenberg/tree/trunk/packages/e2e-test-utils-playwright * Playwright API docs: https://playwright.dev/docs (the version we are using is indicated in the `@wordpress/scripts` package: https://github.com/WordPress/gutenberg/blob/trunk/packages/scripts/package.json) \ No newline at end of file