Skip to content
Open
74 changes: 71 additions & 3 deletions e2e/playwright/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,11 +52,64 @@ docker/run.sh --update-snapshots # rewrite the etalons of the tests that ran
The etalons here are recorded anew, not copied from `e2e/testcafe-devextreme`: the two runners
crop and render differently, so a TestCafe image is not a valid reference for this suite.

It is still the check of the port. Recording writes whatever the test reached, and a verify run
compares against that same file, so neither notices a test that shot the wrong state. Before
deleting the TestCafe etalon, compare the new one with it: the same picture, within the per-pixel
threshold. Three differences are expected:

- A scrollbar. This suite shows it, TestCafe cut it off, so a viewport shot of a page that
overflows comes out up to 15 px larger, with the scrollbar in the extra strip.
- Anything under a TestCafe mask (`<name>_mask.png` next to the etalon, black where ignored).
- Single antialiased pixels on the edges of shapes that sit on fractional coordinates. The TestCafe
comparer ignored antialiasing (looks-same, `ignoreAntialiasing`, tolerance 5), so its etalon
could drift there unnoticed; this suite compares them strictly.

Anything else, a block of pixels or a moved edge, means the test reached a different state; find
out why rather than accept the image.

**Masks.** `testScreenshot` takes no mask, and the usual reason for one is gone:
`toHaveScreenshot` stops CSS animations (an infinite one at its first frame) and hides the caret.
Port a masked test without its mask, then run it strictly in the container a few times in a row
(`--repeat-each 10`). If it is stable, delete the mask with the TestCafe etalon. If it is not, make
the state deterministic (stop the animation, fix the data or the clock) rather than hide it.

Migrating a test means, in one PR: port the test, record its etalon here, and delete the TestCafe
test together with its etalon. The two sets must never cover the same case at once — a stale pair
left behind gets updated by whoever touches the component next and starts asserting a render
nobody checks.

### Record etalons on a Mac

The image is `linux/amd64`, the platform of the CI agent. On Apple Silicon it runs through Rosetta;
under QEMU emulation Chrome is several times slower. [Colima](https://github.com/abiosoft/colima)
gives the `docker` CLI a free runtime that uses Rosetta:

```bash
softwareupdate --install-rosetta --agree-to-license
brew install colima docker docker-buildx
colima start --vm-type vz --vz-rosetta --cpu 4 --memory 8
docker run --rm --platform linux/amd64 ubuntu:24.04 uname -m # prints x86_64
```

4 CPUs and 8 GB are enough for a component folder: `docker/run.sh` takes half the cores, so two
workers. Give the VM more for a larger run (`colima stop`, then `colima start --cpu 8 --memory 12`).
After a reboot, `colima start` brings it back with the same settings.

The container uses the host's `node_modules` and DevExtreme build, so build first and rebuild after
a source change. Then, from `e2e/playwright`, record and verify the etalons of the folder:

```bash
pnpm exec nx build devextreme-scss # from the repository root
pnpm exec nx build devextreme -c testing

docker/run.sh --update-snapshots tests/editors/selectBox/

docker/run.sh tests/editors/selectBox/
```

The first run builds the image and takes a few minutes. Look through the new files before
committing them: a new etalon is only as right as the render it captured.

## Themes

A test runs in the default theme unless it says otherwise. The jobs that run the whole suite in
Expand Down Expand Up @@ -159,6 +212,13 @@ viewport is 1200×800 and the widgets of the previous test are disposed.
Port a test one to one: the same steps, the same checks, the same screenshot names. A title changes
only when it repeats in the file, which Playwright rejects: add the loop parameters to it.

**`runInTheme`.** In TestCafe it was a filter, not a setting: the test ran only in the job whose
theme matched — usually one of the whole-suite theme jobs — and the folder jobs skipped it.
`test.use({ theme })` instead runs the test in that theme in the job that takes its folder. That
is the port: the result does not depend on the job, and a screenshot keeps one etalon named after
the pinned theme. Do not add a theme tag on top of it: the
test would run once more in the tagged job.

**Typing.** `fill` sets the value at once and sends no key events, so masked, numeric and date
editors do not react to it. Type with `pressSequentially`, and press keys under their Playwright
names: `Enter`, `Escape`, `ArrowLeft`, `Control+a`.
Expand All @@ -170,13 +230,21 @@ whenever the page is a moment late.
**Waiting.** A step waits for the state it needs, never for time. `helpers/`, `models/` and the
test folders migrated under this rule (see `eslint.config.mjs`) may not use `waitForTimeout`,
`setTimeout`, `force: true`, `toPass`, a `timeout` option or a call that raises a timeout
(`test.setTimeout`, `page.setDefaultTimeout`). When a widget reacts to a change
through an observer and leaves nothing to assert on, `waitForNextFrame` waits for the frame in which
the observer runs.
(`test.setTimeout`, `page.setDefaultTimeout`).

**Clicks.** A click is a real one: it lands on the topmost element at that point and fails when
that is not the target. TestCafe delivered a click to a covered element anyway, so a test that
relied on it needs the target made reachable, not the check switched off.

**Scrolling before a click.** TestCafe scrolled the whole target into view before a click;
Playwright scrolls only as far as the click point. When the target is wider or taller than the
window, the focus then scrolls the page after the click, and a drop-down closes on that scroll —
sometimes before the next step, sometimes after it. Scroll the target into view first, as TestCafe
did: `element.scrollIntoView({ block: 'nearest', inline: 'nearest' })` in `locator.evaluate`.

**Page errors.** A test fails on an uncaught error in the page, as it did in TestCafe. A test whose
TestCafe version ended with `expect(true).ok()` checks exactly that and needs no assertion.

**Old models.** A model in `packages/testcafe-models` that the ported tests used goes in the same
PR, but only after a search proves nothing else uses it: grep all of `e2e/` and `packages/` for its
import path, since other tests and other models import models too. A model still in use stays.
23 changes: 11 additions & 12 deletions e2e/playwright/helpers/createWidget.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,28 +2,27 @@ import type { Page } from '@playwright/test';
import type { WidgetName, WidgetOptions } from '../models/types';
import { DEFAULT_SELECTOR } from './const';

export interface CreateWidgetOptions {
disableFxAnimation: boolean;
export interface CreateWidgetOptions<TArg = undefined> {
disableFxAnimation?: boolean;
arg?: TArg;
}

const DEFAULT_OPTIONS: CreateWidgetOptions = {
disableFxAnimation: true,
};
type WidgetConfig<TWidgetName> = TWidgetName extends keyof WidgetOptions
? WidgetOptions[TWidgetName]
: object;

export const createWidget = async <TWidgetName extends WidgetName>(
export const createWidget = async <TWidgetName extends WidgetName, TArg = undefined>(
page: Page,
widgetName: TWidgetName,
widgetOptions: TWidgetName extends keyof WidgetOptions
? WidgetOptions[TWidgetName]
| (() => WidgetOptions[TWidgetName] | Promise<WidgetOptions[TWidgetName]>)
: unknown,
widgetOptions: WidgetConfig<TWidgetName>
| ((arg: TArg) => WidgetConfig<TWidgetName> | Promise<WidgetConfig<TWidgetName>>),
selector: string = DEFAULT_SELECTOR,
{ disableFxAnimation } = DEFAULT_OPTIONS,
{ disableFxAnimation = true, arg }: CreateWidgetOptions<TArg> = {},
): Promise<void> => {
// Handlers cannot pass through the "page.evaluate" arguments, so a configuration with them
// arrives as a factory that Playwright runs in the page.
const options = typeof widgetOptions === 'function'
? await page.evaluateHandle(widgetOptions as () => unknown)
? await page.evaluateHandle(widgetOptions as (value: any) => unknown, arg)
: widgetOptions;

await page.evaluate(({
Expand Down
17 changes: 0 additions & 17 deletions e2e/playwright/helpers/domUtils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,17 +14,6 @@ export const setAttribute = async (
{ elementSelector: selector, name: attribute, attributeValue: value },
);

export const removeAttribute = async (
page: Page,
selector: string,
attribute: string,
): Promise<void> => page.evaluate(
({ elementSelector, name }) => {
document.querySelector(elementSelector)?.removeAttribute(name);
},
{ elementSelector: selector, name: attribute },
);

export const appendAttributeValue = async (
page: Page,
selector: string,
Expand Down Expand Up @@ -152,9 +141,3 @@ export const addFocusableElementBefore = async (
},
{ selector: targetSelector, id: elementId },
);

export const waitForNextFrame = async (page: Page): Promise<void> => page.evaluate(
() => new Promise<void>((resolve) => {
requestAnimationFrame(() => requestAnimationFrame(() => resolve()));
}),
);
13 changes: 13 additions & 0 deletions e2e/playwright/models/dropDownButton.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
import type { Locator } from '@playwright/test';
import type { WidgetName } from './types';
import DropDownList from './internal/dropDownList';

export default class DropDownButton extends DropDownList {
// eslint-disable-next-line class-methods-use-this
public getName(): WidgetName { return 'dxDropDownButton'; }

// The widget element itself names the popup content.
public getPopupOwner(): Locator {
return this.element;
}
}
13 changes: 13 additions & 0 deletions e2e/playwright/models/internal/classMatcher.ts
Original file line number Diff line number Diff line change
@@ -1 +1,14 @@
import type { Locator } from '@playwright/test';
import { expect } from '@playwright/test';

export const classMatcher = (className: string): RegExp => new RegExp(`(^|\\s)${className}(\\s|$)`);

export const expectClass = async (
locator: Locator,
className: string,
present = true,
): Promise<void> => {
const assertion = expect(locator);

await (present ? assertion : assertion.not).toHaveClass(classMatcher(className));
};
59 changes: 59 additions & 0 deletions e2e/playwright/models/internal/dropDownList.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
import type { Locator, Page } from '@playwright/test';
import { expect } from '@playwright/test';
import List from '../list';
import TextBox from '../textBox';

const ATTR = {
popupId: 'aria-owns',
};

const CLASS = {
dropDownButton: 'dx-dropdowneditor-button',
invisible: 'dx-state-invisible',
list: 'dx-list',
};

export default abstract class DropDownList extends TextBox {
public readonly dropDownButton: Locator;

constructor(page: Page, selector: Locator | string) {
super(page, selector);

this.dropDownButton = this.element.locator(`.${CLASS.dropDownButton}`);
}

// The element that names the popup content in "aria-owns" once the popup is rendered; the
// content itself sits in an overlay wrapper elsewhere in the body.
public getPopupOwner(): Locator {
return this.input;
}

public isOpened(): Promise<boolean> {
return this.getPopupOwner().evaluate((owner, { attr, invisible }) => {
const popupId = owner.getAttribute(attr);
const popup = popupId ? document.getElementById(popupId) : null;

return !!popup && !popup.parentElement?.classList.contains(invisible);
}, { attr: ATTR.popupId, invisible: CLASS.invisible });
}

public async getPopup(): Promise<Locator> {
const owner = this.getPopupOwner();

await expect(owner).toHaveAttribute(ATTR.popupId, /./);

const popupId = await owner.getAttribute(ATTR.popupId);

return this.page.locator(`#${popupId}`);
}

public async getList(): Promise<List> {
await expect.poll(() => this.isOpened()).toBe(true);

return new List(this.page, (await this.getPopup()).locator(`.${CLASS.list}`));
}

public async open(): Promise<void> {
await this.invoke('open');
}
}
9 changes: 3 additions & 6 deletions e2e/playwright/models/internal/widget.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
import type { Locator, Page } from '@playwright/test';
import { expect } from '@playwright/test';
import type { WidgetName } from '../types';
import { hasClass } from './hasClass';
import { classMatcher } from './classMatcher';
import { expectClass } from './classMatcher';

const CLASS = {
focused: 'dx-state-focused',
Expand All @@ -29,10 +28,8 @@ export default abstract class Widget {
return `dx-${widgetName.slice(2).toLowerCase() + (className ? `-${className}` : '')}`;
}

public async expectClass(className: string, present = true): Promise<void> {
const assertion = expect(this.element);

await (present ? assertion : assertion.not).toHaveClass(classMatcher(className));
public expectClass(className: string, present = true): Promise<void> {
return expectClass(this.element, className, present);
}

public expectState(state: WidgetState, present = true): Promise<void> {
Expand Down
28 changes: 28 additions & 0 deletions e2e/playwright/models/list/checkBox.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
import type { Locator } from '@playwright/test';
import { expectClass } from '../internal/classMatcher';

const CLASS = {
checkBox: 'dx-checkbox',
checked: 'dx-checkbox-checked',
focused: 'dx-state-focused',
};

export default class ListItemCheckBox {
public readonly element: Locator;

private readonly item: Locator;

constructor(item: Locator) {
this.item = item;
this.element = item.locator(`.${CLASS.checkBox}`);
}

public expectChecked(present = true): Promise<void> {
return expectClass(this.element, CLASS.checked, present);
}

// The list moves the focus between its items; the check box itself never takes it.
public expectFocused(present = true): Promise<void> {
return expectClass(this.item, CLASS.focused, present);
}
}
34 changes: 34 additions & 0 deletions e2e/playwright/models/list/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
import type { Locator, Page } from '@playwright/test';
import type { WidgetName } from '../types';
import Widget from '../internal/widget';
import ListItem from './item';

const CLASS = {
item: 'dx-list-item',
nestedItem: 'nested-item',
selectAll: 'dx-list-select-all',
};

export default class List extends Widget {
public readonly items: Locator;

public readonly selectAll: ListItem;

constructor(page: Page, selector: Locator | string) {
super(page, selector);

this.items = this.element.locator(`.${CLASS.item}:not(.${CLASS.nestedItem})`);
this.selectAll = new ListItem(this.element.locator(`.${CLASS.selectAll}`));
}

// eslint-disable-next-line class-methods-use-this
public getName(): WidgetName { return 'dxList'; }

public getItem(index = 0): ListItem {
return new ListItem(this.items.nth(index));
}

public async scrollTo(value: number): Promise<void> {
await this.invoke('scrollTo', value);
}
}
22 changes: 22 additions & 0 deletions e2e/playwright/models/list/item.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
import type { Locator } from '@playwright/test';
import { expectClass } from '../internal/classMatcher';
import ListItemCheckBox from './checkBox';

const CLASS = {
disabled: 'dx-state-disabled',
};

export default class ListItem {
public readonly element: Locator;

public readonly checkBox: ListItemCheckBox;

constructor(element: Locator) {
this.element = element;
this.checkBox = new ListItemCheckBox(element);
}

public expectDisabled(present = true): Promise<void> {
return expectClass(this.element, CLASS.disabled, present);
}
}
25 changes: 25 additions & 0 deletions e2e/playwright/models/lookup.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
import type { Locator, Page } from '@playwright/test';
import type { WidgetName } from './types';
import DropDownList from './internal/dropDownList';

const CLASS = {
field: 'dx-lookup-field',
};

export default class Lookup extends DropDownList {
public readonly field: Locator;

constructor(page: Page, selector: Locator | string) {
super(page, selector);

this.field = this.element.locator(`.${CLASS.field}`);
}

// eslint-disable-next-line class-methods-use-this
public getName(): WidgetName { return 'dxLookup'; }

// Lookup has no text input: the field takes its place.
public getPopupOwner(): Locator {
return this.field;
}
}
Loading
Loading