Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
85 changes: 82 additions & 3 deletions e2e/playwright/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,14 @@ as a regular expression. The static server for the test page starts on its own.
comparer did. Etalons live in `etalons/` next to the test and carry the theme in the file name —
`PivotGrid markup (fluent.blue.light).png` — the same convention the TestCafe run uses.

An element shot takes the box of the element. The TestCafe one took its scroll size, so what
overflowed the element — a popup rendered into it, a validation message beside it — was in the
picture. Pass `withOverflow: true` when the test is about that content:

```ts
await testScreenshot(page, 'DateBox popup.png', { element: '#container', withOverflow: true });
```

The pixel budget follows the renderer: the agent and the container draw the same pixels and are
both judged strictly (`maxDiffPixels` 0, `threshold` 0.1), the UI mode included: no pixel may drift
beyond the per-pixel threshold, which is what the TestCafe comparer enforced. A run on the host is
Expand All @@ -52,11 +60,67 @@ docker/run.sh --update-snapshots # rewrite the etalons of the tests that ran
The etalons here are recorded anew, not copied from `e2e/testcafe-devextreme`: the two runners
crop and render differently, so a TestCafe image is not a valid reference for this suite.

It is still the check of the port. Recording writes whatever the test reached, and a verify run
compares against that same file, so neither notices a test that shot the wrong state. Before
deleting the TestCafe etalon, compare the new one with it: the same picture, within the per-pixel
threshold. Three differences are expected:

- A scrollbar. This suite shows it, TestCafe cut it off, so a viewport shot of a page that
overflows comes out up to 15 px larger, with the scrollbar in the extra strip.
- Anything under a TestCafe mask (`<name>_mask.png` next to the etalon, black where ignored).
- Single antialiased pixels on the edges of shapes that sit on fractional coordinates. The TestCafe
comparer ignored antialiasing (looks-same, `ignoreAntialiasing`, tolerance 5), so its etalon
could drift there unnoticed; this suite compares them strictly.

Anything else, a block of pixels or a moved edge, means the test reached a different state; find
out why rather than accept the image.

**Masks.** `testScreenshot` takes no mask, and the usual reason for one is gone:
`toHaveScreenshot` stops CSS animations (an infinite one at its first frame) and hides the caret.
Port a masked test without its mask, then run it strictly in the container a few times in a row
(`--repeat-each 10`). If it is stable, delete the mask with the TestCafe etalon. If it is not, make
the state deterministic (stop the animation, fix the data or the clock) rather than hide it.

Migrating a test means, in one PR: port the test, record its etalon here, and delete the TestCafe
test together with its etalon. The two sets must never cover the same case at once — a stale pair
left behind gets updated by whoever touches the component next and starts asserting a render
nobody checks.

### Record etalons on a Mac

The image is `linux/amd64`, the platform of the CI agent. On Apple Silicon it runs through Rosetta;
under QEMU emulation Chrome is several times slower. [Colima](https://github.com/abiosoft/colima)
gives the `docker` CLI a free runtime that uses Rosetta:

```bash
softwareupdate --install-rosetta --agree-to-license
brew install colima docker docker-buildx
colima start --vm-type vz --vz-rosetta --cpu 4 --memory 8
docker run --rm --platform linux/amd64 ubuntu:24.04 uname -m # prints x86_64
```

4 CPUs and 8 GB are enough for a component folder: `docker/run.sh` takes half the cores, so two
workers. Give the VM more for a larger run (`colima stop`, then `colima start --cpu 8 --memory 12`).
After a reboot, `colima start` brings it back with the same settings.

The container uses the host's `node_modules` and DevExtreme build, so build first and rebuild after
a source change. Then, from `e2e/playwright`, record and verify the etalons of the folder in both
themes the CI jobs run:

```bash
pnpm exec nx build devextreme-scss # from the repository root
pnpm exec nx build devextreme -c testing

docker/run.sh --update-snapshots tests/editors/selectBox/
THEME=fluent-next.blue.light docker/run.sh --update-snapshots tests/editors/selectBox/

docker/run.sh tests/editors/selectBox/
THEME=fluent-next.blue.light docker/run.sh tests/editors/selectBox/
```

The first run builds the image and takes a few minutes. Look through the new files before
committing them, as with etalons from CI.

### Etalons from CI

Without a container, CI records the etalons. A test whose etalon does not exist fails, and
Expand Down Expand Up @@ -183,6 +247,13 @@ viewport is 1200×800 and the widgets of the previous test are disposed.
Port a test one to one: the same steps, the same checks, the same screenshot names. A title changes
only when it repeats in the file, which Playwright rejects: add the loop parameters to it.

**`runInTheme`.** In TestCafe it was a filter, not a setting: the test ran only in the job whose
theme matched — usually one of the whole-suite theme jobs — and the folder jobs skipped it.
`test.use({ theme })` instead runs the test in that theme in every job that takes its folder, the
default and the `- fluent-next` one. That is the port: the result does not depend on the job, and a
screenshot keeps one etalon named after the pinned theme. Do not add a theme tag on top of it: the
test would run once more in the tagged job.

**Typing.** `fill` sets the value at once and sends no key events, so masked, numeric and date
editors do not react to it. Type with `pressSequentially`, and press keys under their Playwright
names: `Enter`, `Escape`, `ArrowLeft`, `Control+a`.
Expand All @@ -194,13 +265,21 @@ whenever the page is a moment late.
**Waiting.** A step waits for the state it needs, never for time. `helpers/`, `models/` and the
test folders migrated under this rule (see `eslint.config.mjs`) may not use `waitForTimeout`,
`setTimeout`, `force: true`, `toPass`, a `timeout` option or a call that raises a timeout
(`test.setTimeout`, `page.setDefaultTimeout`). When a widget reacts to a change
through an observer and leaves nothing to assert on, `waitForNextFrame` waits for the frame in which
the observer runs.
(`test.setTimeout`, `page.setDefaultTimeout`).

**Clicks.** A click is a real one: it lands on the topmost element at that point and fails when
that is not the target. TestCafe delivered a click to a covered element anyway, so a test that
relied on it needs the target made reachable, not the check switched off.

**Scrolling before a click.** TestCafe scrolled the whole target into view before a click;
Playwright scrolls only as far as the click point. When the target is wider or taller than the
window, the focus then scrolls the page after the click, and a drop-down closes on that scroll —
sometimes before the next step, sometimes after it. Scroll the target into view first, as TestCafe
did: `element.scrollIntoView({ block: 'nearest', inline: 'nearest' })` in `locator.evaluate`.

**Page errors.** A test fails on an uncaught error in the page, as it did in TestCafe. A test whose
TestCafe version ended with `expect(true).ok()` checks exactly that and needs no assertion.

**Old models.** A model in `packages/testcafe-models` that the ported tests used goes in the same
PR, but only after a search proves nothing else uses it: grep all of `e2e/` and `packages/` for its
import path, since other tests and other models import models too. A model still in use stays.
17 changes: 0 additions & 17 deletions e2e/playwright/helpers/domUtils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,17 +14,6 @@ export const setAttribute = async (
{ elementSelector: selector, name: attribute, attributeValue: value },
);

export const removeAttribute = async (
page: Page,
selector: string,
attribute: string,
): Promise<void> => page.evaluate(
({ elementSelector, name }) => {
document.querySelector(elementSelector)?.removeAttribute(name);
},
{ elementSelector: selector, name: attribute },
);

export const appendAttributeValue = async (
page: Page,
selector: string,
Expand Down Expand Up @@ -164,9 +153,3 @@ export const addFocusableElementBefore = async (
},
{ selector: targetSelector, id: elementId },
);

export const waitForNextFrame = async (page: Page): Promise<void> => page.evaluate(
() => new Promise<void>((resolve) => {
requestAnimationFrame(() => requestAnimationFrame(() => resolve()));
}),
);
52 changes: 39 additions & 13 deletions e2e/playwright/helpers/screenshots.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,9 @@ import { changeTheme, getCurrentTheme } from './testPageUtils';

export interface ScreenshotOptions {
element?: Locator | string | null;
// Shoots the scroll size of the element instead of its box, the way the TestCafe element shot
// did: what overflows the element, such as a popup rendered into it, stays in the picture.
withOverflow?: boolean;
theme?: string;
shouldTestInCompact?: boolean;
compactCallBack?: () => Promise<unknown>;
Expand Down Expand Up @@ -48,6 +51,7 @@ const viewportClip = async (page: Page): Promise<{
const clippedToViewport = async (
page: Page,
locator: Locator,
withOverflow: boolean,
): Promise<{ x: number; y: number; width: number; height: number } | null> => {
const box = await locator.boundingBox();

Expand All @@ -56,17 +60,27 @@ const clippedToViewport = async (
}

const viewport = await viewportClip(page);

if (box.width <= viewport.width && box.height <= viewport.height) {
const scrollSize = withOverflow
? await locator.evaluate((element) => ({
width: element.clientLeft + element.scrollWidth,
height: element.clientTop + element.scrollHeight,
}))
: { width: 0, height: 0 };
const fitsViewport = Math.max(box.width, scrollSize.width) <= viewport.width
&& Math.max(box.height, scrollSize.height) <= viewport.height;

if (fitsViewport && !withOverflow) {
return null;
}

// The clip has to stay inside the viewport: a full-page capture widens the viewport to the whole
// document, the scrollbar goes away, and a widget that fills its container re-lays out to the
// wider space — the shot would show a layout the page never had.
await locator.evaluate((element) => {
element.scrollIntoView({ block: 'start', inline: 'start' });
});
if (!fitsViewport) {
// The clip has to stay inside the viewport: a full-page capture widens the viewport to the
// whole document, the scrollbar goes away, and a widget that fills its container re-lays out
// to the wider space — the shot would show a layout the page never had.
await locator.evaluate((element) => {
element.scrollIntoView({ block: 'start', inline: 'start' });
});
}

const scrolled = await locator.boundingBox() ?? box;
const x = Math.max(scrolled.x, 0);
Expand All @@ -75,22 +89,23 @@ const clippedToViewport = async (
return {
x,
y,
width: Math.min(scrolled.width, viewport.width - x),
height: Math.min(scrolled.height, viewport.height - y),
width: Math.min(Math.max(scrolled.width, scrollSize.width), viewport.width - x),
height: Math.min(Math.max(scrolled.height, scrollSize.height), viewport.height - y),
};
};

const expectScreenshot = async (
page: Page,
locator: Locator | null,
name: string,
withOverflow: boolean,
): Promise<void> => {
if (!locator) {
await expect(page).toHaveScreenshot([name]);
return;
}

const clip = await clippedToViewport(page, locator);
const clip = await clippedToViewport(page, locator, withOverflow);

if (clip) {
await expect(page).toHaveScreenshot([name], { clip });
Expand All @@ -107,6 +122,7 @@ export async function testScreenshot(
): Promise<void> {
const {
element,
withOverflow = false,
theme,
shouldTestInCompact = false,
compactCallBack,
Expand All @@ -123,7 +139,12 @@ export async function testScreenshot(
// No element means the whole viewport, the way the TestCafe comparer read a missing element.
const target = resolveLocator(page, element);

await expectScreenshot(page, target, getScreenshotName(screenshotName, theme ?? activeTheme));
await expectScreenshot(
page,
target,
getScreenshotName(screenshotName, theme ?? activeTheme),
withOverflow,
);

if (shouldTestInCompact) {
// The theme of a "- compact" job already ends with the suffix; appending it twice would ask
Expand All @@ -133,7 +154,12 @@ export async function testScreenshot(
await changeTheme(page, compactTheme);
await compactCallBack?.();

await expectScreenshot(page, target, getScreenshotName(screenshotName, compactTheme));
await expectScreenshot(
page,
target,
getScreenshotName(screenshotName, compactTheme),
withOverflow,
);
}

if (theme || shouldTestInCompact) {
Expand Down
123 changes: 123 additions & 0 deletions e2e/playwright/models/calendar/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,123 @@
import type { Locator } from '@playwright/test';
import type { WidgetName } from '../types';
import Button from '../button';
import Widget from '../internal/widget';
import CalendarView from './view';

const CLASS = {
widget: 'dx-widget',
calendarViewsWrapper: 'dx-calendar-views-wrapper',
todayButton: 'dx-calendar-today-button',
otherMonth: 'dx-calendar-other-month',
cellInRange: 'dx-calendar-cell-in-range',
cellInRangeStart: 'dx-calendar-range-start-date',
cellInRangeEnd: 'dx-calendar-range-end-date',
cellInHoveredRange: 'dx-calendar-cell-range-hover',
cellInHoveredRangeStart: 'dx-calendar-cell-range-hover-start',
cellInHoveredRangeEnd: 'dx-calendar-cell-range-hover-end',
navigatorNextView: 'dx-calendar-navigator-next-view',
navigatorPrevView: 'dx-calendar-navigator-previous-view',
navigatorCaption: 'dx-calendar-caption-button',
};

export default class Calendar extends Widget {
// eslint-disable-next-line class-methods-use-this
public getName(): WidgetName { return 'dxCalendar'; }

public getSelectedRangeCells(): Locator {
return this.element.locator(`.${CLASS.cellInRange}`);
}

public getSelectedRangeStartCell(): Locator {
return this.element.locator(`.${CLASS.cellInRangeStart}:not(.${CLASS.otherMonth})`);
}

public getSelectedRangeEndCell(): Locator {
return this.element.locator(`.${CLASS.cellInRangeEnd}`);
}

public getHoveredRangeCells(): Locator {
return this.element.locator(`.${CLASS.cellInHoveredRange}`);
}

public getHoveredRangeStartCell(): Locator {
return this.element.locator(`.${CLASS.cellInHoveredRangeStart}`);
}

public getHoveredRangeEndCell(): Locator {
return this.element.locator(`.${CLASS.cellInHoveredRangeEnd}`);
}

public getView(): CalendarView {
return new CalendarView(
this.page,
this.element.locator(`.${CLASS.calendarViewsWrapper} .${CLASS.widget}`).first(),
);
}

public getNavigatorNextButton(): Button {
return new Button(this.page, this.element.locator(`.${CLASS.navigatorNextView}`));
}

public getNavigatorPrevButton(): Button {
return new Button(this.page, this.element.locator(`.${CLASS.navigatorPrevView}`));
}

public getNavigatorCaption(): Button {
return new Button(this.page, this.element.locator(`.${CLASS.navigatorCaption}`));
}

public getTodayButton(): Button {
return new Button(this.page, this.element.locator(`.${CLASS.todayButton}`));
}

public getCellByDate(date: string): Locator {
return this.element.locator(`*[data-value="${date}"]:not(.${CLASS.otherMonth})`);
}

public async showGestureCover(): Promise<void> {
await this.element.evaluate((element) => {
const $element = $(element);
const { left = 0 } = $element.offset() ?? {};

$element.trigger($.Event('dxpointerdown', {
pageX: left,
pointers: [{ pointerId: 1 }],
}));

$element.trigger($.Event('dxpointermove', {
pageX: left + 20,
pointers: [{ pointerId: 1 }],
}));

$element.trigger($.Event('mouseup', {
pointers: [{ pointerId: 1 }],
}));
});
}

public async swipeStart(): Promise<void> {
await this.element.evaluate((element) => {
$(element).trigger($.Event('dxswipestart', {
pointers: [{ pointerId: 1 }],
}));
});
}

public async swipe(offset: number): Promise<void> {
await this.element.evaluate((element, swipeOffset) => {
$(element).trigger($.Event('dxswipe', {
offset: swipeOffset,
pointers: [{ pointerId: 1 }],
}));
}, offset);
}

public async swipeEnd(): Promise<void> {
await this.element.evaluate((element) => {
$(element).trigger($.Event('dxswipeend', {
pointers: [{ pointerId: 1 }],
}));
});
}
}
Loading
Loading