Skip to content

Add automated browser-based tests to check examples on web editor for console errors#85

Open
nbogie wants to merge 7 commits into
processing:mainfrom
nbogie:nb-add-browser-based-tests
Open

Add automated browser-based tests to check examples on web editor for console errors#85
nbogie wants to merge 7 commits into
processing:mainfrom
nbogie:nb-add-browser-based-tests

Conversation

@nbogie
Copy link
Copy Markdown

@nbogie nbogie commented May 31, 2026

Addresses #84

What is this?

Adds playwright-based config and tests to check the p5.sound examples in place on the web editor, checking they load and don't have any errors in the console.

How to run the tests (with test-runner UI)

This is recommended as it allows easier exploration of test failures:

  1. once, to install playwright
npm install
  1. launch the test runner's UI
npm run test:integration:ui
playwright-ui-start-tests
  1. in the gui that opens, click "projects" and ensure "firefox" and "chromium" are checked. (see image)

  2. Also in the gui, press the play button on the top test or above it. (see image)

Alternative: How to run the tests (unattended):

npm run test:integration 

This will present a html report when it finishes.

What are the additions and changes?

  • Adds playwright as a dev dependency in package.json
    • (playwright also downloads browsers (chromium and firefox) but will not do so until the user runs the tests for the first time)
  • Adds playwright.config.js, specifying, for example, where the tests are (./tests/) and which browsers to use.
  • Adds tests/integration/test-examples-on-web-editor.spec.js This is the main test file that instructs playwright what web pages to load , what to click, and what to check for.
  • Adds tests/integration/about-these-tests.md documenting the testing (and some problems faced in automated testing of the web editor).
  • Adds the playwright-generated .github/workflows/playwright.yml.DISABLED - this is untested.

Limitations:

  • It runs the tests against firefox and chromium but not safari (webkit) nor on any mobile browsers. Research is needed in how to have webkit granted microphone and webcam permissions and to allow audio context start without user interaction.
  • Because I've only written ONE standardised test to exercise ALL p5.sound examples, certain conditions have to be met for an example to be compatible:
    • Example must render a canvas.
    • Example must not require user permissions beyond microphone and camera.
  • No CI setup has been tested. It is envisaged this tool will be plenty useful when used on an ad-hoc basis. Certainly once that proves true it would be worth hooking up the CI. The playwright-generated workflow for github actions has been placed in the repo but disabled: .github/workflows/playwright.yml.DISABLED

AI usage disclosure

Claude Code did most of the implementation on this, including research into chrome and firefox support for pre-emptive grants of microphone + video, troubleshooting the main issue with testing the web editor, and the bulk of the write-up about-these-tests.md. That said, I've reworked the code to how I want it, I have read and understood each line, and the PR write-up is all mine.

@nbogie
Copy link
Copy Markdown
Author

nbogie commented May 31, 2026

update: silenced the headless firefox test run to match chrome.

@nbogie nbogie marked this pull request as ready for review June 3, 2026 10:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant