Skip to content

Add AI agents documentation page - #2846

Merged
diemol merged 2 commits into
trunkfrom
copse/selenium-documentation-for-llms-we-need-217896
Sep 28, 2026
Merged

diemol merged 2 commits into
trunkfrom
copse/selenium-documentation-for-llms-we-need-217896

Conversation

@AutomatedTester

@AutomatedTester AutomatedTester commented Sep 24, 2026 •

Copy link
Copy Markdown
Member

Description

Adds a new documentation page, Documentation → AI Agents (/documentation/ai_agents/), on getting AI coding agents and LLMs to write Selenium code that is current, correct, and not flaky. It sits in the sidebar between IDE and Test Practices (weight 11).

The page is structured as:

  1. Give the agent the current documentation — llms.txt first, then this site, the examples/ directory, the per-binding API docs, and the changelogs, with the instruction that matters most: if an API isn't in the current docs, it doesn't exist.
  2. Write the project rules down — a paste-ready block for AGENTS.md / CLAUDE.md / rules files, followed by a tabpane of the binding-specific removals models get wrong most often (Java Duration and findElementBy*, Python find_element_by_* / executable_path / desired_capabilities, C# AddAdditionalCapability, Ruby webdrivers, JavaScript unawaited promises, Kotlin).
  3. What to correct, and why — the reasoning behind each rule: Selenium Manager vs third-party driver managers, Options vs DesiredCapabilities, explicit waits vs sleeping (including the second-order failure where an agent "fixes" a flaky test by raising a timeout), locator durability, BiDi vs CDP, and Grid 4 subcommands vs Grid 3 -role hub.
  4. Let the agent drive a real browser — the throwaway-script approach first, since it needs no new tooling and exercises the same stack the real test will; then MCP servers as a category.
  5. Let the agent read the failure — pointing agents at our common errors pages and at logging, rather than at "the test failed".
  6. A workflow that works and Next steps.

Motivation and Context

A large share of Selenium code is now written with an agent's help, and it is wrong in a consistent, predictable set of ways. The cause is the training data: models learned Selenium from more than a decade of blog posts, forum answers, and tutorials describing Selenium 2 and 3. Those APIs were removed years ago, and the most-repeated patterns in that material — sleeping to wait for the page, downloading driver binaries by hand, copying XPath out of DevTools — were never good practice even when they compiled.

We already document all of the correct answers, but they are spread across the upgrade guide, the waits page, Selenium Manager, locators, and BiDi. This page collects what an agent specifically gets wrong into one place, and gives users something they can paste into a rules file today.

On llms.txt

The second commit links llms.txt as the first entry in the documentation table and adds it to the sample rules block, along with a rule against using the legacy and CDP pages as a basis for new code. The page explains that the index is curated rather than exhaustive, and why those two sets of pages are deliberately excluded from it.

No change to layouts/index.llms.txt was needed: the curated index picks up new pages under /documentation automatically, so the new page is already listed under "Getting started" in the generated output, and the template's drift check reports nothing.

Notes for reviewers

A couple of judgment calls worth a second opinion:

  • No MCP server is named. Several community Selenium MCP servers exist and none is official, so the page describes the category and says to evaluate them as you would any dependency, rather than linking one from selenium.dev. Happy to name one if the project wants to.
  • Version-specific claims. The sample rules block pins 4.49.0 to match examples/. The per-binding removal claims are worth an eye — particularly Ruby's :desired_capabilities and Python's options.headless.
  • English only for now, per the translation process in the style guide.

Types of changes

  • Change to the site (I have double-checked the Netlify deployment, and my changes look good)
  • Code example added (and I also added the example to all translated languages)
  • Improved translation
  • Added new translation (and I also added a notice to each document missing translation)

Checklist

  • I have read the contributing document.
  • I have used hugo to render the site/docs locally and I am sure it works.

Verified locally with a full hugo --minify build — clean, no warnings, every cross-reference resolves and the llms.txt drift check is silent — and visually in hugo server: the tabpane, the table, the cross-links, and the sidebar ordering all render correctly.

@netlify

netlify Bot commented Sep 24, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for selenium-dev ready!

Name Link
🔨 Latest commit 68d5b3c
🔍 Latest deploy log https://app.netlify.com/projects/selenium-dev/deploys/6aba43545717ec00084ef910
😎 Deploy Preview https://deploy-preview-2846--selenium-dev.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
🤖 Make changes Run an agent on this branch

To edit notification comments on pull requests, go to your Netlify project configuration.

@diemol diemol left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be a blog post? Maybe I don't see it the same way.

@AutomatedTester

Copy link
Copy Markdown
Member Author

Fair question, and I went back and forth on it.

The main argument for docs is that the whole point of this page is to be found by an agent reading our documentation. llms.txt only lists the blog under Optional — "release announcements and technical deep dives" — so as a post it is much less likely to end up in front of the thing it is trying to correct. As a docs page the curated index picks it up automatically; it is already listed under "Getting started" in the generated output.

The other pull is maintenance. This is probably the most time-sensitive content on the site, because the specific things models get wrong will shift as they improve. Docs get reviewed, translated and updated as part of the normal process. A post dated 2026 stays frozen and keeps ranking.

And a fair amount of the page is really just routing — waits, Selenium Manager, locators, BiDi, the upgrade guide. It is mostly a navigation layer over things we already document, which feels like a docs job.

That said, I don't think it has to be either/or, and the blog is much better at attention than the docs are. Happy to write a post announcing it and pointing at the page — blog for reach, docs for durability. Want me to do that as a follow-up?

@AutomatedTester

Copy link
Copy Markdown
Member Author

Also, I got the idea from wdio, they are adding this as a separate page.

AutomatedTester and others added 2 commits September 28, 2026 11:36
Adds a documentation page on getting coding agents and LLMs to write
current, correct Selenium code.

LLM training data for browser automation is dominated by Selenium 2 and
3 era material, so agents reliably emit removed APIs and patterns that
were never good practice. The page gives readers three things: where to
point an agent for current documentation, a paste-ready rules block for
AGENTS.md/CLAUDE.md with per-binding tabs, and the reasoning behind each
rule so bad output is recognisable in review.

Covers driver management vs Selenium Manager, Options vs
DesiredCapabilities, explicit waits vs sleeping, locator durability,
BiDi vs CDP, and Grid 4 vs Grid 3 invocations. Also covers letting an
agent drive a real browser to verify locators, and pointing it at the
common errors and logging pages when a test fails.

Sits in the sidebar between IDE and Test Practices.

Co-Authored-By: Copse <noreply@copse.dev>
Copse-Models: acp:claude-agent-acp#opus[1m]
The site already renders a curated llms.txt from layouts/index.llms.txt,
which is exactly the entry point this page should hand to an agent. Adds
it as the first row of the documentation table, explains that it is
curated rather than exhaustive, and notes why the legacy and CDP pages
are deliberately kept out of the main index.

Also adds it to the sample rules block, along with a rule against using
the legacy and CDP pages as a basis for new code.

The page needed no change to the llms.txt template: its curated index
picks up new pages under /documentation automatically, and the drift
check reports nothing.

Co-Authored-By: Copse <noreply@copse.dev>
Copse-Models: acp:claude-agent-acp#opus[1m]
@AutomatedTester
AutomatedTester force-pushed the copse/selenium-documentation-for-llms-we-need-217896 branch from a51ae7b to 68d5b3c Compare September 28, 2026 10:37
@diemol
diemol merged commit 51b6d28 into trunk Sep 28, 2026
6 checks passed
@diemol
diemol deleted the copse/selenium-documentation-for-llms-we-need-217896 branch September 28, 2026 13:43
selenium-ci added a commit that referenced this pull request Sep 28, 2026
* Add AI agents documentation page

Adds a documentation page on getting coding agents and LLMs to write
current, correct Selenium code.

LLM training data for browser automation is dominated by Selenium 2 and
3 era material, so agents reliably emit removed APIs and patterns that
were never good practice. The page gives readers three things: where to
point an agent for current documentation, a paste-ready rules block for
AGENTS.md/CLAUDE.md with per-binding tabs, and the reasoning behind each
rule so bad output is recognisable in review.

Covers driver management vs Selenium Manager, Options vs
DesiredCapabilities, explicit waits vs sleeping, locator durability,
BiDi vs CDP, and Grid 4 vs Grid 3 invocations. Also covers letting an
agent drive a real browser to verify locators, and pointing it at the
common errors and logging pages when a test fails.

Sits in the sidebar between IDE and Test Practices.

Co-Authored-By: Copse <noreply@copse.dev>
Copse-Models: acp:claude-agent-acp#opus[1m]

* Link llms.txt from the AI agents page

The site already renders a curated llms.txt from layouts/index.llms.txt,
which is exactly the entry point this page should hand to an agent. Adds
it as the first row of the documentation table, explains that it is
curated rather than exhaustive, and notes why the legacy and CDP pages
are deliberately kept out of the main index.

Also adds it to the sample rules block, along with a rule against using
the legacy and CDP pages as a basis for new code.

The page needed no change to the llms.txt template: its curated index
picks up new pages under /documentation automatically, and the drift
check reports nothing.

Co-Authored-By: Copse <noreply@copse.dev>
Copse-Models: acp:claude-agent-acp#opus[1m]

---------

Co-authored-by: Copse <noreply@copse.dev>

[deploy site] 51b6d28
diemol pushed a commit that referenced this pull request Sep 29, 2026
Announces the new AI agents documentation page added in #2846.

The post takes the angle that agents are not hallucinating when they emit
Selenium 3 code - they were taught by a decade of blog posts and tutorials,
a good share of which this community wrote, and which still outnumber
everything published since. From there it pulls out the three things worth
acting on: point the agent at llms.txt, write the project rules into
AGENTS.md/CLAUDE.md, and let it drive a real browser so locators are
verified rather than inferred.

Singles out one pattern for review: agents "fixing" a flaky test by raising
a timeout or adding a sleep, which hides the race instead of removing it.

Links to the documentation page rather than restating it, so the post stays
short and the docs remain the reference.


Copse-Models: acp:claude-agent-acp#opus[1m]

Co-authored-by: Copse <noreply@copse.dev>

[deploy site]
selenium-ci added a commit that referenced this pull request Sep 29, 2026
Announces the new AI agents documentation page added in #2846.

The post takes the angle that agents are not hallucinating when they emit
Selenium 3 code - they were taught by a decade of blog posts and tutorials,
a good share of which this community wrote, and which still outnumber
everything published since. From there it pulls out the three things worth
acting on: point the agent at llms.txt, write the project rules into
AGENTS.md/CLAUDE.md, and let it drive a real browser so locators are
verified rather than inferred.

Singles out one pattern for review: agents "fixing" a flaky test by raising
a timeout or adding a sleep, which hides the race instead of removing it.

Links to the documentation page rather than restating it, so the post stays
short and the docs remain the reference.

Copse-Models: acp:claude-agent-acp#opus[1m]

Co-authored-by: Copse <noreply@copse.dev>

[deploy site] 94c7cbd
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants