From 5a18e738093e8482b8685d11eb78d45da24e3e3a Mon Sep 17 00:00:00 2001 From: AutomatedTester Date: Thu, 24 Sep 2026 12:39:36 +0100 Subject: [PATCH 1/2] Add AI agents documentation page Adds a documentation page on getting coding agents and LLMs to write current, correct Selenium code. LLM training data for browser automation is dominated by Selenium 2 and 3 era material, so agents reliably emit removed APIs and patterns that were never good practice. The page gives readers three things: where to point an agent for current documentation, a paste-ready rules block for AGENTS.md/CLAUDE.md with per-binding tabs, and the reasoning behind each rule so bad output is recognisable in review. Covers driver management vs Selenium Manager, Options vs DesiredCapabilities, explicit waits vs sleeping, locator durability, BiDi vs CDP, and Grid 4 vs Grid 3 invocations. Also covers letting an agent drive a real browser to verify locators, and pointing it at the common errors and logging pages when a test fails. Sits in the sidebar between IDE and Test Practices. Co-Authored-By: Copse Copse-Models: acp:claude-agent-acp#opus[1m] --- .../content/documentation/ai_agents.en.md | 307 ++++++++++++++++++ 1 file changed, 307 insertions(+) create mode 100644 website_and_docs/content/documentation/ai_agents.en.md diff --git a/website_and_docs/content/documentation/ai_agents.en.md b/website_and_docs/content/documentation/ai_agents.en.md new file mode 100644 index 00000000000..a772d7b3694 --- /dev/null +++ b/website_and_docs/content/documentation/ai_agents.en.md @@ -0,0 +1,307 @@ +--- +title: "Using AI coding agents with Selenium" +linkTitle: "AI Agents" +weight: 11 +description: >- + How to get coding agents and large language models to write Selenium code that is current, + correct, and not flaky. +--- + +A large share of Selenium code is now written with the help of a coding agent, +and much of it is wrong in the same handful of ways. + +The reason is the training data. +Models learned Selenium from more than a decade of blog posts, forum answers, and tutorials, +and most of that material describes Selenium 2 and Selenium 3. +Those APIs were removed years ago. +Worse, some of the most-repeated patterns in that material — +sleeping to wait for the page, +downloading driver binaries by hand, +copying XPath out of DevTools — +were never good practice even when they compiled. +An agent will reproduce all of it, confidently, and with a plausible explanation attached. + +None of this is a reason to keep agents away from your test suite. +It is a reason to give an agent what you would give a new team member: +the version you are on, +the place the current documentation lives, +and the conventions the project expects. +This page shows how to do that, and what specifically to correct. + +## Give the agent the current documentation + +An agent that can read the current docs stops guessing. +Point it at these, in this order: + +| Resource | Use it for | +| --- | --- | +| [This documentation](https://www.selenium.dev/documentation/) | The current behaviour of every feature, in every binding. | +| [The examples directory](https://github.com/SeleniumHQ/seleniumhq.github.io/tree/trunk/examples) | Real, runnable code. | +| API docs for [Java](https://seleniumhq.github.io/selenium/docs/api/java/index.html), [Python](https://seleniumhq.github.io/selenium/docs/api/py/index.html), [.NET](https://seleniumhq.github.io/selenium/docs/api/dotnet/index.html), [Ruby](https://seleniumhq.github.io/selenium/docs/api/rb/index.html), and [JavaScript](https://seleniumhq.github.io/selenium/docs/api/javascript/index.html) | Checking whether a method actually exists, and what it takes. | +| [The Selenium changelogs](https://github.com/SeleniumHQ/selenium/tree/trunk/java) | What changed between your version and whatever the model remembers. | + +The examples directory deserves particular attention. +Every code tab on this site is a link into that repository, +and the code in it is executed against current Selenium releases. +It is the single most reliable body of Selenium example code to point an agent at, +because unlike a blog post it cannot silently rot. + +A useful instruction to give an agent is a negative one: +if an API cannot be found in the current documentation or API reference, it does not exist, +however familiar it looks. + +## Write the project rules down + +Agents follow conventions far more reliably when the conventions are written in a file +rather than repeated in chat. +Add something like the following to your project's `AGENTS.md`, `CLAUDE.md`, or rules file, +and adjust the version and commands to match your project. + +```markdown +## Browser automation + +This project uses Selenium 4.49.0. + +### Docs +- Reference: https://www.selenium.dev/documentation/ +- Examples: https://github.com/SeleniumHQ/seleniumhq.github.io/tree/trunk/examples +- Fetch the relevant page before using an API you are not certain about. +- Do not use APIs from Selenium 3 or earlier. If an API is not in the + current documentation or API reference, it does not exist. + +### Drivers and browsers +- Selenium Manager downloads and caches the drivers. Do not add a + driver-manager dependency, do not download drivers, and do not set + a path to a driver binary. +- Browser configuration goes in that browser's Options class. Do not + use DesiredCapabilities and do not pass raw capability maps. +- Headless is a browser argument: `--headless=new`. + +### Waiting +- Never sleep in a test. +- Use an explicit wait, and wait for the condition the next line + actually depends on. +- Do not set an implicit wait and also use explicit waits in the + same session. +- When a test is flaky, find the condition that was not yet true. + Do not increase a timeout. + +### Locators +- Prefer `id` and `name`, then a CSS selector on a stable attribute + such as `data-test`. +- Do not write absolute XPath and do not use generated class names. +- Declare locators separately from the code that finds the element. + +### Sessions +- One fresh session per test. Always quit the driver in teardown - + `quit`, not `close`. +- Do not share a driver between tests or hold one in a global. + +### Events and network +- Use WebDriver BiDi for console logs, JavaScript errors, and network + interception. Do not use the Chrome DevTools Protocol. + +### Running tests +- All tests: +- A single test: +- If you are unsure whether a locator or a flow works, write a + throwaway script and run it against the application instead of + guessing. +``` + +Then add the rules for your binding. +These are the removals and replacements that models get wrong most often. + +{{< tabpane text=true >}} +{{% tab header="Java" %}} + +* Timeouts and waits take a `java.time.Duration`, not a `(long, TimeUnit)` pair. +* The `findElementBy*` and `findElementsBy*` helpers were removed. + Use `driver.findElement(By.id("..."))`. +* `DesiredCapabilities` is replaced by `ChromeOptions`, `FirefoxOptions`, and so on. +* `merge` returns a new object rather than mutating the caller; assign the result. +* `setHeadless(true)` was removed. Use `addArguments("--headless=new")`. +* Use the `Browser` enum rather than the older `BrowserType` interface. +* Do not add WebDriverManager. Selenium Manager is built in. +{{% /tab %}} +{{% tab header="Python" %}} + +* `find_element_by_id` and every other `find_element_by_*` method was removed in 4.3. + Use `driver.find_element(By.ID, "...")`. +* The `executable_path` and `desired_capabilities` keyword arguments were removed in 4.10. + Pass `service=` and `options=` instead. +* `options.headless = True` was removed. Use `options.add_argument("--headless=new")`. +* `driver.switch_to_alert()` was removed. Use `driver.switch_to.alert`. +* Do not install `webdriver-manager` or call `ChromeDriverManager().install()`. +{{% /tab %}} +{{% tab header="CSharp" %}} + +* `AddAdditionalCapability` is replaced by `AddAdditionalOption`. +* `DesiredCapabilities` is replaced by `ChromeOptions`, `FirefoxOptions`, and so on. +* Do not pass a driver directory to the `ChromeDriver` constructor. +* `Close()` closes one window; `Quit()` ends the session. Teardown needs `Quit()`. +{{% /tab %}} +{{% tab header="Ruby" %}} + +* `:desired_capabilities` was removed. Pass `options:` instead. +* Setting `Selenium::WebDriver::Chrome.driver_path=` is unnecessary. +* Headless is an argument: `options.add_argument('--headless=new')`. +* Do not add the `webdrivers` gem. Selenium Manager is built in. +{{% /tab %}} +{{% tab header="JavaScript" %}} + +* Every command returns a promise. Await all of them, including `driver.quit()`. +* Build the driver with `new Builder().forBrowser(Browser.CHROME).build()`. +* Explicit waits are `driver.wait(until.elementLocated(...), timeout)`. +* The `webdriver-manager` npm package is a Protractor-era tool and is unrelated to Selenium. + Do not install it. +{{% /tab %}} +{{% tab header="Kotlin" %}} + +* Kotlin uses the Java bindings, so every Java rule applies. +* Timeouts and waits take a `java.time.Duration`. +* The `findElementBy*` helpers were removed. Use `driver.findElement(By.id("..."))`. +* `DesiredCapabilities` is replaced by `ChromeOptions`, `FirefoxOptions`, and so on. +* `setHeadless(true)` was removed. Use `addArguments("--headless=new")`. +{{% /tab %}} +{{< /tabpane >}} + +## What to correct, and why + +The rules above are short on purpose. +This section is the reasoning behind them, +and it is worth reading so you can recognise the bad output when you see it. + +### Nobody downloads drivers any more + +The most common single artefact of stale training data is driver management: +a third-party driver manager, a hard-coded path to `chromedriver`, or a `PATH` setup step. +Since 4.6, [Selenium Manager]({{< ref "/documentation/selenium_manager.md" >}}) discovers the installed browser, +resolves the matching driver, downloads it, and caches it — +and since 4.11 it will download the browser too, if there isn't one. +Starting a session takes no path and no extra dependency. + +When an agent adds a driver manager, it isn't adding redundancy. +It is adding a second, slower, less reliable source of truth for which driver version to use. + +### Configuration lives in Options + +`DesiredCapabilities` and free-form capability maps predate the W3C WebDriver standard. +Non-standard capabilities now have to be vendor-prefixed, and unprefixed ones can fail +the session outright. +The [browser Options classes]({{< ref "/documentation/webdriver/drivers/options.md" >}}) produce a valid +payload by construction, which is why they are the only supported route. +The [upgrade guide]({{< ref "/documentation/webdriver/troubleshooting/upgrade_to_selenium_4.md" >}}) +has before-and-after code for every binding. + +### Sleeping is not waiting + +Sleeping is the most-repeated pattern in the training data and the most damaging. +A fixed sleep is either too short, and the test fails, or too long, and the suite crawls. +It is the main cause of flaky Selenium tests. + +Use an [explicit wait]({{< ref "/documentation/webdriver/waits.md" >}}) for the condition that the next line +depends on — element clickable, text present, spinner gone — not for a duration. +And do not mix implicit and explicit waits in the same session: +as the waits documentation explains, +the two combine unpredictably, +so a 10-second implicit wait and a 15-second explicit wait can time out after 20. + +Watch for the second-order version of this mistake: +when a test fails intermittently, +an agent will very often "fix" it by raising a timeout or adding a sleep. +That hides the race rather than removing it. +Insist that it identify which condition was not yet true. + +### Locators have to survive a deploy + +Agents are fond of absolute XPath and generated class names, +partly because so much tutorial code contains them. +Both break on the next front-end change. +Prefer `id` and `name`, then CSS on a stable attribute. +See [Locator strategies]({{< ref "/documentation/webdriver/elements/locators.md" >}}) and the +[locator suggestions]({{< ref "/documentation/test_practices/encouraged/locators.md" >}}) +in our encouraged test practices. + +This is also the place where an agent is most likely to invent something. +It cannot see your application, so unless it has actually loaded the page it is pattern-matching +on what a login form usually looks like. +Giving it a way to check — see below — matters more here than anywhere else. + +### BiDi, not CDP + +For console logs, JavaScript errors, and network interception, +models reach for the Chrome DevTools Protocol, because that is what the last few years of +examples used. +CDP is Chromium-only and has no stable API between browser versions. +[WebDriver BiDi]({{< ref "/documentation/webdriver/bidi/" >}}) is the W3C standard that replaces it, +and it works across browsers. +CDP support in Selenium exists as a stopgap, not as the recommended path. + +### Grid 4 is not Grid 3 + +Agents frequently produce Grid 3 invocations — +`-role hub`, `-role node`, and a JSON node config file. +Grid 4 uses subcommands: +`java -jar selenium-server-.jar standalone`, or `hub` and `node`. +See [Grid getting started]({{< ref "/documentation/grid/getting_started.md" >}}). + +## Let the agent drive a real browser + +An agent that can only write code is guessing about your application. +An agent that can open it can check. + +The lightest-weight option needs nothing new: +tell the agent it may write a throwaway Selenium script, run it, and print what it finds. +This is genuinely effective — +it can navigate to the page, dump the candidate elements, try a locator, and report back — +and it exercises the same stack the real test will use, +which no external tool can claim. + +There are also community-maintained Model Context Protocol servers that expose Selenium as a set +of tools an agent can call directly, so it can open a browser, click, type, and take screenshots +inside a conversation. +These are third-party projects rather than part of the Selenium project, +so evaluate them as you would any other dependency. + +Either way, the rule to write down is the same: +verify locators against the running application instead of inferring them. + +## Let the agent read the failure + +Selenium's exceptions are specific, and an agent that is handed the real exception — +rather than "the test failed" — will usually fix the right thing. + +* Our [common errors]({{< ref "/documentation/webdriver/troubleshooting/errors/" >}}) pages + describe the likely cause and the possible solutions for + `NoSuchElementException`, `StaleElementReferenceException`, + `ElementClickInterceptedException`, `SessionNotCreatedException`, and the rest. + Pointing an agent at the page for the exception it just hit is a short path to a correct fix. +* [Logging]({{< ref "/documentation/webdriver/troubleshooting/logging.md" >}}) can be turned up so the agent can + see the actual commands sent to the driver rather than reasoning about what it thinks happened. +* A screenshot taken at the moment of failure tells an agent more about an + `ElementClickInterceptedException` — a cookie banner, an overlay — than any stack trace. + +## A workflow that works + +1. Have the agent open the feature under test and propose locators, either with a throwaway + script or an MCP server. Review the locators before any test is written. +2. Let it write the test, following the project rules, fetching documentation pages as needed. +3. Have it run that one test, not the suite, and iterate until it passes. +4. Run the test a few times before you believe it. A test that passes once has not been shown + to be free of races. +5. Review the diff for the patterns above. Sleeps, driver paths, and absolute XPath are the ones + that survive review most often, because they work on the machine where they were written. + +## Next steps + +* [Getting started]({{< ref "/documentation/webdriver/getting_started/" >}}) — + what a first script looks like in each binding. +* [Upgrade to Selenium 4]({{< ref "/documentation/webdriver/troubleshooting/upgrade_to_selenium_4.md" >}}) — + the before-and-after reference for almost everything an agent gets wrong. +* [Waiting strategies]({{< ref "/documentation/webdriver/waits.md" >}}) — + why sleeping fails and what to do instead. +* [Encouraged test practices]({{< ref "/documentation/test_practices/encouraged/" >}}) — + the conventions worth putting in your rules file. +* [Selenium Manager]({{< ref "/documentation/selenium_manager.md" >}}) — + what replaced driver management. From 68d5b3c45facffd32c5eda2c76fbc7a3f6288e03 Mon Sep 17 00:00:00 2001 From: AutomatedTester Date: Thu, 24 Sep 2026 12:45:28 +0100 Subject: [PATCH 2/2] Link llms.txt from the AI agents page The site already renders a curated llms.txt from layouts/index.llms.txt, which is exactly the entry point this page should hand to an agent. Adds it as the first row of the documentation table, explains that it is curated rather than exhaustive, and notes why the legacy and CDP pages are deliberately kept out of the main index. Also adds it to the sample rules block, along with a rule against using the legacy and CDP pages as a basis for new code. The page needed no change to the llms.txt template: its curated index picks up new pages under /documentation automatically, and the drift check reports nothing. Co-Authored-By: Copse Copse-Models: acp:claude-agent-acp#opus[1m] --- .../content/documentation/ai_agents.en.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/website_and_docs/content/documentation/ai_agents.en.md b/website_and_docs/content/documentation/ai_agents.en.md index a772d7b3694..556f4f80696 100644 --- a/website_and_docs/content/documentation/ai_agents.en.md +++ b/website_and_docs/content/documentation/ai_agents.en.md @@ -35,11 +35,22 @@ Point it at these, in this order: | Resource | Use it for | | --- | --- | +| [llms.txt](https://www.selenium.dev/llms.txt) | A curated index of this site, for an agent to orient itself. Start here. | | [This documentation](https://www.selenium.dev/documentation/) | The current behaviour of every feature, in every binding. | | [The examples directory](https://github.com/SeleniumHQ/seleniumhq.github.io/tree/trunk/examples) | Real, runnable code. | | API docs for [Java](https://seleniumhq.github.io/selenium/docs/api/java/index.html), [Python](https://seleniumhq.github.io/selenium/docs/api/py/index.html), [.NET](https://seleniumhq.github.io/selenium/docs/api/dotnet/index.html), [Ruby](https://seleniumhq.github.io/selenium/docs/api/rb/index.html), and [JavaScript](https://seleniumhq.github.io/selenium/docs/api/javascript/index.html) | Checking whether a method actually exists, and what it takes. | | [The Selenium changelogs](https://github.com/SeleniumHQ/selenium/tree/trunk/java) | What changed between your version and whatever the model remembers. | +[`llms.txt`](https://www.selenium.dev/llms.txt) follows the +[llmstxt.org](https://llmstxt.org) convention: +a single plain-text file listing the documentation in a sensible reading order, +so an agent can find the right page without crawling the site or guessing at URLs. +It is worth pointing at explicitly, because it is curated rather than exhaustive. +The Selenium 2 and 3 era [legacy documentation]({{< ref "/documentation/legacy/" >}}) +and the [CDP pages]({{< ref "/documentation/webdriver/bidi/cdp/" >}}) are deliberately +kept out of the main index — +both are accurate, and both are the worst possible input for a model about to write new code. + The examples directory deserves particular attention. Every code tab on this site is a link into that repository, and the code in it is executed against current Selenium releases. @@ -63,9 +74,11 @@ and adjust the version and commands to match your project. This project uses Selenium 4.49.0. ### Docs +- Index: https://www.selenium.dev/llms.txt - Reference: https://www.selenium.dev/documentation/ - Examples: https://github.com/SeleniumHQ/seleniumhq.github.io/tree/trunk/examples - Fetch the relevant page before using an API you are not certain about. +- Do not use the legacy or CDP pages as a basis for new code. - Do not use APIs from Selenium 3 or earlier. If an API is not in the current documentation or API reference, it does not exist.