Support stagehand 4.x

[vdusek]

The stagehand integration targets the 3.x API and is capped at stagehand>=3.19.5,<4.0.0 (#2153). Version 4.0.0, published on 2026-08-10, is a breaking redesign, so the cap has to stay until the integration is ported.

What changed upstream

• AsyncStagehand and AsyncSession are gone. The public surface is now a single Stagehand, plus Page, Locator, BrowserContext, StagehandBrowser, and friends, so the separate sync/async clients appear to have been collapsed.
• The whole stagehand.types.session_* module tree was removed. All of session_act_params, session_act_response, session_execute_params, session_execute_response, session_extract_params, session_extract_response, session_observe_params and session_observe_response raise ModuleNotFoundError on 4.0.0.

What needs porting

• src/crawlee/browsers/_stagehand_browser_plugin.py - top-level from stagehand import AsyncStagehand, plus the client construction and the init params.
• src/crawlee/browsers/_stagehand_browser_controller.py - typed against AsyncStagehand and AsyncSession.
• src/crawlee/browsers/_stagehand_types.py - AsyncSession and the eight stagehand.types.session_* imports; this is the bulk of the work, since StagehandOptions and StagehandPage are built on those request/response models.

Once ported, StagehandCrawler and _stagehand_crawling_context should follow without changes, since they only touch our own wrappers.

Notes

• The upper bound needs to be widened in the same change, and the stagehand extra in pyproject.toml is the only place it is declared.
• Whether 4.x can be supported alongside 3.x is worth deciding early. Given that the session type modules are gone rather than renamed, a clean break to 4.x only looks more realistic than a compatibility shim.
• Our unit tests mock the stagehand client, so they will keep passing across the port and will not catch surface changes on their own. The templates E2E job is what actually resolves the dependency fresh and exercises the real API.

✍️ Drafted by Claude Code