feat(integrations): Claude Agent SDK harness example - #2719
Merged
Conversation
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
Contributor
There was a problem hiding this comment.
All reported issues were addressed across 11 files
Architecture diagram
sequenceDiagram
participant User as User/CLI
participant Agent as Claude Agent SDK
participant MCP as Stagehand MCP Server
participant Facade as Stagehand Facade
participant Browser as Browserbase Browser
participant Env as Environment Filter
Note over User,Browser: NEW: Claude Code Facade Integration Flow
User->>Agent: Run instruction via query()
Agent->>Agent: Load FACADE_AGENT_INSTRUCTIONS as system prompt
Agent->>MCP: Spawn stdio server (mcpServers.stagehand)
Agent->>Env: Request env allowlist
Env->>Env: Filter to STAGEHAND_*/BROWSERBASE_* vars
Env-->>Agent: Allowlisted env (ANTHROPIC_API_KEY excluded)
Agent->>MCP: Pass allowlisted env to subprocess
Note over Agent: Tool permission enforcement
Agent->>Agent: Check canUseTool guard
alt Tool starts with "mcp__stagehand__"
Agent->>Agent: Allow tool execution
Agent->>MCP: Call mcp__stagehand__{run,snapshot,screenshot}
MCP->>Facade: Execute Stagehand operation
Facade->>Browser: Drive browser (via Browserbase)
Browser-->>Facade: Browser result
Facade-->>MCP: Tool result
MCP-->>Agent: MCP response
else Non-stagehand tool
Agent->>Agent: Deny with message "Only stagehand browser tools permitted"
end
Agent->>Agent: Stream agent messages
alt Success result
Agent-->>User: Print final result
else Error/Cancelled result
Agent-->>User: Throw error, set exit code 1
end
Note over User,MCP: Alternative: Claude Code CLI with .mcp.json
User->>MCP: Connect via project-scoped .mcp.json
Note over MCP: CLI inherits shell env (no ${VAR} expansion)
MCP-->>User: Tool access available
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
|
akeimach
approved these changes
Aug 14, 2026
Runnable project embedding a Claude agent via
@anthropic-ai/claude-agent-sdk with the stagehand-facade stdio server
mounted programmatically (env allowlist, allowedTools + canUseTool guard
for headless runs, FACADE_AGENT_INSTRUCTIONS as system prompt). Matches
the pattern established by the evals claude_code harness. Secondary
README section covers connecting a running Claude Code CLI via
.mcp.json (verified: CLI inherits shell env; ${VAR} is not expanded).
tsconfig checks src directly (include listed a nonexistent extensions dir), server path resolves inside main so failures surface through the error handler, and a stream ending without a result now exits nonzero.
miguelg719
force-pushed
the
miguel/harness-claude-code
branch
from
August 14, 2026 20:02
45ea444 to
129610f
Compare
miguelg719
added a commit
that referenced
this pull request
Aug 22, 2026
PRs #2719/#2720 added @anthropic-ai/claude-agent-sdk (0.3.224) and @openai/codex-sdk (0.147.0) to the pnpm catalog for the integration examples, forking the versions evals pinned directly (^0.2.141 and 0.125.0). Move both evals deps to catalog: so the monorepo tracks a single version per SDK. Compat verified against the installed types: every query() option the claude_code harness passes (canUseTool, systemPrompt preset+append, settingSources, mcpServers, createSdkMcpServer/tool) exists in 0.3.224, and every codex option (startThread config, runStreamed outputSchema/ signal, constructor config bag incl. show_raw_agent_reasoning) exists in 0.147.0 with unchanged ThreadEvent kinds. Evals unit suite passes (420 tests).
miguelg719
added a commit
that referenced
this pull request
Aug 30, 2026
…2743) ## What Moves both external-harness SDK deps in `packages/evals` onto the workspace catalog: - `@anthropic-ai/claude-agent-sdk`: `^0.2.141` → `catalog:` (**0.3.224**) - `@openai/codex-sdk`: `0.125.0` → `catalog:` (**0.147.0**) ## Why #2719 and #2720 added both SDKs to the catalog for the integration examples, forking the versions evals pinned directly — two copies of each SDK in the monorepo. This converges on one version per SDK, as a standalone pre-step to the broader harness-adapter consolidation. ## Compat verification Evals types both SDKs structurally (dynamic import), so `tsc` cannot catch drift — verified against the installed `.d.ts` instead: - **claude-agent-sdk 0.3.224**: every `query()` option evals passes exists — `canUseTool` (`updatedInput` now optional), `systemPrompt` preset+append, `settingSources`, `mcpServers`, `pathToClaudeCodeExecutable`, `createSdkMcpServer`/`tool`. Publish history shows 0.3.142 directly succeeds 0.2.141 (version-scheme continuation). - **codex-sdk 0.147.0**: `startThread` options (`sandboxMode`, `approvalPolicy`, `networkAccessEnabled`, `webSearchMode`, `skipGitRepoCheck`), `runStreamed({outputSchema, signal})`, constructor `config` bag (arbitrary `--config` overrides, so `show_raw_agent_reasoning` still flows) all present; `ThreadEvent` kinds unchanged. ## Testing - `turbo run build typecheck --filter @browserbasehq/stagehand-evals` ✅ - Evals unit suite: **420/420** ✅ - Connected smoke (`evals run b:webvoyager --harness claude_code|codex --tool stagehand_code -l 1 -e browserbase`): **pending** — blocked on Browserbase credentials in the runner env; both attempts failed at `Stagehand init` before any SDK code executed. Draft until the smoke passes. <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Converges the evals harness SDK dependencies onto the workspace catalog so the monorepo tracks one version per SDK. `@anthropic-ai/claude-agent-sdk` moves from ^0.2.141 to `catalog:` (0.3.224) and `@openai/codex-sdk` from 0.125.0 to `catalog:` (0.147.0); no runtime changes expected, only lockfile updates. - Verified API compatibility against the installed types; all claude-code harness query options and codex thread/run options are retained. - Evals unit tests pass (420/420); connected smoke is blocked on missing Browserbase credentials and fails before any SDK calls. <sup>Written for commit 3083860. Summary will update on new commits.</sup> <a href="https://cubic.dev/pr/browserbase/stagehand/pull/2743?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Runnable project embedding a Claude agent via
@anthropic-ai/claude-agent-sdk, with thestagehand-facadestdio server mounted programmatically — matching the pattern established by the evalsclaude_codeharness (query()+mcpServers).allowedToolsrestricted tomcp__stagehand__{run,snapshot,screenshot}plus acanUseToolguard (headless runs hang on unanswered permission prompts).FACADE_AGENT_INSTRUCTIONSimported from the facade package; server spawned with aSTAGEHAND_*/BROWSERBASE_*env allowlist.pnpm --filter @browserbasehq/stagehand-integrations-example-claude-code-facade start "your instruction".mcp.json(verified: the CLI inherits the shell env;${VAR}in .mcp.json is not expanded).Verified: contract + allowlist tests in CI; Browserbase smoke — "The page heading is 'Example Domain', per snapshot element [0-19]".
Summary by cubic
Adds a runnable Claude agent harness using
@anthropic-ai/claude-agent-sdkthat mounts the Stagehand facade MCP stdio server. Provides a one-command example and an optional.mcp.jsonfor the Claude Code CLI; onlySTAGEHAND_*/BROWSERBASE_*env reach the browser.@browserbasehq/stagehand-integrations-example-claude-code-facade(Node 24+). Run:pnpm --filter @browserbasehq/stagehand-integrations-example-claude-code-facade start "your instruction". Choose model viaCLAUDE_STAGEHAND_MODEL(defaultclaude-sonnet-5).@browserbasehq/stagehand-integrations/facade/stdio-serverwith an env allowlist;ANTHROPIC_API_KEYis never forwarded.mcp__stagehand__{run,snapshot,screenshot}viaallowedToolsand a deny-by-defaultcanUseTool. UsesFACADE_AGENT_INSTRUCTIONSas the system prompt..mcp.jsonenables theclaudeCLI in this directory; the CLI inherits the shell env (note:${VAR}is not expanded in.mcp.json).main, addimport.meta.mainguard, and exit nonzero if the agent stream ends without a result. Workspace updates add@anthropic-ai/claude-agent-sdkto the catalog/lockfile and aturbotypecheck target; no changes to other packages’ runtime behavior.Written for commit 129610f. Summary will update on new commits.