Skip to content

feat(integrations): Claude Agent SDK harness example - #2719

Merged
miguelg719 merged 3 commits into
mainfrom
miguel/harness-claude-code
Aug 14, 2026
Merged

feat(integrations): Claude Agent SDK harness example#2719
miguelg719 merged 3 commits into
mainfrom
miguel/harness-claude-code

Conversation

@miguelg719

@miguelg719 miguelg719 commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator

Runnable project embedding a Claude agent via @anthropic-ai/claude-agent-sdk, with the stagehand-facade stdio server mounted programmatically — matching the pattern established by the evals claude_code harness (query() + mcpServers).

  • allowedTools restricted to mcp__stagehand__{run,snapshot,screenshot} plus a canUseTool guard (headless runs hang on unanswered permission prompts).
  • System prompt is FACADE_AGENT_INSTRUCTIONS imported from the facade package; server spawned with a STAGEHAND_*/BROWSERBASE_* env allowlist.
  • One-line run after install + core build: pnpm --filter @browserbasehq/stagehand-integrations-example-claude-code-facade start "your instruction"
  • Secondary README section: connecting a running Claude Code CLI via the project-scoped .mcp.json (verified: the CLI inherits the shell env; ${VAR} in .mcp.json is not expanded).

Verified: contract + allowlist tests in CI; Browserbase smoke — "The page heading is 'Example Domain', per snapshot element [0-19]".


Summary by cubic

Adds a runnable Claude agent harness using @anthropic-ai/claude-agent-sdk that mounts the Stagehand facade MCP stdio server. Provides a one-command example and an optional .mcp.json for the Claude Code CLI; only STAGEHAND_*/BROWSERBASE_* env reach the browser.

  • New package: @browserbasehq/stagehand-integrations-example-claude-code-facade (Node 24+). Run: pnpm --filter @browserbasehq/stagehand-integrations-example-claude-code-facade start "your instruction". Choose model via CLAUDE_STAGEHAND_MODEL (default claude-sonnet-5).
  • Spawns @browserbasehq/stagehand-integrations/facade/stdio-server with an env allowlist; ANTHROPIC_API_KEY is never forwarded.
  • Restricts tools to mcp__stagehand__{run,snapshot,screenshot} via allowedTools and a deny-by-default canUseTool. Uses FACADE_AGENT_INSTRUCTIONS as the system prompt.
  • .mcp.json enables the claude CLI in this directory; the CLI inherits the shell env (note: ${VAR} is not expanded in .mcp.json).
  • Tests cover the tool name contract and env allowlist. Robustness: resolve server path inside main, add import.meta.main guard, and exit nonzero if the agent stream ends without a result. Workspace updates add @anthropic-ai/claude-agent-sdk to the catalog/lockfile and a turbo typecheck target; no changes to other packages’ runtime behavior.

Written for commit 129610f. Summary will update on new commits.

Review in cubic

@socket-security

socket-security Bot commented Aug 14, 2026

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addednpm/​@​anthropic-ai/​claude-agent-sdk@​0.3.224801009210070

View full report

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 11 files

Architecture diagram
sequenceDiagram
    participant User as User/CLI
    participant Agent as Claude Agent SDK
    participant MCP as Stagehand MCP Server
    participant Facade as Stagehand Facade
    participant Browser as Browserbase Browser
    participant Env as Environment Filter

    Note over User,Browser: NEW: Claude Code Facade Integration Flow

    User->>Agent: Run instruction via query()
    Agent->>Agent: Load FACADE_AGENT_INSTRUCTIONS as system prompt

    Agent->>MCP: Spawn stdio server (mcpServers.stagehand)
    Agent->>Env: Request env allowlist
    Env->>Env: Filter to STAGEHAND_*/BROWSERBASE_* vars
    Env-->>Agent: Allowlisted env (ANTHROPIC_API_KEY excluded)
    Agent->>MCP: Pass allowlisted env to subprocess

    Note over Agent: Tool permission enforcement
    Agent->>Agent: Check canUseTool guard
    alt Tool starts with "mcp__stagehand__"
        Agent->>Agent: Allow tool execution
        Agent->>MCP: Call mcp__stagehand__{run,snapshot,screenshot}
        MCP->>Facade: Execute Stagehand operation
        Facade->>Browser: Drive browser (via Browserbase)
        Browser-->>Facade: Browser result
        Facade-->>MCP: Tool result
        MCP-->>Agent: MCP response
    else Non-stagehand tool
        Agent->>Agent: Deny with message "Only stagehand browser tools permitted"
    end

    Agent->>Agent: Stream agent messages
    alt Success result
        Agent-->>User: Print final result
    else Error/Cancelled result
        Agent-->>User: Throw error, set exit code 1
    end

    Note over User,MCP: Alternative: Claude Code CLI with .mcp.json
    User->>MCP: Connect via project-scoped .mcp.json
    Note over MCP: CLI inherits shell env (no ${VAR} expansion)
    MCP-->>User: Tool access available
Loading

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread packages/integrations/claude-code/src/agent.ts
Comment thread packages/integrations/claude-code/README.md
Comment thread packages/integrations/claude-code/README.md
Comment thread packages/integrations/claude-code/src/env.ts
Comment thread packages/integrations/claude-code/tsconfig.json Outdated
Comment thread packages/integrations/claude-code/src/agent.ts Outdated
Comment thread packages/integrations/claude-code/src/agent.ts
@changeset-bot

changeset-bot Bot commented Aug 14, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 129610f

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Runnable project embedding a Claude agent via
@anthropic-ai/claude-agent-sdk with the stagehand-facade stdio server
mounted programmatically (env allowlist, allowedTools + canUseTool guard
for headless runs, FACADE_AGENT_INSTRUCTIONS as system prompt). Matches
the pattern established by the evals claude_code harness. Secondary
README section covers connecting a running Claude Code CLI via
.mcp.json (verified: CLI inherits shell env; ${VAR} is not expanded).
tsconfig checks src directly (include listed a nonexistent extensions
dir), server path resolves inside main so failures surface through the
error handler, and a stream ending without a result now exits nonzero.
@miguelg719
miguelg719 force-pushed the miguel/harness-claude-code branch from 45ea444 to 129610f Compare August 14, 2026 20:02
@miguelg719
miguelg719 merged commit bc88526 into main Aug 14, 2026
49 checks passed
miguelg719 added a commit that referenced this pull request Aug 22, 2026
PRs #2719/#2720 added @anthropic-ai/claude-agent-sdk (0.3.224) and
@openai/codex-sdk (0.147.0) to the pnpm catalog for the integration
examples, forking the versions evals pinned directly (^0.2.141 and
0.125.0). Move both evals deps to catalog: so the monorepo tracks a
single version per SDK.

Compat verified against the installed types: every query() option the
claude_code harness passes (canUseTool, systemPrompt preset+append,
settingSources, mcpServers, createSdkMcpServer/tool) exists in 0.3.224,
and every codex option (startThread config, runStreamed outputSchema/
signal, constructor config bag incl. show_raw_agent_reasoning) exists
in 0.147.0 with unchanged ThreadEvent kinds. Evals unit suite passes
(420 tests).
miguelg719 added a commit that referenced this pull request Aug 30, 2026
…2743)

## What

Moves both external-harness SDK deps in `packages/evals` onto the
workspace catalog:

- `@anthropic-ai/claude-agent-sdk`: `^0.2.141` → `catalog:`
(**0.3.224**)
- `@openai/codex-sdk`: `0.125.0` → `catalog:` (**0.147.0**)

## Why

#2719 and #2720 added both SDKs to the catalog for the integration
examples, forking the versions evals pinned directly — two copies of
each SDK in the monorepo. This converges on one version per SDK, as a
standalone pre-step to the broader harness-adapter consolidation.

## Compat verification

Evals types both SDKs structurally (dynamic import), so `tsc` cannot
catch drift — verified against the installed `.d.ts` instead:

- **claude-agent-sdk 0.3.224**: every `query()` option evals passes
exists — `canUseTool` (`updatedInput` now optional), `systemPrompt`
preset+append, `settingSources`, `mcpServers`,
`pathToClaudeCodeExecutable`, `createSdkMcpServer`/`tool`. Publish
history shows 0.3.142 directly succeeds 0.2.141 (version-scheme
continuation).
- **codex-sdk 0.147.0**: `startThread` options (`sandboxMode`,
`approvalPolicy`, `networkAccessEnabled`, `webSearchMode`,
`skipGitRepoCheck`), `runStreamed({outputSchema, signal})`, constructor
`config` bag (arbitrary `--config` overrides, so
`show_raw_agent_reasoning` still flows) all present; `ThreadEvent` kinds
unchanged.

## Testing

- `turbo run build typecheck --filter @browserbasehq/stagehand-evals` ✅
- Evals unit suite: **420/420** ✅
- Connected smoke (`evals run b:webvoyager --harness claude_code|codex
--tool stagehand_code -l 1 -e browserbase`): **pending** — blocked on
Browserbase credentials in the runner env; both attempts failed at
`Stagehand init` before any SDK code executed. Draft until the smoke
passes.

<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Converges the evals harness SDK dependencies onto the workspace catalog
so the monorepo tracks one version per SDK.
`@anthropic-ai/claude-agent-sdk` moves from ^0.2.141 to `catalog:`
(0.3.224) and `@openai/codex-sdk` from 0.125.0 to `catalog:` (0.147.0);
no runtime changes expected, only lockfile updates.

- Verified API compatibility against the installed types; all
claude-code harness query options and codex thread/run options are
retained.
- Evals unit tests pass (420/420); connected smoke is blocked on missing
Browserbase credentials and fails before any SDK calls.

<sup>Written for commit 3083860.
Summary will update on new commits.</sup>

<a
href="https://cubic.dev/pr/browserbase/stagehand/pull/2743?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>

<!-- End of auto-generated description by cubic. -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants