Add .NET BYOK E2E coverage by roji · Pull Request #2010 · github/copilot-sdk

roji · 2026-07-16T22:19:29Z

The SDK's E2E suites currently validate only the CAPI path, leaving BYOK protocol regressions undetected until later integration stages. This adds .NET coverage for Anthropic Messages, OpenAI Responses, and OpenAI Chat Completions while continuing to use the existing conceptual snapshot corpus.

Approach

Add protocol adapters that normalize BYOK requests into the existing canonical Chat Completions representation and render protocol-complete JSON/SSE responses back to the runtime.
Select one protocol descriptor per backend so CAPI and BYOK share endpoint routing, canonical matching, errors, response delivery, and exchange inspection.
Keep snapshots model-agnostic and reusable across all backends. CAPI remains the sole snapshot-capture path; BYOK replay is explicitly read-only, including compaction replay through the normal snapshot matcher.
Select the E2E backend centrally in the .NET fixture, preserving providers and models explicitly configured by individual tests.
Add three Ubuntu in-process BYOK legs alongside the existing CAPI matrix without multiplying them across transports or operating systems.
Classify CAPI-only and self-configured-backend tests explicitly, and add path-scoped authoring guidance so future .NET E2E tests continue to exercise every provider.

The shared replay harness remains protocol-neutral despite retaining its existing TypeScript class name for compatibility with other SDKs. The .NET wrapper is renamed to ReplayProxy.

Validation

Replay harness: 54 tests passed; TypeScript type-check passed.
Full CAPI .NET baseline after replay unification: 681 passed, 3 skipped.
Anthropic Messages compatible .NET E2E suite: 458 passed, 3 skipped.
Streaming and compaction replay scenarios passed for CAPI and all three BYOK protocols.
Existing CAPI baseline and all three compatible BYOK suites passed before the final harness simplification; the remaining local full Responses failures reflect current CLI tool-catalog drift against older snapshots (apply_patch/rg versus edit/grep), not response-envelope rendering.

Reuse conceptual replay snapshots across Anthropic Messages, OpenAI Responses, and OpenAI Chat Completions, and add three Ubuntu in-process CI legs. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 85a40918-bd95-440b-b6e9-97c757a71f8c

Collapse runtime-specific blank-line expansion so conceptual replay snapshots match recovery turns consistently. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 85a40918-bd95-440b-b6e9-97c757a71f8c

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 85a40918-bd95-440b-b6e9-97c757a71f8c

Make non-CAPI replay explicitly read-only so provider response parsing and SSE aggregation are unnecessary. Keep protocol-complete forward rendering, clarify backend trait naming, and rename the .NET proxy wrapper to reflect its protocol-neutral role. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 85a40918-bd95-440b-b6e9-97c757a71f8c

Select a protocol descriptor once per backend so CAPI and BYOK share routing, canonical matching, errors, JSON/SSE responses, and exchange inspection. Replay compaction directly from canonical snapshots instead of synthesizing provider-specific responses. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 85a40918-bd95-440b-b6e9-97c757a71f8c

Preserve both protocol coalescing and stable built-in tool normalization while resolving the replay proxy conflict. Replace UnreachableException with a net472-compatible exception and remove the unused adapter import. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 85a40918-bd95-440b-b6e9-97c757a71f8c

github-actions · 2026-07-17T06:57:10Z

SDK Consistency Review ✅

This PR adds two features across all six SDK implementations (Node.js, Python, Go, .NET, Java, Rust) with consistent API design:

1. `metadata` field on tool definitions

Opaque, host-defined metadata for tool definitions — added consistently to all SDKs:

Node.js: metadata?: Record<string, unknown> on Tool and defineTool()
Python: metadata: dict[str, Any] | None on Tool dataclass and define_tool()
Go: Metadata map[string]any on Tool struct
.NET: Metadata IDictionary<string, JsonNode?>? on CopilotToolOptions
Java: Map<String, Object> metadata record component on ToolDefinition
Rust: metadata: IndexMap<String, Value> with with_metadata() builder

2. `reasoningEffort` on `CustomAgentConfig`

Per-agent reasoning effort override — added consistently to all SDKs:

Node.js: reasoningEffort?: ReasoningEffort
Python: reasoning_effort: NotRequired[ReasoningEffort]
Go: ReasoningEffort string
.NET: ReasoningEffort string?
Java: String reasoningEffort with getReasoningEffort() / setReasoningEffort()
Rust: reasoning_effort: Option<String> with with_reasoning_effort() (serializes to reasoningEffort via #[serde(rename_all = "camelCase")])

No consistency issues found. All features maintain proper language-idiomatic naming while preserving the same JSON wire format (reasoningEffort, metadata) across all SDKs.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

Generated by SDK Consistency Review Agent for #2010 · 42.5 AIC · ⌖ 5.19 AIC · ⊞ 5K · ◷

Copilot

Pull request overview

Adds .NET BYOK E2E coverage by adapting Anthropic and OpenAI protocols to the canonical replay snapshot format.

Changes:

Adds Anthropic Messages and OpenAI Responses adapters.
Centralizes .NET E2E backend/provider selection.
Adds three Ubuntu in-process BYOK CI legs and backend traits.

Show a summary per file

File	Description
`test/harness/responsesApiAdapter.ts`	Adapts OpenAI Responses requests and responses.
`test/harness/replayingCapiProxy.ts`	Adds protocol-aware, read-only BYOK replay.
`test/harness/modelProtocolAdapterShared.ts`	Defines shared adapter utilities.
`test/harness/modelProtocolAdapters.test.ts`	Tests adapters and backend replay.
`test/harness/anthropicMessagesAdapter.ts`	Adapts Anthropic Messages requests and responses.
`dotnet/test/Unit/E2ETestBackendTests.cs`	Tests backend configuration.
`dotnet/test/Harness/ReplayProxy.cs`	Renames and configures the replay proxy wrapper.
`dotnet/test/Harness/E2ETestContext.cs`	Applies selected backends to sessions.
`dotnet/test/Harness/E2ETestBase.cs`	Routes session operations through context.
`dotnet/test/Harness/E2ETestBackend.cs`	Defines backend configuration and traits.
`dotnet/test/E2E/ToolsE2ETests.cs`	Uses backend-aware session creation.
`dotnet/test/E2E/TelemetryExportE2ETests.cs`	Uses backend-aware session creation.
`dotnet/test/E2E/SuspendE2ETests.cs`	Uses backend-aware create/resume helpers.
`dotnet/test/E2E/SubagentHooksE2ETests.cs`	Uses backend-aware session creation.
`dotnet/test/E2E/StreamingFidelityE2ETests.cs`	Adds backend classification and resume routing.
`dotnet/test/E2E/SessionFsSqliteE2ETests.cs`	Routes sessions through context.
`dotnet/test/E2E/SessionFsE2ETests.cs`	Routes create/resume operations through context.
`dotnet/test/E2E/SessionE2ETests.cs`	Adds backend classifications and resume routing.
`dotnet/test/E2E/SessionConfigE2ETests.cs`	Classifies provider-specific scenarios.
`dotnet/test/E2E/RpcTasksAndHandlersE2ETests.cs`	Marks a CAPI-only scenario.
`dotnet/test/E2E/RpcSessionStateExtrasE2ETests.cs`	Classifies model/provider tests.
`dotnet/test/E2E/RpcSessionStateE2ETests.cs`	Uses backend-aware session creation.
`dotnet/test/E2E/RpcServerE2ETests.cs`	Classifies and routes RPC session tests.
`dotnet/test/E2E/RpcMcpAndSkillsE2ETests.cs`	Routes sessions through context.
`dotnet/test/E2E/RpcExtensionsLoadedE2ETests.cs`	Routes extension sessions through context.
`dotnet/test/E2E/ProviderEndpointE2ETests.cs`	Classifies provider endpoint tests.
`dotnet/test/E2E/PerSessionAuthE2ETests.cs`	Marks authentication tests CAPI-only.
`dotnet/test/E2E/PermissionE2ETests.cs`	Uses backend-aware resume handling.
`dotnet/test/E2E/PendingWorkResumeE2ETests.cs`	Routes pending-work sessions through context.
`dotnet/test/E2E/MultiProviderRegistryE2ETests.cs`	Marks self-configured provider coverage.
`dotnet/test/E2E/MultiClientE2ETests.cs`	Routes multi-client sessions through context.
`dotnet/test/E2E/MultiClientCommandsElicitationE2ETests.cs`	Routes multi-client command sessions.
`dotnet/test/E2E/ModeHandlersE2ETests.cs`	Marks mode-handler tests CAPI-only.
`dotnet/test/E2E/ModeEmptyE2ETests.cs`	Uses backend-aware session creation.
`dotnet/test/E2E/GitHubTelemetryForwardingE2ETests.cs`	Marks telemetry forwarding CAPI-only.
`dotnet/test/E2E/CopilotRequestWebSocketE2ETests.cs`	Marks self-configured WebSocket coverage.
`dotnet/test/E2E/CopilotRequestSessionIdE2ETests.cs`	Classifies CAPI and BYOK session-ID tests.
`dotnet/test/E2E/CopilotRequestCancelErrorE2ETests.cs`	Routes request-handler sessions through context.
`dotnet/test/E2E/ClientOptionsE2ETests.cs`	Applies backend-aware session helpers.
`dotnet/test/E2E/ClientE2ETests.cs`	Integrates the E2E context fixture.
`dotnet/test/E2E/ByokBearerTokenProviderE2ETests.cs`	Marks explicit BYOK provider tests.
`dotnet/test/AssemblyInfo.cs`	Updates replay proxy terminology.
`.github/workflows/dotnet-sdk-tests.yml`	Adds BYOK CI matrix legs and filtering.
`.github/instructions/dotnet-e2e.instructions.md`	Documents backend-aware E2E authoring.

Review details

Files reviewed: 44/44 changed files
Comments generated: 3
Review effort level: Medium

        var isolatedClient = isolatedCtx.CreateClient();

-        await using var session = await isolatedClient.CreateSessionAsync(new SessionConfig
+        await using var session = await Ctx.CreateSessionAsync(isolatedClient, new SessionConfig


+  const skeleton = {
+    ...fullResponse,
+    output: [],
+    output_text: "",
+    usage: undefined,
+  };


+    const item = fullResponse.output[outputIndex];
+    chunks.push(
+      event("response.output_item.added", {
+        output_index: outputIndex,
+        item,
+      }),


roji added 8 commits July 14, 2026 20:18

Normalize Anthropic adjacent user turns

551fa02

Collapse runtime-specific blank-line expansion so conceptual replay snapshots match recovery turns consistently. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 85a40918-bd95-440b-b6e9-97c757a71f8c

Document potential BYOK E2E gaps

97a7c05

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 85a40918-bd95-440b-b6e9-97c757a71f8c

Restore workflow path quoting

6b6c00e

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 85a40918-bd95-440b-b6e9-97c757a71f8c

Preserve the .NET test matrix

25645be

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 85a40918-bd95-440b-b6e9-97c757a71f8c

Clarify replay harness test leg

581fc1d

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 85a40918-bd95-440b-b6e9-97c757a71f8c

Use existing replay harness test coverage

34b6a60

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 85a40918-bd95-440b-b6e9-97c757a71f8c

github-advanced-security AI found potential problems Jul 16, 2026

View reviewed changes

Comment thread test/harness/replayingCapiProxy.ts Fixed

Comment thread test/harness/responsesApiAdapter.ts Fixed

roji added 2 commits July 17, 2026 00:41

roji marked this pull request as ready for review July 17, 2026 07:27

roji requested a review from a team as a code owner July 17, 2026 07:27

Copilot AI review requested due to automatic review settings July 17, 2026 07:27

Copilot started reviewing on behalf of roji July 17, 2026 07:27 View session

Copilot AI reviewed Jul 17, 2026

View reviewed changes

roji mentioned this pull request Jul 17, 2026

Fix consistency review PR delta detection #2018

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add .NET BYOK E2E coverage#2010

Add .NET BYOK E2E coverage#2010
roji wants to merge 10 commits into
mainfrom
roji-dotnet-byok-e2e

roji commented Jul 16, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Jul 17, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

roji commented Jul 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Approach

Validation

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Jul 17, 2026

SDK Consistency Review ✅

1. metadata field on tool definitions

2. reasoningEffort on CustomAgentConfig

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Review details

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

roji commented Jul 16, 2026 •

edited

Loading

1. `metadata` field on tool definitions

2. `reasoningEffort` on `CustomAgentConfig`