Launch managed SDK servers through the Rust runtime wrapper

[roji]

Managed out-of-process SDK connections currently launch the root Copilot SEA directly. The runtime now provides a dedicated copilot-runtime executable that loads the adjacent runtime.node, so SDK-managed subprocess connections should use that coherent wrapper/addon pair while preserving explicit external server and in-process modes.

What changed

• Update Node.js, Python, Go, .NET, Rust, and Java managed launch paths to resolve and spawn copilot-runtime (copilot-runtime.exe on Windows).
• Materialize and validate the wrapper beside runtime.node, retaining the root copilot executable only for residual and in-process compatibility.
• Preserve auxiliary runtime assets such as ripgrep, tgrep, built-in definitions, skills, and future package additions. Each SDK keeps unknown npm package entries by default while excluding known CLI/UI-only content.
• Preserve explicit CLI path, process, and server URL overrides without silently falling back to the SEA when managed wrapper artifacts are incomplete.
• Extend language-specific packaging, extraction, permissions, cache, and test coverage for the coherent runtime distribution.
• Fix a Python subprocess stderr-reader race and Java MCP stdio cwd serialization issue discovered during end-to-end validation.

Validation

Validated all six SDKs against the immutable same-tip runtime release candidate adce886fd1 across supported root SEA, in-process, and hostless wrapper transports. The final focused cache/resume gates and supported full-suite cells passed with exact artifact hash checks and no leaked runtime processes.

The auxiliary-asset follow-up adds focused staging, cache-migration, filtering, permission, and path-safety tests in all six SDKs. A real Node hostless-wrapper integration test also confirms built-in grep completes successfully through the staged ripgrep binary.

Known non-runtime exclusions remain documented from validation: hostless Node extension/factory lifecycle behavior (including the corresponding .NET extension tests), the Go invalid-model fixture, the Java literal-YAML fixture and order-dependent socket-helper contamination, and Rust forced-environment unit-test contamination.

Review notes

This is a draft while the runtime platform packages and Java classifier publication complete. The production platform package is treated as the runtime distribution source: the root copilot[.exe], selected prebuilds/<platform>/runtime.node and copilot-runtime[.exe], and retained package-relative runtime assets are staged as one coherent version. Mixed or incomplete artifact sets are rejected.

[SteveSandersonMS]

The hostless wrapper direction looks right, but this is not complete yet because runtime.node is not self-contained.

I manually tested all 48 combinations: 6 SDKs × stdio/TCP/in-process/existing URL × source/published consumption, using @github/copilot@1.0.81-9. Startup succeeded without launching the SEA in every case. However, built-in grep failed with ENOENT in every stdio/TCP case and in 10 of 12 in-process cases. Existing-URL cases passed because that runtime used the intact npm package layout.

The missing piece is preserving the hostless runtime assets relative to runtime.node. A reasonable minimal payload is:

prebuilds/<platform>/runtime.node
prebuilds/<platform>/copilot-runtime[.exe]
ripgrep/
tgrep/
definitions/
builtin/
builtin-skills/

Node-only and delivery-only content can be excluded: copilot, app.js, index.js, npm-loader.js, sea-loader.js, copilot-sdk/, sdk/, cli-native.node, preloads/, foundry-local-sdk/, pvrecorder/, voice-*.js, webview/, tree-sitter*.wasm, queries/, UI assets, metadata, and the duplicate copilot-runtime-bin.

I validated this approach on .NET in both development and after dotnet publish. Preserving ripgrep/bin/<platform>/rg fixed built-in grep for stdio, TCP, and in-process. Adding tgrep/bin/<platform>/tgrep also worked with USE_TGREP=true. I repeated the runs after deleting the SEA; all passed. In-process worked using adjacent runtime.node as the entrypoint/root hint.

Before merging, I think this PR needs to:

1. Materialize the complete hostless asset set in all six SDKs while preserving npm-relative paths.
2. Remove the SEA from published outputs and embedded bundles.
3. Make in-process resolution use the hostless runtime layout rather than retaining the SEA.
4. Add E2E coverage that asserts successful tool.execution_complete events for built-in grep/tgrep in source and published consumers. Checking startup or final assistant text is insufficient because the model can silently fall back to another tool.
5. Future enhancement: Ideally define this asset set once upstream as a manifest/archive so six SDK implementations do not drift.

[Copilot]

Pull request overview

Routes managed subprocess SDK connections through the coherent copilot-runtime/runtime.node distribution while preserving explicit and in-process launch modes.

Changes:

• Updates all six SDK launch paths and packaging.
• Retains auxiliary runtime assets with cache and integrity handling.
• Expands tests and documentation for runtime resolution.

Show a summary per file

File	Description
rust/tests/e2e/support.rs	Preserves explicit CLI environment paths.
rust/tests/cli_resolution_test.rs	Tests wrapper resolution and extraction.
rust/src/startup_timings.rs	Updates resolution timing documentation.
rust/src/resolve.rs	Resolves and validates runtime pairs.
rust/src/lib.rs	Exposes bundled runtime installation.
rust/src/ffi.rs	Supports adjacent runtime libraries and musl.
rust/src/embeddedcli.rs	Extracts coherent runtime bundles.
rust/README.md	Documents bundled runtime behavior.
rust/build/in_process.rs	Builds and caches runtime distributions.
rust/build.rs	Unifies build implementation.
python/test_client.py	Tests explicit launch overrides.
python/test_cli_download.py	Tests runtime bundle provisioning.
python/README.md	Documents runtime downloads and paths.
python/copilot/client.py	Selects the managed wrapper.
python/copilot/_cli_download.py	Downloads and stages runtime assets.
nodejs/test/runtimeArtifacts.test.ts	Tests runtime materialization.
nodejs/test/e2e/factory.e2e.test.ts	Disables factory E2E coverage.
nodejs/test/e2e/extension_env_access.e2e.test.ts	Disables extension-host coverage.
nodejs/test/e2e/builtin_tools.e2e.test.ts	Adds grep runtime validation.
nodejs/test/client.test.ts	Tests explicit path precedence.
nodejs/src/runtimeArtifacts.ts	Materializes platform runtime assets.
nodejs/src/client.ts	Launches the bundled runtime wrapper.
nodejs/README.md	Documents managed wrapper launches.
java/sdk/src/test/java/com/github/copilot/ffi/NativeRuntimeLoaderTest.java	Tests classifier runtime extraction.
java/sdk/src/test/java/com/github/copilot/CliServerManagerTest.java	Tests explicit CLI paths.
java/sdk/src/main/java/com/github/copilot/ffi/NativeRuntimeLoader.java	Extracts wrapper and retained assets.
java/sdk/src/main/java/com/github/copilot/CliServerManager.java	Launches the resolved wrapper.
java/README.md	Documents Java runtime provisioning.
java/copilot-native/scripts/fetch-native.test.mjs	Tests native asset staging.
java/copilot-native/scripts/fetch-native.mjs	Builds classifier runtime trees.
java/copilot-native/pom.xml	Verifies packaged wrappers.
go/README.md	Documents embedded runtime resolution.
go/internal/ffihost/resolve.go	Supports adjacent runtime.node.
go/internal/embeddedcli/embeddedcli.go	Installs wrapper pairs and assets.
go/internal/embeddedcli/embeddedcli_test.go	Tests runtime installation.
go/cmd/bundler/main.go	Bundles coherent runtime artifacts.
go/cmd/bundler/main_test.go	Tests filtering and generated embeds.
go/client.go	Selects the managed runtime executable.
go/client_test.go	Tests runtime selection failures.
dotnet/test/Unit/RuntimeWrapperTests.cs	Tests wrapper resolution behavior.
dotnet/test/Unit/MSBuildTargetsTests.cs	Tests runtime asset copying.
dotnet/test/E2E/RpcExtensionsLoadedE2ETests.cs	Disables extension lifecycle tests.
dotnet/test/E2E/BuiltinToolsE2ETests.cs	Re-enables ripgrep tool tests.
dotnet/src/Client.cs	Resolves and validates runtime pairs.
dotnet/src/build/GitHub.Copilot.SDK.targets	Packages filtered runtime assets.
dotnet/README.md	Documents managed runtime launches.

Review details

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Suppressed comments (1)

rust/build/in_process.rs:136

• Removing the shared install directory is not coordinated across build processes. Two concurrent Cargo builds can both observe an incomplete cache, then one can delete the directory while the other is publishing its file-level staging entries, causing a panic or a mixed/incomplete cache. Serialize refreshes with a cross-process lock, or stage a complete unique directory and publish it atomically without deleting another writer's live directory.

• Files reviewed: 46/46 changed files
• Comments generated: 9
• Review effort level: Balanced

[SteveSandersonMS]

Here's some output from a review agent. Sorry the formatting is a bit weird after the paste. Hopefully your agent can make sense of it even though the tables are misaligned.

---

Manual validation. I rebuilt minimal consumers and reran all 48 combinations: six languages × stdio/TCP/in-process/existing URL × source/published. I required successful grep tool events and physically removed the SEA where possible. 23/48 satisfy both gates.

SDK	Result
Node	Runtime works without SEA, but every published npm deployment contains the 159 MB platform SEA.
Python	Best result: clean 556 KB wheel; stdio/TCP/URL pass. Published in-process still downloads SEA + native library without auxiliary assets, so grep fails.
.NET	Published output contains the 159 MB SEA. Removing it leaves stdio/TCP working, but default in-process startup fails until COPILOT_CLI_PATH points to copilot-runtime.
Go	Published ~285 MB binaries embed the compressed SEA; in-process resolves through embeddedcli.Path(), requiring it.
Rust	Default embedded archive contains copilot; removing it exposes an in-process BinaryNotFound(copilot) failure.
Java	Published shaded JAR is 140.6 MB; removing SEA reduces it to 48.2 MB and stdio/TCP still pass, but default in-process requires copilot.

Minimal reproduction of the common defect:

File.Delete("runtimes/linux-x64/native/copilot");
await StartAsync(RuntimeConnection.ForStdio());     // passes
await StartAsync(RuntimeConnection.ForInProcess()); // currently fails

Additional blockers. The PR disables real extension and factory E2Es because the new default loses the Node extension lifecycle (Node, .NET). That requires an explicit product decision or implementation, not skipped coverage.

---

Also rather than disabling the e2e tests for extensions/factories, could we leave them there but have the test supply an external Node binary since we now support configuring that?

[roji]

Addressed in e3f7726. The extension/factory E2Es remain enabled and explicitly opt into an external legacy copilot SEA over stdio when they need Node extension hosting. Default managed out-of-process and in-process startup remain SEA-free. I also removed the SEA from Node, Python, and Java hostless runtime staging (Go, .NET, and Rust already excluded it from their hostless runtime assets) and validated wrapper plus in-process ping/shutdown paths across all six SDKs with no SEA available.

[github-actions]

Generated by SDK Consistency Review Agent for #2395 · sonnet46 136.3 AIC · ⌖ 5.83 AIC · ⊞ 6.6K

[github-actions]

Generated by SDK Consistency Review Agent for #2395 · sonnet46 132.2 AIC · ⌖ 5.63 AIC · ⊞ 6.6K

[github-actions]

Generated by SDK Consistency Review Agent for #2395 · sonnet46 136.6 AIC · ⌖ 5.61 AIC · ⊞ 6.6K

[github-actions]

Cross-SDK Consistency Review ✅

This PR demonstrates excellent cross-SDK consistency. All six SDK implementations (Node.js, Python, Go, .NET, Java, and Rust) are updated in parallel with equivalent changes.

What was checked

Out-of-process transport (managed child-process launch)
All SDKs now resolve and spawn copilot-runtime / copilot-runtime.exe paired with an adjacent runtime.node, replacing the legacy root SEA executable. The resolution priority (explicit path → COPILOT_CLI_PATH env → bundled runtime pair → error) is consistent across all six languages.

In-process (FFI) transport
All SDKs updated FfiRuntimeHost.Create / equivalent to accept libraryPath as the primary parameter and make cliEntrypoint optional (string | undefined / str | None / string? / Option<PathBuf> depending on language). The buildArgvJson / equivalent correctly emits an empty argv when no legacy entrypoint is provided.

Runtime pair validation
Each SDK validates that the wrapper binary and adjacent runtime.node are both present and non-empty before attempting to launch. The validation logic is conceptually identical across .NET (ValidateRuntimePair), Rust (validate_runtime_pair), Node.js (validateRuntimeBundle), Python (inline checks in ensure_runtime_wrapper), Go (via ResolveLibraryPath + bundler validation), and Java (NativeRuntimeLoader).

Asset filtering / exclusion lists
The EXCLUDED_TOP_LEVEL sets in Node.js (runtimeArtifacts.ts), Python (_cli_download.py), Java (fetch-native.mjs), and Go (cmd/bundler/main.go) are consistent, excluding the same UI-only and CLI-only top-level entries.

SDK-specific mechanisms are intentionally different

• The .copilot-explicit-cli marker file is .NET-only, tied to MSBuild's GitHub.Copilot.SDK.targets packaging model — other SDKs use different packaging mechanisms (Go bundler binary, Rust build-time embedding, Node.js runtime npm resolution, Python on-demand download, Java Maven classifier JARs).
• The Python stderr-reader race fix and Java MCP stdio cwd serialization fix are correctness fixes discovered during validation — not missing from other SDKs.

No cross-SDK consistency gaps found.

Generated by SDK Consistency Review Agent for #2395 · sonnet46 56.1 AIC · ⌖ 8.34 AIC · ⊞ 6.6K · ◷