Skip to content

Disable server/discover probe timeout in in-memory tests to fix flaky Windows Debug CI hang (#1701)#1702

Open
PranavSenthilnathan wants to merge 1 commit into
mainfrom
pranavsenthilnathan/fix-task-cancellation-test-hang
Open

Disable server/discover probe timeout in in-memory tests to fix flaky Windows Debug CI hang (#1701)#1702
PranavSenthilnathan wants to merge 1 commit into
mainfrom
pranavsenthilnathan/fix-task-cancellation-test-hang

Conversation

@PranavSenthilnathan

@PranavSenthilnathan PranavSenthilnathan commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes the flaky Windows Debug CI test-host hang tracked in #1701, which surfaced most often as TaskCancellationIntegrationTests.TaskTool_CancellationToken_GetTaskShowsWorkingBeforeCancel.

The fix is a one-line, test-only change: ClientServerTestBase.CreateMcpClientForServer now forces DiscoverProbeTimeout = Timeout.InfiniteTimeSpan for the in-memory client. The in-memory pipe server always answers server/discover instantly, so the probe never legitimately needs a timeout — but on a slow/overloaded CI agent the default 5s probe timeout could be spuriously exceeded, triggering a protocol-fallback cascade that ends in a hang (details below).

Root cause

A July2026-capable client (the default) does not use the initialize handshake. Instead it calls server/discover to learn the server's capabilities, and only falls back to the legacy initialize handshake if the probe fails or times out. That probe is bounded by McpClientOptions.DiscoverProbeTimeout (default 5s).

On a slow CI agent, the in-memory server/discover round-trip could occasionally take longer than 5s. When that happened:

  1. The probe timed out and the client fell back to the initialize handshake, negotiating an older protocol version.
  2. The older protocol does not negotiate the tasks extension, so the server treated the call as an ordinary (non-task) tool invocation.
  3. TaskCancellationIntegrationTests registers a tool whose body is Task.Delay(Timeout.Infinite, cancellationToken). Without the tasks extension, the server ran that tool inline on the tools/call request instead of dispatching it as a cancellable background task.
  4. The inline tool never completes, so the server never sends the tools/call response, and the client's await CallToolRawAsync(...) blocks forever.
  5. Execution never reaches the test body's asserts — it's a true hang, not a failed assertion — so the 7-minute blame/hang-dump timeout eventually kills the test host, failing the whole run.

Because the trigger is purely environmental timing, the failure was intermittent and Windows-Debug-CI-specific.

How it was diagnosed (hang dump)

The blame-timeout hang dump was the key. Walking the managed thread stacks:

  • Client thread was parked inside CallToolRawAsyncSendRequestAsync, awaiting the JSON-RPC response for tools/call — i.e. the client had sent the request and was waiting for a reply that never came.
  • Server thread was inside the tool handler executing Task.Delay(Timeout.Infinite, …) synchronously on the request-handling path — proving the tool was running inline rather than as a background task.

That pairing (client awaiting a tools/call response while the server is blocked inside the tool on the same call) is only possible when the tasks extension was not negotiated. Confirming the negotiated protocol version was older than July2026 pointed directly at a server/discoverinitialize fallback, and the only thing that forces that fallback in the in-memory harness is the probe timeout expiring — i.e. CI slowness tripping the 5s DiscoverProbeTimeout.

The fix

protected async Task<McpClient> CreateMcpClientForServer(McpClientOptions? clientOptions = null)
{
    clientOptions ??= new McpClientOptions();

    // Disable the server/discover probe timeout to avoid CI slowness spuriously tripping it (issue #1701).
    // Tests that need a specific probe timeout should create their own client instead of using this helper.
    clientOptions.DiscoverProbeTimeout = Timeout.InfiniteTimeSpan;

    return await McpClient.CreateAsync(/* ... */);
}

Within ClientServerTestBase, a fallback can only be caused by the probe timeout (the shared in-memory server always supports server/discover, and back-compat tests that want an older protocol pin the client version, skipping the probe entirely). Disabling the probe timeout therefore removes the spurious-fallback class outright, with no per-caller churn and no production code change.

Tests that intentionally exercise the probe-timeout fallback (July2026ProtocolFallbackTests) build their own transport and call McpClient.CreateAsync directly, so they bypass this helper and are unaffected.

Testing

  • dotnet build — clean (0 warnings / 0 errors).
  • Full in-memory test suite: 2174 passed, 0 failed, 4 skipped. (The external-process stdio families are excluded here only due to a pre-existing local TestServer.exe PATH issue unrelated to this change; they build and run in CI.)
  • The previously-flaky task-cancellation, protocol-fallback, and back-compat tests all pass.

@PranavSenthilnathan
PranavSenthilnathan marked this pull request as draft July 14, 2026 01:21
@PranavSenthilnathan PranavSenthilnathan changed the title Fix flaky task cancellation test hang on Windows Debug CI (#1701) [WIP] Fix flaky task cancellation test hang on Windows Debug CI (#1701) Jul 14, 2026
Comment thread src/ModelContextProtocol.Core/Server/McpServerImpl.cs Outdated
On slow Windows Debug CI, the default 5s DiscoverProbeTimeout can be spuriously exceeded when a July2026-capable client probes server/discover over the in-memory pipe. The client then falls back to the initialize handshake and negotiates an older protocol, dropping tasks-extension negotiation. For TaskCancellationIntegrationTests, whose tool runs Task.Delay(Timeout.Infinite), the fallback makes the server run the tool inline instead of as a background task, so the tools/call never returns and the test host hangs until the blame timeout.

Force DiscoverProbeTimeout = Timeout.InfiniteTimeSpan in ClientServerTestBase.CreateMcpClientForServer. The in-memory pipe server always answers server/discover immediately, so the probe never legitimately needs to time out. This is applied even when a caller supplies its own options so no test can reintroduce the flake. Tests that intentionally exercise the probe-timeout fallback (July2026ProtocolFallbackTests) build their own transport and bypass this helper, so they are unaffected.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@PranavSenthilnathan
PranavSenthilnathan force-pushed the pranavsenthilnathan/fix-task-cancellation-test-hang branch from 42a99f0 to 0726434 Compare July 16, 2026 00:47
@PranavSenthilnathan PranavSenthilnathan changed the title [WIP] Fix flaky task cancellation test hang on Windows Debug CI (#1701) Disable server/discover probe timeout in in-memory tests to fix flaky Windows Debug CI hang (#1701) Jul 16, 2026
@PranavSenthilnathan
PranavSenthilnathan marked this pull request as ready for review July 16, 2026 01:09

// Disable the server/discover probe timeout to avoid CI slowness spuriously tripping it (issue #1701).
// Tests that need a specific probe timeout should create their own client instead of using this helper.
clientOptions.DiscoverProbeTimeout = Timeout.InfiniteTimeSpan;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would the 60 second TestConstant.DefaultTimeout timeout be better?

Suggested change
clientOptions.DiscoverProbeTimeout = Timeout.InfiniteTimeSpan;
clientOptions.DiscoverProbeTimeout = TestConstant.DefaultTimeout;

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants