fsi --typecheck-only sometimes deadlocks on TcImports locks while registering referenced assemblies (SDK 10.0.403) #20966

Drafted with Claude Code (automated). Reviewed and filed on behalf of the reporter.

Summary

dotnet fsi --typecheck-only sometimes deadlocks while it registers referenced assemblies, when many fsi processes run at the same time. The hung process uses 0% CPU. Its thread-pool threads wait on Monitor.Enter inside TcImports, and other threads hold those locks and wait in Async.RunSynchronously. Each process runs alone, so the processes do not share state. The deadlock is inside one process, and load only makes it more likely.

Environment

• .NET SDK 10.0.403, Microsoft (R) F# Interactive version 15.2.403.0 for F# 10.0
• Windows 11 x64, 24 logical processors, 64 GB
• Each script references about 20 project assemblies with #r (found through --lib:<folder> and #I) and #loads several other scripts, some of which declare a namespace and load .paket/load package scripts.

How it shows

A build tool runs dotnet fsi --typecheck-only <script>.fsx for about 280 scripts, 24 processes at a time. Alone, each script type-checks in about 10 to 40 seconds.

• In each full run, 1 to 4 processes out of 100 to 300 never finish.
• A different script hangs on each run. The hung script passes when it runs again alone.
• A hung process used 0 s of CPU over 15 s, and it had 57 threads.
• DOTNET_ThreadPool_ForceMinWorkerThreads=40 (64 worker threads) did not stop the hangs, so this is a lock cycle, not thread-pool starvation.

The workaround in our tool is a 3-minute timeout and one retry for each script.

Stacks of a hung process (dotnet-stack report, grouped)

Main thread:

FSharpAsync.RunSynchronously
<- CompilerImports.RequireReferences
<- FsiDynamicCompiler.ProcessDelayedReferences
<- FsiDynamicCompiler.EvalSourceFiles
<- FsiInteractionProcessor.PartiallyProcessHashDirective
<- ... <- FsiInteractionProcessor.LoadInitialFiles <- FsiEvaluationSession.Run

Thread-pool threads waiting for a lock:

4x  Monitor.Enter <- TcImports.OpenILBinaryModule <- tryGetAssemblyData
2x  Monitor.Enter <- CompilerImports.look <- fixupThunk <- PickledDataWithReferences.OptionalFixup
        <- PrepareToImportReferencedFSharpAssembly <- registerDll
1x  Monitor.Enter <- fixupThunk <- PickledDataWithReferences.OptionalFixup
        <- PrepareToImportReferencedFSharpAssembly <- registerDll
1x  Monitor.Enter <- TcImports.PrepareToImportReferencedILAssembly <- registerDll
        <- RegisterAndImportReferencedAssemblies
1x  Monitor.Enter <- TcImports.RegisterCcu <- TcImports.PrepareToImportReferencedFSharpAssembly <- registerDll
1x  Monitor.Enter <- TcImports.TryResolveAssemblyReference <- tryFile
        <- TcImports.ImplicitLoadIfAllowed <- fixupThunk <- PickledDataWithReferences.OptionalFixup

Thread-pool threads waiting for an async result while they hold their caller's locks:

13x FSharpAsync.RunSynchronously <- CompilerImports.tryFile
        <- TcImports.ImplicitLoadIfAllowed <- fixupThunk <- PickledDataWithReferences.OptionalFixup
        <- PrepareToImportReferencedFSharpAssembly <- registerDll

registerDll runs on several pool threads at once. While a thread fixes up a pickled F# assembly, ImplicitLoadIfAllowed calls tryFile, which blocks in Async.RunSynchronously on work that needs a TcImports lock that another registering thread holds.

Related

• Audit RequireCompilationThread and related not enforced phantom token/lock invariants wrt parallel compilation #20804 (audit of lock invariants now that checking is concurrent)
• PR AsyncLocal diagnostics, clean slate #16779 mentions a deadlock in RegisterAndImportReferencedAssemblies when FSI calls it.

I do not have a minimal public repro yet, because the hang is random and needs many referenced F# assemblies. I can capture a full dump of a hung process if that helps.