Skip to content

Commit 3e64dbd

Browse files
Update C# Runtime dependencies to support NativeAOT-LLVM prerequisites (#4601)
# Description of Changes * Add dotnet-experimental feed + package source mapping for LLVM packages in `sdks/csharp/tools~/write-nuget-config.sh`, so generated NuGet.Config files include NativeAOT-LLVM prerequisites. * Make LLVM toolchain packages explicit dependencies in `SpacetimeDB.Runtime` to ensure restores succeed even when LLVM dependencies are only referenced through the `.nupkg`. * Import the LLVM targets from the package when `EXPERIMENTAL_WASM_AOT=1` to enable NativeAOT build steps without relying on downstream package reference resolution. # Context Changes are required to get `NativeAOT-LLVM` in #4515 to build correct, but moving the packages closer to the build, to ensure they get into the Nuget restore successfully. Additional changes where needed to `write-nuget-config.sh‎` to allow `Nuget.Config` files generated with required changes during regression testing. # API and ABI breaking changes None. # Expected complexity level and risk 2 (Low–moderate). Changes are scoped to build/restore infrastructure and package configuration. # Testing - [X] Built CLI locally - [X] Ran `run-regression-tests.sh` without errors --------- Co-authored-by: Jason Larabie <jason@clockworklabs.io>
1 parent 1b01ebb commit 3e64dbd

6 files changed

Lines changed: 188 additions & 8 deletions

File tree

.github/workflows/ci.yml

Lines changed: 33 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -869,11 +869,41 @@ jobs:
869869
exit 1
870870
}
871871
872+
# NativeAOT-LLVM smoketest runs on Windows because runtime.linux-x64.Microsoft.DotNet.ILCompiler.LLVM
873+
# does not have 8.0.0 versions available on the dotnet-experimental NuGet feed.
874+
csharp-aot-smoketest:
875+
needs: [lints]
876+
runs-on: windows-latest
877+
timeout-minutes: 15
878+
steps:
879+
- name: Checkout repository
880+
uses: actions/checkout@v4
881+
882+
- name: Setup dotnet
883+
uses: actions/setup-dotnet@v3
884+
with:
885+
global-json-file: global.json
886+
887+
- name: Install emscripten (Windows)
888+
shell: pwsh
889+
run: |
890+
git clone https://github.com/emscripten-core/emsdk.git $env:USERPROFILE\emsdk
891+
cd $env:USERPROFILE\emsdk
892+
.\emsdk install 4.0.21
893+
.\emsdk activate 4.0.21
894+
872895
- name: Smoketest C# AOT build (NativeAOT-LLVM)
896+
shell: pwsh
873897
run: |
874-
cd modules/sdk-test-cs
875-
EXPERIMENTAL_WASM_AOT=1 dotnet publish -c Release
876-
test -f bin/Release/net8.0/wasi-wasm/publish/StdbModule.wasm
898+
$env:EXPERIMENTAL_WASM_AOT = "1"
899+
if (Test-Path "$env:USERPROFILE\emsdk\emsdk_env.ps1") {
900+
& "$env:USERPROFILE\emsdk\emsdk_env.ps1" | Out-Null
901+
}
902+
dotnet publish -c Release modules/sdk-test-cs
903+
if (-not (Test-Path "modules/sdk-test-cs/bin/Release/net8.0/wasi-wasm/publish/StdbModule.wasm")) {
904+
Write-Error "StdbModule.wasm not found"
905+
exit 1
906+
}
877907
878908
internal-tests:
879909
name: Internal Tests
Lines changed: 124 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,124 @@
1+
# Converting a SpacetimeDB 2.0.x project to use NativeAOT-LLVM
2+
3+
This guide provides instructions on taking an existing C# module that targets the public-released SpacetimeDB CLI, and guides you through the necessary steps to enable `NativeAOT-LLVM` use.
4+
5+
## Overview
6+
In order to use `NativeAOT-LLVM` on a C# module, we'll need to set the `EXPERIMENTAL_WASM_AOT` environment variable to `1` which SpacetimeDB will check during publishing of a module.
7+
For the module to work, we'll also need the `NuGet.Config` and `.csproj` files with the required package sources and references.
8+
9+
### Prerequisites:
10+
- **.NET SDK 8.x** (same version used by SpacetimeDB)
11+
- **Emscripten SDK (EMSDK)** installed (must contain `upstream/emscripten/emcc.bat`)
12+
- **(Optional) Binaryen (wasm-opt)** installed and on `PATH` (recommended: `version_116`)
13+
14+
## Steps
15+
16+
1. **Install EMSDK**
17+
- Download and extract the `https://github.com/emscripten-core/emsdk` release.
18+
- Example path: `D:\Tools\emsdk`
19+
20+
2. **Set environment variables**
21+
22+
```powershell
23+
$env:EXPERIMENTAL_WASM_AOT=1
24+
$env:EMSDK="D:\Tools\emsdk"
25+
```
26+
27+
3. **Ensure NuGet feed is configured**
28+
NativeAOT-LLVM packages currently come from **dotnet-experimental**:
29+
- Add the `dotnet-experimental` feed to a project-local `NuGet.Config`
30+
```xml
31+
<add key="dotnet-experimental" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-experimental/nuget/v3/index.json" />
32+
```
33+
This should be a `NuGet.Config` placed in the root directory of your module folder (next to the `.csproj`). You can simply add the above line to the `packageSources` of your existing file, or if you need to create a minimal one, you can use:
34+
35+
```xml
36+
<?xml version="1.0" encoding="utf-8"?>
37+
<configuration>
38+
<packageSources>
39+
<clear />
40+
<add key="dotnet-experimental" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-experimental/nuget/v3/index.json" />
41+
<add key="nuget.org" value="https://api.nuget.org/v3/index.json" />
42+
</packageSources>
43+
</configuration>
44+
```
45+
46+
4. **Ensure NativeAOT emits a `.wasm` output**
47+
- For LLVM AOT builds, the CLI currently accepts `dotnet.wasm` under `bin/Release/net8.0/wasi-wasm/publish/`.
48+
- In the module `.csproj`, ensure the AOT package references include:
49+
50+
```xml
51+
<ItemGroup Condition="'$(EXPERIMENTAL_WASM_AOT)' == '1'">
52+
<PackageReference Include="Microsoft.NET.ILLink.Tasks" Version="8.0.0-*" Condition="'$(ILLinkTargetsPath)' == ''" />
53+
<PackageReference Include="Microsoft.DotNet.ILCompiler.LLVM" Version="8.0.0-*" />
54+
<PackageReference Include="runtime.$(NETCoreSdkPortableRuntimeIdentifier).Microsoft.DotNet.ILCompiler.LLVM" Version="8.0.0-*" />
55+
</ItemGroup>
56+
```
57+
58+
The contents of your `.csproj` should look something like this:
59+
60+
```xml
61+
<Project Sdk="Microsoft.NET.Sdk">
62+
<PropertyGroup>
63+
<TargetFramework>net8.0</TargetFramework>
64+
<RuntimeIdentifier>wasi-wasm</RuntimeIdentifier>
65+
<ImplicitUsings>enable</ImplicitUsings>
66+
<Nullable>enable</Nullable>
67+
</PropertyGroup>
68+
<ItemGroup>
69+
<PackageReference Include="SpacetimeDB.Runtime" Version="2.0.*" />
70+
</ItemGroup>
71+
<ItemGroup Condition="'$(EXPERIMENTAL_WASM_AOT)' == '1'">
72+
<PackageReference Include="Microsoft.NET.ILLink.Tasks" Version="8.0.0-*" Condition="'$(ILLinkTargetsPath)' == ''" />
73+
<PackageReference Include="Microsoft.DotNet.ILCompiler.LLVM" Version="8.0.0-*" />
74+
<PackageReference Include="runtime.$(NETCoreSdkPortableRuntimeIdentifier).Microsoft.DotNet.ILCompiler.LLVM" Version="8.0.0-*" />
75+
</ItemGroup>
76+
</Project>
77+
```
78+
79+
- **NU1504 warning**: Because the runtime targets also add these LLVM packages, you may see a duplicate PackageReference warning. It is non-blocking.
80+
81+
5. **(Optional) Install wasm-opt (Binaryen)**
82+
This step is optional, but provides performance improvements, and therefore is recommended.
83+
- Download Binaryen `https://github.com/WebAssembly/binaryen/releases/tag/version_116` for Windows and extract it, e.g. `D:\Tools\binaryen`.
84+
- Add `D:\Tools\binaryen\bin` to `PATH`.
85+
- Verify:
86+
87+
```powershell
88+
wasm-opt --version
89+
```
90+
91+
6. **Publish module**
92+
- Use the SpacetimeDB CLI to publish from the module directory.
93+
- With `EXPERIMENTAL_WASM_AOT=1`, publish should attempt LLVM AOT.
94+
- Ensure the local server is running if publishing to `local`.
95+
96+
## Troubleshooting
97+
98+
### Package source mapping enabled
99+
If you have **package source mapping** enabled in `NuGet.Config`, you must add mappings for the LLVM packages or restores will fail.
100+
Place the following in `NuGet.Config` inside the `configuration` section:
101+
```xml
102+
<packageSourceMapping>
103+
<packageSource key="bsatn-runtime">
104+
<package pattern="SpacetimeDB.BSATN.Runtime" />
105+
</packageSource>
106+
<packageSource key="SpacetimeDB.Runtime">
107+
<package pattern="SpacetimeDB.Runtime" />
108+
</packageSource>
109+
<packageSource key="dotnet-experimental">
110+
<package pattern="Microsoft.DotNet.ILCompiler.LLVM" />
111+
<package pattern="runtime.*" />
112+
</packageSource>
113+
<packageSource key="nuget.org">
114+
<package pattern="*" />
115+
</packageSource>
116+
</packageSourceMapping>
117+
```
118+
119+
### wasi-experimental workload install fails
120+
If the CLI cannot install the `wasi-experimental` workload automatically, install it manually:
121+
122+
```powershell
123+
dotnet workload install wasi-experimental
124+
```

crates/bindings-csharp/Runtime/Runtime.csproj

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@
1212
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
1313
<RootNamespace>SpacetimeDB</RootNamespace>
1414
<EmitCompilerGeneratedFiles>true</EmitCompilerGeneratedFiles>
15+
<RestoreAdditionalProjectSources Condition="'$(EXPERIMENTAL_WASM_AOT)' == '1'">https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-experimental/nuget/v3/index.json;$(RestoreAdditionalProjectSources)</RestoreAdditionalProjectSources>
1516
</PropertyGroup>
1617

1718
<ItemGroup>
@@ -25,6 +26,13 @@
2526
<ProjectReference Include="../Codegen/Codegen.csproj" ReferenceOutputAssembly="false" />
2627
</ItemGroup>
2728

29+
<!-- These must be explicit package dependencies so NuGet consumers can resolve the LLVM toolchain. -->
30+
<ItemGroup Condition="'$(EXPERIMENTAL_WASM_AOT)' == '1'">
31+
<PackageReference Include="Microsoft.DotNet.ILCompiler.LLVM" Version="8.0.0-*" IncludeAssets="All" />
32+
<PackageReference Include="runtime.$(NETCoreSdkPortableRuntimeIdentifier).Microsoft.DotNet.ILCompiler.LLVM" Version="8.0.0-*" IncludeAssets="All" />
33+
<PackageReference Include="Microsoft.NET.ILLink.Tasks" Version="8.0.0-*" Condition="'$(ILLinkTargetsPath)' == ''" IncludeAssets="All" />
34+
</ItemGroup>
35+
2836
<ItemGroup>
2937
<None Include="README.md" Pack="true" PackagePath="" />
3038
<None Include="build/*" Pack="true" PackagePath="build" />

crates/bindings-csharp/Runtime/build/SpacetimeDB.Runtime.targets

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,9 @@
11
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
22

3+
<Import
4+
Project="$(PkgMicrosoft_DotNet_ILCompiler_LLVM)\build\Microsoft.DotNet.ILCompiler.LLVM.targets"
5+
Condition="'$(EXPERIMENTAL_WASM_AOT)' == '1' and '$(ILCompilerTargetsPath)' == '' and '$(PkgMicrosoft_DotNet_ILCompiler_LLVM)' != '' and Exists('$(PkgMicrosoft_DotNet_ILCompiler_LLVM)\build\Microsoft.DotNet.ILCompiler.LLVM.targets')" />
6+
37
<ItemGroup Condition="'$(EXPERIMENTAL_WASM_AOT)' == '1'">
48
<NativeLibrary Include="$(MSBuildThisFileDirectory)..\bindings.c" />
59
<UnmanagedEntryPointsAssembly Include="SpacetimeDB.Runtime" />
@@ -42,10 +46,6 @@
4246
<WasmImport Include="spacetime_10.4!datastore_index_scan_point_bsatn" />
4347
<WasmImport Include="spacetime_10.4!datastore_delete_by_index_scan_point_bsatn" />
4448

45-
<PackageReference Include="Microsoft.DotNet.ILCompiler.LLVM" Version="8.0.0-*" />
46-
<PackageReference Include="runtime.$(NETCoreSdkPortableRuntimeIdentifier).Microsoft.DotNet.ILCompiler.LLVM" Version="8.0.0-*" />
47-
<PackageReference Include="Microsoft.NET.ILLink.Tasks" Version="8.0.0-*" Condition="'$(ILLinkTargetsPath)' == ''" />
48-
4949
<CustomLinkerArg Include="-DEXPERIMENTAL_WASM_AOT" />
5050
</ItemGroup>
5151

modules/sdk-test-cs/sdk-test-cs.csproj

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,5 +9,9 @@
99
<ProjectReference Include="../../crates/bindings-csharp/Codegen/Codegen.csproj" OutputItemType="Analyzer" ReferenceOutputAssembly="false" />
1010
<ProjectReference Include="../../crates/bindings-csharp/Runtime/Runtime.csproj" />
1111
</ItemGroup>
12-
12+
<ItemGroup Condition="'$(EXPERIMENTAL_WASM_AOT)' == '1'">
13+
<PackageReference Include="Microsoft.NET.ILLink.Tasks" Version="8.0.0-*" Condition="'$(ILLinkTargetsPath)' == ''" />
14+
<PackageReference Include="Microsoft.DotNet.ILCompiler.LLVM" Version="8.0.0-*" GeneratePathProperty="true" />
15+
<PackageReference Include="runtime.$(NETCoreSdkPortableRuntimeIdentifier).Microsoft.DotNet.ILCompiler.LLVM" Version="8.0.0-*" />
16+
</ItemGroup>
1317
</Project>

sdks/csharp/tools~/write-nuget-config.sh

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,8 @@ cat >NuGet.Config <<EOF
1515
<configuration>
1616
<packageSources>
1717
<clear />
18+
<!-- Experimental NuGet feed for Microsoft.DotNet.ILCompiler.LLVM packages -->
19+
<add key="dotnet-experimental" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-experimental/nuget/v3/index.json" />
1820
<!-- Local NuGet repositories -->
1921
<add key="Local SpacetimeDB.BSATN.Runtime" value="${SPACETIMEDB_REPO_PATH}/crates/bindings-csharp/BSATN.Runtime/bin/Release" />
2022
<!-- We need to override the module runtime as well because the examples use it -->
@@ -30,6 +32,11 @@ cat >NuGet.Config <<EOF
3032
<packageSource key="Local SpacetimeDB.Runtime">
3133
<package pattern="SpacetimeDB.Runtime" />
3234
</packageSource>
35+
<!-- Experimental packages for NativeAOT-LLVM compilation -->
36+
<packageSource key="dotnet-experimental">
37+
<package pattern="Microsoft.DotNet.ILCompiler.LLVM" />
38+
<package pattern="runtime.*" />
39+
</packageSource>
3340
<!-- Fallback for other packages (e.g. test deps). -->
3441
<packageSource key="nuget.org">
3542
<package pattern="*" />
@@ -43,6 +50,8 @@ cat >"${SPACETIMEDB_REPO_PATH}/NuGet.Config" <<EOF
4350
<configuration>
4451
<packageSources>
4552
<clear />
53+
<!-- Experimental NuGet feed for Microsoft.DotNet.ILCompiler.LLVM packages -->
54+
<add key="dotnet-experimental" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-experimental/nuget/v3/index.json" />
4655
<!-- Local NuGet repositories -->
4756
<add key="Local SpacetimeDB.BSATN.Runtime" value="crates/bindings-csharp/BSATN.Runtime/bin/Release" />
4857
<!-- We need to override the module runtime as well because the examples use it -->
@@ -58,6 +67,11 @@ cat >"${SPACETIMEDB_REPO_PATH}/NuGet.Config" <<EOF
5867
<packageSource key="Local SpacetimeDB.Runtime">
5968
<package pattern="SpacetimeDB.Runtime" />
6069
</packageSource>
70+
<!-- Experimental packages for NativeAOT-LLVM compilation -->
71+
<packageSource key="dotnet-experimental">
72+
<package pattern="Microsoft.DotNet.ILCompiler.LLVM" />
73+
<package pattern="runtime.*" />
74+
</packageSource>
6175
<!-- Fallback for other packages (e.g. test deps). -->
6276
<packageSource key="nuget.org">
6377
<package pattern="*" />

0 commit comments

Comments
 (0)