[SDKv2] Unify the WinML and non-WinML flavors into one package per SDK#824
Draft
bmehta001 wants to merge 13 commits into
Draft
[SDKv2] Unify the WinML and non-WinML flavors into one package per SDK#824bmehta001 wants to merge 13 commits into
bmehta001 wants to merge 13 commits into
Conversation
The macOS ORT staging renamed the shipped libonnxruntime.dylib to a versioned name and symlinked the bare name back — the reverse of the Linux layout sitting right beside it, and an extra rename for no benefit. Keep the library under its shipped name (libonnxruntime.dylib) and add only the soname symlink (libonnxruntime.1.dylib -> libonnxruntime.dylib), exactly mirroring the Linux libonnxruntime.so / libonnxruntime.so.1 handling. Both names still resolve: foundry_local loads ORT by its soname install_name, GenAI dlopens the unversioned name. Net one symlink instead of a rename plus symlink. Applied in the C++ build (CMakeLists.txt) and the JS published install (install-native.cjs); the JS runtime preload (native.ts) now lists the shipped name first with the soname as fallback, matching the Linux ordering. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Remove the FOUNDRY_LOCAL_USE_WINML build flag and the WinML/non-WinML native split. WinML 2.x is reg-free, delay-loaded, and degrades gracefully when the DLL is absent, so there is no reason to gate it: find_package(WinMLEpCatalog) now runs unconditionally on Windows (if(WIN32)) and is skipped elsewhere. All downstream behavior already keyed off WinMLEpCatalog_FOUND, so the EP-catalog link, the FOUNDRY_LOCAL_HAS_EP_CATALOG define, and the WinML DLL post-build copy are unchanged. - CMakeLists.txt: drop the option; gate the find_package on WIN32. - build.py: drop --use_winml; keep --winml_sdk_version as an optional version override (CMake decides inclusion by platform). - FindOnnxRuntime.cmake: remove the dead FOUNDRY_LOCAL_USE_WINML message block. - nuget/pack.py: the runtime package always forwards the WinML DLL when present (no logic change — it already did); comments/help updated, single package id. - build_and_test_all.ps1: drop the -UseWinml variant switch. The native runtime NuGet (Microsoft.AI.Foundry.Local.Runtime) now ships the WinML DLL on Windows automatically; the separate .Runtime.WinML package is gone. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The C# layer is a pure P/Invoke wrapper with the same net8.0;net9.0 TFM for both flavors, so the WinML/non-WinML split only differed by package id and which native Runtime package it referenced. Collapse to one package: - Remove the UseWinML property, the win-only RuntimeIdentifiers override, and the WinML override PropertyGroup (PackageId/AssemblyName Microsoft.AI.Foundry.Local.WinML). - Reference the single Microsoft.AI.Foundry.Local.Runtime (which now ships the WinML DLL on Windows); drop the .Runtime.WinML conditional reference. - Tests: drop UseWinML; net462 coverage now keys only on Windows. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The base and WinML wheels shared identical ORT/GenAI pins and source; only the distribution name differed (set via FL_PYTHON_PACKAGE_NAME). With one flavor: - _build_backend: drop the FL_PYTHON_PACKAGE_NAME name-rewrite; keep only the ORT/GenAI pin rewriting from deps_versions.json (rename the helper accordingly, drop the now-unused os import). - installer.py: drop the --winml flag; always (re)install foundry-local-sdk. - pyproject.toml: drop the WinML-variant comment. The WinML DLL is bundled on Windows by the native artifact the wheel stages, so the single wheel carries it automatically (no packaging change needed). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The published JS package previously shipped only foundry_local and never the WinML EP catalog DLL, so WinML hardware EPs never worked out of the box. Bundle Microsoft.Windows.AI.MachineLearning.dll into prebuilds/ on Windows (copied as an optional sibling — skipped with a warning if absent), matching the single-package WinML-included flavor used by the C#/Python SDKs. ORT/GenAI are still fetched by the install-native.cjs postinstall hook. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The Windows native build now always includes the reg-free WinML 2.x runtime, so there is no second flavor to build, pack, or test. Remove the duplicate variant stages and parameters across the v2 pipeline: - stages-sdk-v2: invoke C# and Python stages once (drop the winml invocations). - stages-build-native: drop the cpp_build_win_*_winml stages and the cpp_pack_nuget_winml pack; the single cpp-nuget now carries WinML on Windows. - stages-cs / stages-python: drop the variant parameter and the winml stages; artifacts lose the -base/-winml suffix (cs-sdk-v2, python-sdk-<rid>). - steps-build-windows: always prefetch WinML and stage the WinML DLL; drop the useWinml branches (build.py auto-enables WinML on Windows). - steps-build/test-cs, steps-build/test-python: drop the isWinML parameter and the now-no-op /p:UseWinML; net462 coverage keys only on Windows. - steps-pack-nuget: drop the variant parameter; pack one Runtime package. - steps-build-js: stage Microsoft.Windows.AI.MachineLearning.dll into the Windows prebuild dir so the npm package bundles WinML (excluded from ESRP signing — it is Microsoft-signed). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The single SDK package now bundles the WinML runtime on Windows, so the samples no longer need the OS-conditional split between Microsoft.AI.Foundry.Local.WinML (Windows) and Microsoft.AI.Foundry.Local (other platforms). Collapse every sample csproj to one unconditional package reference, drop the .WinML PackageVersion from Directory.Packages.props, and update the READMEs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…vors The single canonical package now bundles WinML on Windows, so the docs and sample manifests no longer describe a separate WinML SKU/wheel/npm package or the UseWinML / --winml / FL_PYTHON_PACKAGE_NAME switches: - README, sdk_v2 cs/python READMEs, DEVELOPMENT.md, CppPortGuide.md, EpDetectionPlan.md: install one package; WinML acceleration is built in on Windows. - sdk_v2-pipeline-plan.md: collapse the variant stages/artifacts/graph to one flavor; correct decision 10 — the Windows native stage now stages the delay-loaded Microsoft.Windows.AI.MachineLearning.dll into the single nupkg. - samples (js package.json, python requirements.txt): reference the single foundry-local-sdk package; drop the optional/platform-split winml pins. - samples-integration-test.yml: drop the now-dead WinML pack step (the samples no longer reference the .WinML package — this was the NU1202 failure source). - build_and_test_all.ps1 / pyproject.toml: drop the dead FL_PYTHON_PACKAGE_NAME plumbing; the PEP 517 backend now only rewrites ORT pins. - memories/repo/cs-local-packages.md: one pack command, one package. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
With the WinML and non-WinML package flavors merged into a single canonical package per SDK, there is no separate WinML install command to offer. Drop the per-SDK Windows/WinML button (and its winmlId/winmlCommand fields) and the now redundant Cross-platform badge, leaving one install command per SDK. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Once a FoundryLocalManager exists, ONNX Runtime's process-wide teardown races with Node's environment teardown on a natural exit and crashes (access violation / heap corruption) — a long-standing ORT-on-exit issue, not specific to this SDK. It reproduces as far back as the first JS SDK commit; the crash is avoided only by releasing the Manager and then leaving via process.exit(), which skips Node's graceful teardown. Track live managers and replicate that automatically: on 'beforeExit' dispose them and exit explicitly; on 'exit' dispose so a direct process.exit() releases the ORT environment before the C runtime tears the ORT libraries down. Natural exit, process.exit(), and dispose()+exit are now all clean, and the JS test workers no longer segfault on teardown. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
The #821 web-server vision sample was added with the old WinML/non-WinML flavor split (net9.0-windows10.0.18362.0 + Microsoft.AI.Foundry.Local.WinML on Windows, base package elsewhere). Collapse it to the unified pattern used by every other C# sample: a single net9.0 target and a single central-managed Microsoft.AI.Foundry.Local PackageReference. Builds against the unified package alongside all 14 C# samples. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR unifies the previously separate WinML and non-WinML SDK flavors into a single package per SDK (C#, Python, JS). The reg-free WinML 2.x EP catalog is now always enabled on Windows at the native layer, so consumers no longer choose between a base SKU and a .WinML SKU; other platforms are unaffected. The change spans the native C++ build, all three v2 SDK packaging paths, the macOS ORT dylib staging (simplified to mirror Linux), the CI pipeline matrix, C# samples (central package management), and docs/website install instructions. It also lands a JS process-exit fix to avoid the ORT-on-teardown crash after a native Manager exists.
Changes:
- Native/build: WinML EP catalog gated on
if(WIN32)(always on for Windows); WinML runtime DLL bundled/staged unconditionally; macOS ORT dylib layout reversed to keep the unversioned file as the real binary with a versioned soname symlink (Linux parity). - Packaging per SDK: dropped the
.WinMLC# package +UseWinMLswitch, thefoundry-local-sdk-winmlPython wheel +--winmlinstaller flag +FL_PYTHON_PACKAGE_NAMEbuild-backend name override, and the JS WinML install flavor; CI matrix collapsed to one build per SDK. - JS runtime fix + docs/www:
FoundryLocalManagernow disposes live managers onbeforeExit/exitand exits explicitly; docs and the download dropdown describe a single install command per SDK.
Reviewed changes
Copilot reviewed 83 out of 83 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
sdk_v2/cpp/CMakeLists.txt |
Gates WinML EP catalog on if(WIN32); reverses macOS ORT dylib to real-file + versioned symlink |
sdk_v2/js/src/foundryLocalManager.ts |
Adds liveManagers tracking + beforeExit/exit handlers to avoid ORT teardown crash |
sdk_v2/js/script/install-native.cjs |
Keeps unversioned ORT dylib/so, adds versioned soname symlink with copy fallback (mirrors CMake) |
sdk_v2/js/src/detail/native.ts |
Reorders darwin ORT candidate basenames to match the new layout |
sdk_v2/python/_build_backend/__init__.py |
Removes name-override logic; _maybe_patch_name → _rewrite_version_pins (ORT/GenAI pins only) |
sdk_v2/python/_native/installer.py |
Drops --winml flag; always installs foundry-local-sdk |
sdk_v2/python/pyproject.toml / README.md |
Single wheel; updated build-backend comments and install docs |
.pipelines/v2/templates/* |
Collapses WinML/non-WinML matrix into one build per SDK; unconditional WinML DLL staging |
samples/cs/Directory.Packages.props + *.csproj |
Central package management; removes Microsoft.AI.Foundry.Local.WinML references |
www/src/lib/components/download-dropdown.svelte |
Removes WinML install option; one command per SDK |
Comment on lines
+14
to
16
| <ItemGroup> | ||
| <PackageReference Include="Microsoft.AI.Foundry.Local" /> | ||
| </ItemGroup> |
…s, backend
The flavor split is gone from sdk_v2 code, but stale references to it lingered
in comments and docs. Reword them to describe the unified reality — WinML is
always on on Windows, so the distinction is platform, not flavor:
- cpp/CMakeLists.txt, cpp/test/CMakeLists.txt, js/script/copy-native.mjs:
"no WinML SKU" / "WinML build" / "non-WinML build" -> platform wording.
- cpp/cmake/FindOnnxRuntime.cmake: drop the now-vacuous "for both flavors".
- .pipelines/v2: steps-build-windows.yml (Standard-vs-WinML build split -> one
Windows build), steps-test-python.yml ("Both variants" -> per-job wheel),
and the plan docs ("per variant"; deferred "WinML variant" bullet).
- build_and_test_all.ps1: tighten the net9.0/net462 test comment.
Also simplify python/_build_backend: with deps_versions_winml.json gone there is
a single deps file, so drop the _STD suffix and the deps_file parameter threaded
through _read_versions/_patch_pyproject_text. Verified it still rewrites all four
ORT/GenAI pins from deps_versions.json.
sdk_v2-only; the V1 sdk/ + .pipelines/v1 generation keeps its flavor split.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…lias
onnxruntime-genai's InitApi() (built with dlopen on macOS-non-framework and
Linux, per its CMakeLists) honors the ORT_LIB_PATH env var: it dlopens that exact
file before its hardcoded unversioned libonnxruntime.{dylib,so} fallback. Set it
in the JS addon loader to the ORT we ship, so the package no longer needs the
unversioned alias purely for GenAI:
- detail/native.ts: applyOrtLibPath() sets ORT_LIB_PATH (only if unset) to the
resolved ORT in the prebuilds dir, or the configured libraryPath, before the
addon -- and thus GenAI's lazy InitApi -- loads. No-op on Windows (ORT is linked
directly, no dlopen). ortCandidateBasenames now prefers the versioned soname.
- install-native.cjs: rename the extracted unversioned ORT to the versioned soname
(libonnxruntime.1.dylib / .so.1) that libfoundry_local records, instead of
symlinking and keeping both -- one file, no symlink.
- copy-native.mjs: dev staging copies the versioned soname only, matching ship.
macOS CI (js_test_osx_arm64) runs npm ci (install-native postinstall) + vitest
with FOUNDRY_TEST_DATA_DIR set, so the real-model path that loads GenAI exercises
this end to end. The C++ build keeps its own ORT symlink for the C++ test harness,
which does not set ORT_LIB_PATH.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Collapses the separate WinML and non-WinML SDK flavors into a single package per SDK (C#, Python, JS). Consumers no longer pick between a
.WinMLSKU and a base SKU: on Windows the reg-free WinML 2.x EP catalog is always available, and other platforms are unaffected.What changed
Microsoft.AI.Foundry.Localpackage; drop the.WinMLSKU and theUseWinMLswitch..pipelines/v2) — collapse the WinML/non-WinML matrix into one build per SDK.Directory.Packages.props.beforeExit/exithandlers), avoiding the ORT-teardown crash on graceful Node exit.Notes / follow-ups
main(picks up deletes sdk_legacy folder #813sdk_legacyremoval, Add vision (image understanding) web server samples for C#, JS, Rust #821 vision samples, Bump the npm_and_yarn group across 2 directories with 1 update #810 npm bump, Updated "Load a model" Step For Python Setup #806).