Skip to content

Load an installed package by id through POST /v1/models/load - #1098

Merged
0xShug0 merged 2 commits into
0xShug0:mainfrom
HamSanders:feat/load-by-package-id
Oct 30, 2026
Merged

0xShug0 merged 2 commits into
0xShug0:mainfrom
HamSanders:feat/load-by-package-id

Conversation

@HamSanders

Copy link
Copy Markdown
Contributor

Following #1093: with the native model manager, POST /v1/models/load accepts "package_id" in place of "path". As you scoped it, the management handler resolves the installed package to its model path and then calls the existing handle_model_load unchanged. The runtime only ever sees a path.

Change

  • Routing: only in builds with AUDIOCPP_HAS_NATIVE_MODEL_MANAGER, the load route goes through handle_package_model_load (in runtime.cpp and parallel_runtime.cpp).
    • It rewrites package_id to the absolute path and calls handle_model_load.
    • Requests that send path, and builds without the manager, behave exactly as before.
  • Lookup: PackageManager::load_target(id) returns the package's family, its path under the current models root, and whether it's installed. ModelInstaller passes it through. The request is rewritten in server_ui.cpp (resolve_ui_package_model_load).
  • Which path an id loads: the same one the WebUI loads (packageModelPath in webui/native/src/lib/catalog.ts):
    • a GGUF package's .gguf file;
    • the package directory for non-GGUF packages and for auk, lfm2_audio, liveavatar, minimax_music3 and yue2;
    • dit.gguf or dit_int8.gguf for minimax_h3.
  • family is filled in from the package when it's left out. A different family is rejected.
  • Errors:
    • unknown id → 404 unknown package id: …;
    • known but not installed → 404 package is not installed: …;
    • path and package_id together → 400;
    • empty package_id → 400;
    • family mismatch → 400.
  • Docs: app/server/api.md gets a "Loading an installed package by id" section. Test: tests/unittests/test_server_model_installer.cpp covers the resolution rule.

14 files, +241 −1.

Validation

CPU build with the native model manager, audiocpp_server --ui --ui-management --backend cpu, Windows 11, run against an existing models folder.

POST /v1/ui/models-root {"path":"…/models/audiocpp"} → 200
POST /v1/models/load {"id":"sep-pkg","package_id":"htdemucs_q8_0","task":"sep"} → 200 {"id":"sep-pkg","loaded":true,"reconfigured":false}
GET  /v1/models → …{"id":"sep-pkg","family":"htdemucs","task":"sep","loaded":true,"path":"…\HTDemucs-GGUF\htdemucs-q8_0.gguf"}
POST /v1/tasks/run {"model":"sep-pkg","request":{"audio_base64":<2 s stereo WAV>}} → 200, stems: drums, bass, other, vocals
POST /v1/models/load {"id":"x","package_id":"no_such_package","task":"sep"} → 404 unknown package id: no_such_package
POST /v1/models/load {"id":"x2","package_id":"kokoro_82m_bf16"} → 404 package is not installed: kokoro_82m_bf16
POST /v1/models/load {…"package_id":"htdemucs_q8_0","path":"…"} → 400 send either path or package_id, not both
POST /v1/models/load {…"package_id":"htdemucs_q8_0","family":"kokoro_tts"} → 400 package htdemucs_q8_0 is for family htdemucs
POST /v1/models/load {…"package_id":""} → 400 package_id must be a non-empty string
POST /v1/models/load {"id":"sep-path","path":"…/htdemucs-q8_0.gguf","family":"htdemucs","task":"sep"} → 200, and /v1/tasks/run on it → 200, same 4 stems

The three server unit tests pass, including the new resolution test. The tests compile the server without the native manager, so that branch builds too.

A question for you

package_model_path hard-codes which families load a directory instead of a file, mirroring packageModelPath in the WebUI, so the two lists can drift as families are added. If you'd rather the package spec declared its load target (for example an entry-file field in the catalogue), I'm happy to rework it that way. I kept this version to match current WebUI behaviour exactly.

Packages that need session options (lfm2_audio, minimax_music3) still need the client to send them, as with path loading. api.md says so.

@0xShug0

0xShug0 commented Oct 27, 2026

Copy link
Copy Markdown
Owner

@HamSanders Thanks for the PR. One minor issue is both runtimes acquire model_installer_mutex_ before checking for package_id. Ordinary path loads therefore wait behind unrelated installer work, even for resident models, outside the model’s busy timeout. Check for package_id first; path-only requests should bypass the installer.

The real blocker is the boundary issue: the package manager now duplicates the UI’s model-loading rules and hard-codes model-specific behavior. This needs a framework-level solution so both consume the same package metadata instead of maintaining separate mappings. I’ll fix this issue myself and circle back to this PR afterward.

@0xShug0

0xShug0 commented Oct 28, 2026

Copy link
Copy Markdown
Owner

@HamSanders My laziness came back to bite me hard 😓 The loading-boundary refactor (#1127) is now merged. Could you rebase this PR onto main and use package_info() followed by resolve_package_load() instead of the hard-coded family rules? Please forward the resolved path, family, load options and session options to the existing load handler. This keeps model-specific loading decisions in each model, while the package manager supplies metadata and the server handles the request. Also, check for package_id before acquiring model_installer_mutex_ in both runtimes, so path-based loads bypass the installer.

With the native model manager, POST /v1/models/load accepts "package_id"
in place of "path". The management handler reads the package with
PackageManager::package_info() and asks the family's loader through
ModelRegistry::resolve_package_load() for the path, family, load options
and session options, then hands the rewritten request to the existing
handle_model_load. Model-specific loading decisions stay in each model.

The package_id check runs before model_installer_mutex_ is taken, in both
runtimes, so path-based loads never touch the installer.

- Unknown or not-installed package: 404 not_found.
- Both path and package_id, an empty package_id, or a family that does
  not match the package: 400.
- Resolved load/session options fill in what the request leaves out;
  keys the request sends win.
- Requests with path, and servers built without the native model
  manager, are unchanged.

Validated on a CPU build: htdemucs_q8_0 loaded by id and ran a
/v1/tasks/run separation; minimax_music3_q4_0 loaded by id with no
client session options and got its three component GGUFs from the
resolver; path-based loading still works; server_model_installer_test
passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@HamSanders
HamSanders force-pushed the feat/load-by-package-id branch from 002e0f2 to 92f9a27 Compare October 30, 2026 02:19
@HamSanders

Copy link
Copy Markdown
Contributor Author

Thanks for #1127, this is much cleaner. Rebased onto main and reworked:

  • /v1/models/load with package_id now calls package_info(), then registry.resolve_package_load(), and forwards the resolved path, family, load options and session options to the existing load handler. The hard-coded family rules and load_target() are gone from the package manager.
  • The package_id check happens before model_installer_mutex_ in both runtimes, so path-based loads never touch the installer.
  • Values the request sends in load_options / session_options win over the resolved ones; a mismatched family is still a 400.

Checked on a CPU build: htdemucs_q8_0 loaded by id and ran a separation; minimax_music3_q4_0 loaded by id with no client session options and picked up its three component GGUFs from the resolver; path loads are unchanged; server_model_installer_test passes.

@0xShug0 0xShug0 added the ready label Oct 30, 2026
@0xShug0
0xShug0 merged commit 350670d into 0xShug0:main Oct 30, 2026
12 checks passed
@0xShug0

0xShug0 commented Oct 30, 2026

Copy link
Copy Markdown
Owner

@HamSanders Made a minor change to fix the issue where malformed session_options are silently replaced with defaults and return HTTP 200. Thanks and PR merged!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants