Repository navigation
Load an installed package by id through POST /v1/models/load - #1098
Conversation
|
@HamSanders Thanks for the PR. One minor issue is both runtimes acquire The real blocker is the boundary issue: the package manager now duplicates the UI’s model-loading rules and hard-codes model-specific behavior. This needs a framework-level solution so both consume the same package metadata instead of maintaining separate mappings. I’ll fix this issue myself and circle back to this PR afterward. |
|
@HamSanders My laziness came back to bite me hard 😓 The loading-boundary refactor (#1127) is now merged. Could you rebase this PR onto main and use |
With the native model manager, POST /v1/models/load accepts "package_id" in place of "path". The management handler reads the package with PackageManager::package_info() and asks the family's loader through ModelRegistry::resolve_package_load() for the path, family, load options and session options, then hands the rewritten request to the existing handle_model_load. Model-specific loading decisions stay in each model. The package_id check runs before model_installer_mutex_ is taken, in both runtimes, so path-based loads never touch the installer. - Unknown or not-installed package: 404 not_found. - Both path and package_id, an empty package_id, or a family that does not match the package: 400. - Resolved load/session options fill in what the request leaves out; keys the request sends win. - Requests with path, and servers built without the native model manager, are unchanged. Validated on a CPU build: htdemucs_q8_0 loaded by id and ran a /v1/tasks/run separation; minimax_music3_q4_0 loaded by id with no client session options and got its three component GGUFs from the resolver; path-based loading still works; server_model_installer_test passes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
002e0f2 to
92f9a27
Compare
|
Thanks for #1127, this is much cleaner. Rebased onto main and reworked:
Checked on a CPU build: |
|
@HamSanders Made a minor change to fix the issue where malformed session_options are silently replaced with defaults and return HTTP 200. Thanks and PR merged! |
Following #1093: with the native model manager,
POST /v1/models/loadaccepts"package_id"in place of"path". As you scoped it, the management handler resolves the installed package to its model path and then calls the existinghandle_model_loadunchanged. The runtime only ever sees a path.Change
AUDIOCPP_HAS_NATIVE_MODEL_MANAGER, the load route goes throughhandle_package_model_load(inruntime.cppandparallel_runtime.cpp).package_idto the absolutepathand callshandle_model_load.path, and builds without the manager, behave exactly as before.PackageManager::load_target(id)returns the package's family, its path under the current models root, and whether it's installed.ModelInstallerpasses it through. The request is rewritten inserver_ui.cpp(resolve_ui_package_model_load).packageModelPathinwebui/native/src/lib/catalog.ts):.gguffile;auk,lfm2_audio,liveavatar,minimax_music3andyue2;dit.ggufordit_int8.ggufforminimax_h3.familyis filled in from the package when it's left out. A different family is rejected.unknown package id: …;package is not installed: …;pathandpackage_idtogether → 400;package_id→ 400;app/server/api.mdgets a "Loading an installed package by id" section. Test:tests/unittests/test_server_model_installer.cppcovers the resolution rule.14 files, +241 −1.
Validation
CPU build with the native model manager,
audiocpp_server --ui --ui-management --backend cpu, Windows 11, run against an existing models folder.The three server unit tests pass, including the new resolution test. The tests compile the server without the native manager, so that branch builds too.
A question for you
package_model_pathhard-codes which families load a directory instead of a file, mirroringpackageModelPathin the WebUI, so the two lists can drift as families are added. If you'd rather the package spec declared its load target (for example an entry-file field in the catalogue), I'm happy to rework it that way. I kept this version to match current WebUI behaviour exactly.Packages that need session options (
lfm2_audio,minimax_music3) still need the client to send them, as with path loading.api.mdsays so.