feat(models): enrich /api/models with runtime context length and LiteLLM model_info metadata by spelech · Pull Request #107 · spelech/LocalLLMServerManager

spelech · 2026-11-03T03:15:49Z

Exposes top-level context_length, max_input_tokens, max_output_tokens, and max_tokens on /api/models.
Injects runtime context window matching running instance (via /api/ps) or configured \DefaultOllamaContextLength\ (default: 24,576 / 24k).
Injects nested model_info metadata matching LiteLLM Model Manager specifications (\max_input_tokens, \max_output_tokens, \context_length, \key, \mode).
Adds a configurable Default Context Window setting in the UI under Settings -> Memory & Cache Lifetimes.
Adds test verification in \ServerEndpointsTests.cs.

…LLM model_info metadata

… and architecture

spelech added 2 commits November 2, 2026 21:15

feat(models): enrich /api/models with runtime context length and Lite…

a7ba493

…LLM model_info metadata

feat(models): make max_output_tokens dynamic based on model reasoning…

2a9f84a

… and architecture

spelech merged commit 548a550 into main Nov 3, 2026
3 checks passed

Navigation Menu