Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 9 additions & 5 deletions crates/adaptive/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,8 @@ SPDX-License-Identifier: Apache-2.0

`nemo-relay-adaptive` is the Rust companion crate for adaptive NeMo Relay
runtime behavior. Use it with `nemo-relay` when an agent runtime should learn
from observed executions, inject runtime hints, or persist adaptive state.
from observed executions, inject runtime hints, persist adaptive state, or
cache repeated LLM responses.

Adaptive behavior is installed through the same plugin system used by the core
runtime, so applications can enable it without changing their orchestration
Expand All @@ -41,9 +42,11 @@ framework.
- **`AdaptiveConfig`**: A canonical config contract for the top-level
`adaptive` plugin component.
- **Built-in component settings**: Typed config helpers for telemetry,
adaptive hints, tool parallelism, and the Adaptive Cache Governor.
- **State backends**: In-memory state by default and Redis-backed state behind
the `redis-backend` feature.
adaptive hints, tool parallelism, the Adaptive Cache Governor, and response
caching.
- **Storage backends**: In-memory backends for adaptive state and opt-in
response caching, with Redis-backed storage behind the `redis-backend`
feature.
Comment thread
coderabbitai[bot] marked this conversation as resolved.
- **Learning primitives**: Runtime helpers and learners built on NeMo Relay
events.
- **Adaptive Cache Governor (ACG) module surface**: The canonical
Expand All @@ -58,7 +61,8 @@ Install the published crate alongside the core runtime:
cargo add nemo-relay nemo-relay-adaptive
```

Enable Redis-backed state only when the application needs shared persistence:
Enable Redis-backed storage when adaptive state or the response cache needs
shared persistence:

```bash
cargo add nemo-relay-adaptive --features redis-backend
Expand Down
3 changes: 3 additions & 0 deletions docs/configure-plugins/adaptive/about.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ The plugin can coordinate:
- Adaptive hints injected into outgoing model requests.
- Tool-parallelism observation or scheduling behavior.
- Adaptive Cache Governor (ACG) prompt-cache planning.
- Opt-in response caching for repeated LLM calls.
- Component-local validation policy.

## Use Adaptive When
Expand Down Expand Up @@ -51,6 +52,8 @@ If instrumentation is not in place yet, start with
cache planning accomplishes.
- [Adaptive Hints](/configure-plugins/adaptive/adaptive-hints) explains request hint injection and how
downstream model paths can consume the hints.
- [Response Cache](/configure-plugins/adaptive/response-cache) explains the opt-in LLM response cache:
turning it on, what gets cached, and how savings are reported.
Comment thread
zhongxuanwang-nv marked this conversation as resolved.

State, telemetry, tool parallelism, and policy are whole-plugin configuration
areas. They are documented on [Adaptive Configuration](/configure-plugins/adaptive/configuration) rather
Expand Down
21 changes: 12 additions & 9 deletions docs/configure-plugins/adaptive/configuration.mdx
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: "Adaptive Configuration"
sidebar-title: "Configuration"
description: "Configure the built-in Adaptive component, including state, telemetry, hints, and ACG."
description: "Configure the Adaptive component, including state, telemetry, hints, ACG, and response caching."
position: 2
---
{/* SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
Expand All @@ -11,9 +11,10 @@ SPDX-License-Identifier: Apache-2.0 */}
Use this page when you want to configure the built-in Adaptive plugin component
as a whole. The component kind is `adaptive`.

Adaptive plugin configuration uses the generic NeMo Relay plugin document shape.
Field names stay `snake_case` in every binding and in `plugins.toml`, even when
language helper functions use language-native naming conventions.
Adaptive plugin configuration uses the generic NeMo Relay plugin document
shape. Canonical plugin documents and `plugins.toml` use `snake_case`.
Binding helpers can use language-native names and serialize them to canonical
keys.

For plugin file discovery, precedence, merge behavior, editor controls, and
gateway conflict rules, refer to
Expand All @@ -32,10 +33,12 @@ The top-level adaptive object contains:
| `adaptive_hints` | Request hint-injection behavior. |
| `tool_parallelism` | Tool scheduling observation or scheduling behavior. |
| `acg` | Adaptive Cache Governor prompt-cache planning. |
| `response_cache` | Opt-in LLM response cache for repeated managed calls. Requires a non-empty trust-domain `namespace`. |
| `policy` | Adaptive-local handling for unknown fields and unsupported values. |

The requested area pages cover [Adaptive Cache Governor (ACG)](/configure-plugins/adaptive/acg) and
[Adaptive Hints](/configure-plugins/adaptive/adaptive-hints). State, telemetry, tool parallelism, and
Dedicated pages cover [Adaptive Cache Governor (ACG)](/configure-plugins/adaptive/acg),
[Adaptive Hints](/configure-plugins/adaptive/adaptive-hints), and
[Response Cache](/configure-plugins/adaptive/response-cache). State, telemetry, tool parallelism, and
policy remain whole-plugin settings:

- Use `state.backend.kind = "in_memory"` for local experiments.
Expand Down Expand Up @@ -271,9 +274,9 @@ finally:
</Tab>

<Tab title="Node.js" language="node">
The Node.js binding exposes the built-in adaptive runtime through the adaptive
plugin component helpers. Use the Plugin Configuration example above when
activating adaptive behavior from Node.js.
The Node.js binding exposes `adaptive.AdaptiveRuntime` for manual lifecycle
ownership. Construct it from the same `adaptiveConfig`, then call `register()`,
`waitForIdle()`, and `shutdown()`.
</Tab>

<Tab title="Rust" language="rust">
Expand Down
Loading
Loading