Skip to content

feat: add caller-scoped data spillover for large tool results - #92

Open
jpsiddharth2008 wants to merge 32 commits into
nitrocloudofficial:developfrom
jpsiddharth2008:feature/issue-67-data-spillover
Open

jpsiddharth2008 wants to merge 32 commits into
nitrocloudofficial:developfrom
jpsiddharth2008:feature/issue-67-data-spillover

Conversation

@jpsiddharth2008

Copy link
Copy Markdown
Collaborator

Closes #67.

Large tool results can consume a significant portion of the model’s context.
This PR adds an opt-in interceptor that stores oversized results separately
and returns a bounded preview, a resource URI, and the original payload size.
The caller can retrieve the complete result through MCP resources/read.

Results at or below the configured threshold are returned unchanged.

Changes

  • Added a SpilloverStore protocol with save, retrieve, delete, and expiry
    cleanup operations.
  • Added MemorySpilloverStore with TTL expiry, byte limits, and LRU eviction.
  • Added FileSpilloverStore with persistent ownership and expiry metadata,
    atomic writes, capacity limits, and path traversal and symlink checks.
  • Added DataSpilloverInterceptor, supporting a configurable byte threshold,
    storage backend, and resource URI prefix.
  • Extended @use_interceptors(...) to accept configured interceptor instances
    while preserving existing class-based dependency injection.
  • Added previews for lists, dictionaries, and strings, including head items,
    total counts, and truncated text within a 2 KiB preview budget.
  • Integrated stored results with MCP resource reads. Access is restricted to
    the verified caller or, for anonymous callers, the originating MCP session.
    Missing, expired, and unauthorized results return resource-not-found.
  • Added support for background task results and advertised output schemas
    that allow either the original result or a spillover envelope.
  • Added public API exports, README examples, and an implementation walkthrough.

The default spill threshold is 64 KiB, with a 10 MiB per-payload limit,
100 MiB store capacity, and five-minute TTL. File-store roots are intended
for use by one store instance in one process.

Testing

Feature tests

38 passed, 1 skipped, covering:

  • Threshold boundaries and UTF-8 byte counting.
  • Unchanged small results and bounded previews for large or nested data.
  • Complete payload retrieval through registered MCP resource handlers.
  • Caller, tenant, issuer, and anonymous-session isolation.
  • TTL expiry, deletion, LRU eviction, and storage capacity limits.
  • File persistence, cleanup, path traversal, and symlink rejection.
  • Configured interceptor instances, class-based registration, custom URI
    prefixes, background tasks, OAuth identity, and widget-transformer bypass.

The real filesystem symlink test was skipped because this Windows account
cannot create symlinks. A separate simulated symlink-rejection test passed.

Large-payload checks

24 additional checks passed across memory and file storage, including:

  • 1 MiB, 5 MiB, and 10 MiB text.
  • A list containing 100,000 rows.
  • A dictionary containing 50,000 entries.
  • Approximately 6 MiB of nested data.
  • Multibyte emoji and escaped text.
  • Correct rejection of payloads exceeding the 10 MiB limit, including 20 MiB.

For each spilled result, checks verified byte-for-byte retrieval, the preview
size limit, and denial of access to another caller. These checks exercised
registered MCP handlers in-process rather than a network client.

Full SDK suite

The full suite with coverage reported 868 passed, 1 failed, 1 skipped
and 85.8% coverage.

Full-suite runs encountered Windows localhost connection-aborted errors
in OAuth discovery tests. The entire OAuth test module passed separately,
and an unchanged main-branch baseline also passed its full suite. The cause
of the combined-run failures remains unresolved.

git diff --check passed.

manish-wekan and others added 30 commits September 10, 2026 18:05
…ase PR.

Keep pyproject.toml and uv.lock in sync, skip a bump while a version is still unpublished, and only hit the PyPI environment when an upload will actually run.
Bring the bump/publish workflows in line with develop (via main) so the PR can merge.
…ure/release-automation

Automate version bumps and PyPI publishes from main
.env values were written over os.environ unconditionally, so the PORT=3000
that `nitrostack init` puts in .env overrode the PORT a host passes when
starting the server. NitroStudio starts a second project on 3001; it bound
3000 anyway and died with "address already in use" (NITSTD-222).

Keys already in the process environment are now kept, as with
python-dotenv and Node's dotenv. .env still fills keys the process
doesn't set, and defaults stay lowest.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…process-env-wins-over-dotenv

ConfigService: process environment wins over .env
publish.yml only uploads through PyPI trusted publishing, which has never
run here, and the Bump version workflow can't open its release PR (the org
doesn't allow GitHub Actions to create pull requests). The script covers
both: optionally makes release/<next> from origin/main (--bump), checks the
branch matches origin, the lockfile matches and the version is new, runs the
tests, builds, twine-checks, uploads after asking, verifies a clean install
from PyPI, and opens the release PR. `--tag` tags origin/main after the merge.
`--dry-run` stops before anything leaves the machine.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Run from a temp copy: --bump switches branches, which can rewrite the
  script while bash is still reading it.
- --bump plans from origin/main's files in a scratch folder and checks
  everything before switching, so a failed check no longer leaves you on a
  detached origin/main.
- Only a 404 counts as "not on PyPI"; offline or PyPI errors stop the run
  instead of passing the "version is new" check.
- No terminal and no --yes means "no" instead of a silent exit.
- Tag the uploaded commit right after the upload (no separate step that
  could tag later, unreleased merges on main). --tag stays for catching up
  untagged releases and prefers the release branch head.
- A failed PR creation after the upload prints the manual step.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The README is the PyPI project page. Rewrite it for 0.4.x: building
blocks (tools, resources, prompts, guards, auth, caching, rate limits,
tasks, events, health), env precedence, templates, clients.

tests/test_readme_examples.py compiles every python block, checks every
nitrostack import exists, and runs the Quick Start and Building Blocks
examples through the testing harness so the page can't drift.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e/release-script

Add scripts/release.sh for releasing to PyPI by hand
…/readme-refresh

docs: refresh README (PyPI page) for the current SDK
`nitrostack-py upgrade` failed on every project `init` creates: the
dependency pattern also matched `nitrostack = { path = "vendor/nitrostack" }`
in [tool.uv.sources] and produced an invalid pyproject.toml. Spec rewrites
are now limited to [project].dependencies, and the pattern skips paths
ending in /nitrostack.

Fixing that alone isn't enough: a vendored project runs vendor/nitrostack
whatever spec it declares, and uv accepts a spec the vendored copy doesn't
satisfy. For vendored projects, upgrade now replaces vendor/nitrostack
with the target release (copied from the running CLI when it is that
release, otherwise the PyPI wheel, checked against its sha256). The new
tree is built beside the old one and swapped in, so a failed download
leaves the project as it was.

init's next steps said NitroStudio can't open Python projects; it can.
The pizzaz open-shops hint is printed only for the pizzaz template.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…upgrade-vendored

Fix upgrade on vendored projects; correct init's NitroStudio note
`init` vendors the SDK into vendor/nitrostack. `validate` walked it as
project code, and nitrostack/testing applies @mcp_app to a class defined
inside a function, so every vendored project that imported
nitrostack.testing failed with "uses @mcp_app but no decorated
application class was found after import". Compose's typecheck runs
`validate`, so a NitroStudio build could not get past it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…validate-skip-vendored-sdk

validate: skip the vendored SDK
Align Python pack with the TypeScript CLI: optional inclusion of .env
files in the wheel, with a warning that lists paths without printing values.
…/pack-include-env

pack: add --include-env
…/producthunt-badge

docs: add Product Hunt featured badge to README
…/producthunt-top-post-badge

docs: update Product Hunt badge to nitrocloudofficial#1 Product of the Day top-post badge
@jpsiddharth2008
jpsiddharth2008 changed the base branch from main to develop October 27, 2026 12:32
@AadarshM07

AadarshM07 commented Oct 27, 2026 •

Copy link
Copy Markdown
Collaborator

Hi @jpsiddharth2008, Could you rebase this PR branch with the develop branch and remove the additional commit dependencies?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[interceptors] Data spillover for large tool results

5 participants