Skip to content

fix(prompts): skip cache write when cache_ttl_seconds is 0 - #2114

Open
Mo0nl19ht wants to merge 1 commit into
langfuse:mainfrom
Mo0nl19ht:fix/prompt-cache-ttl-zero-no-store
Open

Mo0nl19ht wants to merge 1 commit into
langfuse:mainfrom
Mo0nl19ht:fix/prompt-cache-ttl-zero-no-store

Conversation

@Mo0nl19ht

@Mo0nl19ht Mo0nl19ht commented Nov 8, 2026 •

Copy link
Copy Markdown

What does this PR do?

get_prompt documents cache_ttl_seconds=0 as disabling caching, but it only skips the cache read. The fetched prompt is still written to the cache with a zero TTL, so a TTL-0 call:

  • replaces the entry that default-TTL callers cached for the same prompt, and
  • makes the next default-TTL call return that entry instead of fetching (as fresh within the same second, as stale with a background refresh after that).

_fetch_prompt_and_update_cache now skips the cache write when the TTL is 0. Eviction on a 404 is unchanged, so the cache still drops prompts the server no longer has. The LANGFUSE_PROMPT_CACHE_DEFAULT_TTL_SECONDS default path is not changed.

Type of change

  • Bug fix
  • New feature
  • Breaking change
  • Refactor
  • Documentation update
  • Tooling, CI, or repo maintenance

Verification

List the main commands you ran:

uv run --frozen ruff check .                                  # All checks passed!
uv run --frozen ruff format --check .                         # 595 files already formatted
uv run --frozen mypy langfuse --no-error-summary              # no issues
uv run --frozen pytest -n auto --dist worksteal tests/unit    # 1016 passed, 2 skipped

The new tests/unit/test_prompt.py::test_ttl_zero_fetch_does_not_write_to_cache covers an empty cache, the following default-TTL call, and an existing entry. It fails on main (the TTL-0 fetch leaves a PromptCacheItem in the cache) and passes with this change. E2E and live-provider suites were not run; the change needs no server.

Checklist

  • I self-reviewed the diff using code_review.md.
  • I added or updated tests for behavior changes.
  • I updated docs, examples, or .env.template if needed. (Not needed; the docstring already describes this behavior.)
  • I did not hand-edit generated files; if generated files changed, I used the upstream regeneration path.
  • I did not commit secrets or credentials.

RetriggerView in GreptileConfidence Score: 5/5

The PR appears safe to merge.

Summary

Zero-TTL calls now fetch prompts without adding or replacing cached entries.

  • TTL-zero prompt fetches leave the shared cache unchanged.

Reviews (5) · Last reviewed commit: "fix(prompts): skip cache write when cach..." · Reviewed by Greptile

get_prompt documents cache_ttl_seconds=0 as disabling caching, but the
fetched prompt was still stored with a zero TTL. That replaced entries
other callers had cached and let the next default-TTL call return the
TTL-0 result instead of fetching. Skip the cache write when the TTL is 0,
and keep evicting the key on a 404 so the cache never holds a prompt the
server no longer has.

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@CLAassistant

CLAassistant commented Nov 8, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@chrikrah chrikrah left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Mo0nl19ht this matches the docstring ("Disables caching if set to 0"), and from my side it can be merged. The new test fails with only client.py reverted:

$ uv run --frozen pytest tests/unit/test_prompt.py -q   # 49452e6, env from .env.template
27 passed in 2.36s

$ (same, client.py from 8e6c60d, head tests)
FAILED tests/unit/test_prompt.py::test_ttl_zero_fetch_does_not_write_to_cache
# not run: the e2e suite against a server

After a TTL-0-only call the cache is simply cold, so the existing fallback guidance for a first call already covers it. Nothing more needed in this PR.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants