feat: add opt-in exact-match caching for LLM responses - #404

Merged

rapids-bot[bot] merged 21 commits into

NVIDIA:mainfrom

zhongxuanwang-nv:feat/response-cache-stage1

Jul 31, 2026

Commits on Jul 14, 2026

feat(response-cache): opt-in exact-match LLM response cache
zhongxuanwang-nv
committed
96df1d9

Commits on Jul 16, 2026

Merge remote-tracking branch 'upstream/main' into feat/response-cache-stage1

zhongxuanwang-nv
and
claude
committed
3b2f565
fix(response-cache): address code review findings
zhongxuanwang-nv
committed
813e711

Commits on Jul 21, 2026

Merge upstream/main into feat/response-cache-stage1
zhongxuanwang-nv
committed
4d0d5e3

Commits on Jul 22, 2026

Merge upstream/main into feat/response-cache-stage1
zhongxuanwang-nv
committed
be0df9a
fix(response-cache): address review feedback
zhongxuanwang-nv
committed
e2a0c79

Commits on Jul 27, 2026

fix(response-cache): address remaining review feedback
zhongxuanwang-nv
committed
d7205ce

Commits on Jul 28, 2026

fix(response-cache): address CodeRabbit review findings
zhongxuanwang-nv
committed
316d77a

Commits on Jul 29, 2026

fix(response-cache): bound Redis initialization and probes
zhongxuanwang-nv
committed
6521a5b
fix(response-cache): remove unsafe skip keys
zhongxuanwang-nv
committed
4a2a594
fix(go): preserve response cache defaults
zhongxuanwang-nv
committed
84c69d6

Commits on Jul 30, 2026

fix(response-cache): require explicit trust boundary
zhongxuanwang-nv
committed
0f73b58
fix(gateway): isolate concurrent upstream attempts
zhongxuanwang-nv
committed
b8911da
merge: sync response cache branch with main
zhongxuanwang-nv
committed
8d614b1
fix(response-cache): preserve routing and response fidelity
zhongxuanwang-nv
committed
87a283b
fix: preserve cache partitions and provider failures
zhongxuanwang-nv
committed
c62b2c7
Merge upstream/main into feat/response-cache-stage1
zhongxuanwang-nv
committed
601c28e
test: remove stream flush timing race
zhongxuanwang-nv
committed
b1f7b46
fix(gateway): keep captured upstream errors opaque
zhongxuanwang-nv
committed
29be012
fix(ci): remove redundant gateway test conversion
zhongxuanwang-nv
committed
20f6fa2
docs: defer response cache guidance to follow-up
zhongxuanwang-nv
committed
5975aa6