⚡ Enable Anthropic prompt caching for eligible models (#135) (#138)
## Summary
Adds request-root `cache_control: {type: "ephemeral"}` for models whose
card advertises `cache_control` in `supported_openai_params`. Anthropic
then maintains a single auto-advancing cache breakpoint.
**Measured over 70 Claude Opus 5 turns: 90.2% cache read, 76% cost
reduction ($12.20 vs $51.78 uncached).**
| Session style | Turns | Read rate | Cost | Uncached | Saved |
| ---------------------- | ----- | --------- | ----- | -------- | -----
|
| Long agentic loop | 38 | 94.2% | $6.40 | $33.92 | 81% |
| Interactive multi-turn | 32 | 82.5% | $5.80 | $17.86 | 68% |
## Design
- **Eligibility:** `cache_control` in `supported_openai_params`.
Deliberately *not* `supports_prompt_caching` — that's also true for
providers with incompatible contracts.
- **Ordering is load-bearing:** `finalizeRequestBody()` enforces bypass
→ strip → prompt-cache. Applying it before the strip would silently drop
the field.
- **Kept separate from gateway caching.** `extra_body.cache` (LiteLLM
response-cache bypass) and `cache_control` (Anthropic prompt cache) are
unrelated despite the shared word — see #135 for the history here.
- **Retry-safe:** rejection paths strip `cache_control` from both chat
and `/responses` shapes, so a retry doesn't repeat the same failure.
- **Ineligible models are fully unstamped** to avoid cross-provider
leakage.
## Also included
- Anthropic root usage fields (`cache_creation_input_tokens` /
`cache_read_input_tokens`) mapped into the token snapshot — previously
ignored, so cache activity was invisible in telemetry.
- `finalizeRequestBody()` de-duplicates a byte-identical tail across
both request builders.
- `xvfb-run -a` in `test-coverage.mjs` — without it the suite cannot
start when a stale X lock is present.
## Validation
| Gate | Result |
| ------------- | --------------------------- |
| compile | clean |
| lint | 0 errors |
| format | clean |
| test:coverage | **1029 passing** (main: 1012) |
| Coverage | main | branch |
| ---------- | ------ | ---------- |
| Statements | 91.10% | **91.20%** |
| Branches | 82.37% | 82.37% |
| Functions | 88.77% | **88.92%** |
| Lines | 91.10% | **91.20%** |
## Deliberately excluded
- **Per-block (Path 2) stamping** — extension providers receive zero
host breakpoints (verified: 23 requests, 470 tool-result parts, none at
any depth). Upstream: microsoft/vscode#312940.
- **Extended 1h TTL** — modelled at 2.0x vs 1.25x write cost: 0.1%
better on real traffic. Not worth it.
- **Own-stamp placement for turn boundaries** — boundary busts proved
intermittent and the trigger isn't isolated. Upstream:
microsoft/vscode#323641, #323642.
Closes #135 D
David Tai committed
8229581ba74b9aed46e69fd0358c1c8d4887fdca
Parent: f0a19af
Committed by GitHub <noreply@github.com>
on 8/24/2026, 5:25:41 PM