Provider-side prompt cache discounts repeated prefixes by 90%. Response cache (opt-in, exact match, rolling out): repeats of an identical deterministic request can be served from Anoman's cache; billed at the normal rate.

Different problems, different solutions

Layer A — Response cache

Anoman-side, opt-in, rolling out — Caches the complete response for an exact repeat of a deterministic request. On hit: served from Anoman's cache instead of a new provider call, billed at the normal rate (same price and weighted-token quota as the original request).

Best for: deterministic repeats such as FAQ bots, eval re-runs and classifier calls at temperature: 0.

Layer B — Provider prompt cache

Upstream-side, transparent — Caches the large system prompt prefix server-side at Anthropic / OpenAI / Google. On hit: still calls upstream but charges 10% on the cached tokens.

Best for: agents with multi-thousand-token system prompts.

Response cache — how it works

Status — rolling out, off by default.
Opt-in per API key — while it rolls out it is enabled per key on request (contact support); there is no dashboard toggle yet. There is no per-request opt-in header.
Exact match only — same model, messages, parameters and guardrail policy. No fuzzy or semantic matching; any difference makes it a different request.
Deterministic requests only — a request is cached only when it uses temperature: 0, or sends X-Anoman-Cache: force (any other value of that header does nothing).
Partitioned and short-lived — entries are partitioned per customer, encrypted at rest, and live at most 1 hour.
Billing — a hit is billed at the normal rate: same price and same weighted-token quota as the original request. It is not free and carries no discount.
Streaming — a hit on a stream: true request replays as a normal SSE stream.
Not cached — free tier, pass / free-bucket calls, document calls, PII tokenize / synthetic modes, reasoning requests (thinking / reasoning_effort), and n > 1.

{
  "choices": [{
    "message": {"role": "assistant", "content": "The capital of Indonesia is Jakarta."}
  }],
  "_anoman": {
    "cache": { "hit": true, "type": "response" }
  }
}

Provider prompt cache — applied to long system prompts

When your request has a system prompt > the provider's minimum, Anoman automatically injects the cache control markers so the upstream caches it server-side. Subsequent requests with the same prefix get the cache discount.

Provider	Minimum prompt	Discount	Cache lifetime
Anthropic	2,048 tokens (~8K chars)	90% off cached input	5 min
OpenAI	Any length (automatic)	90% off cached prefix	~5–10 min
Google Gemini	32,768 tokens (~130K chars)	97.5% off cached input	Explicit CachedContent API

// You sent a 4000-token system prompt that matched a recent cached prefix.
// Provider charges 10% on the cached tokens; output is full price.
{
  "choices": [{
    "message": {"role": "assistant", "content": "..."}
  }],
  "usage": {
    "prompt_tokens": 4127,
    "completion_tokens": 380,
    "prompt_tokens_details": { "cached_tokens": 4000 }  // ← 4000 were cached
  },
  "_anoman": {
    "cache": { "hit": true, "type": "provider" },
    "weighted_tokens": 1600,   // ← discounted from ~14,300 without cache
    "cost_usd": "0.0048"
  }
}

Provider cache is on automatically — no header opt-in. The discount is applied to your billing without any client-side change. To verify it's firing, check usage.prompt_tokens_details.cached_tokens on the response.

Headers that change cache behavior

Header	Values	Effect
`X-Anoman-Cache` (request)	`force`	Cache this request even if it isn't `temperature: 0` (response cache must be on for the key). Any other value does nothing.
`X-Anoman-No-Cache`	`1 / true / yes`	Skip the response cache for this request (provider cache still applies).
`x-anoman-no-provider-cache`	`1 / true`	Skip injecting provider cache markers (rare; useful for A/B testing cost).

# Response cache is opt-in per API key (enabled per key on request while it rolls out).
# With it on, temperature-0 requests are cached automatically.
# To cache a non-zero-temperature request, send the force header:
curl https://api.anoman.io/v1/chat/completions \
  -H "Authorization: Bearer anm-sk-..." \
  -H "X-Anoman-Cache: force" \
  -d '...'

# Always go fresh, even if a cache entry would match.
curl https://api.anoman.io/v1/chat/completions \
  -H "Authorization: Bearer anm-sk-..." \
  -H "X-Anoman-No-Cache: true" \
  -d '...'

When each fires

Workload	Response	Provider	Why
FAQ chatbot (`temperature: 0`)	✓	—	Same questions repeat. Serve exact repeats from cache.
Agentic coding	—	✓	Multi-turn with big system prompt. Cache prefix.
RAG over docs	—	✓	Same retrieved chunks across turns. Provider cache.
Eval / benchmark (`temperature: 0`)	✓	—	Re-running the same prompts repeatedly.
Creative writing	No	—	User wants variation — only deterministic requests are cached.

How caching shows up in billing

Caching feeds directly into weighted-token billing. A response-cache hit is billed at the normal rate (same price and weighted tokens as the original request); a provider prompt-cache hit is charged at about 10% of normal weight, a ~90% weighted-token reduction (see Quota & overage).

# Cache + batch savings for your account over a window
# (cacheSavings, batchSavings and totalSpend).
curl https://api.anoman.io/anoman/v1/usage/savings \
  -H "Authorization: Bearer anm-sk-..."

The X-Anoman-Cache header

Every completion carries an X-Anoman-Cache response header (none, provider or response) plus a matching _anoman.cache block.

X-Anoman-Cache: response

# response  → served from Anoman's response cache, billed at the normal rate
# provider  → live provider call with a cached input prefix
# none      → no cache applied

Related: Batch routing (another ~50% saving on non-interactive work).

See your savings

The dashboard Usage page shows Cache savings and Batch savings as separate columns. Provider-cache savings are baked into the per-request cost_usd already. Response-cache hits are billed at the normal rate, so they add nothing to your savings.

Inspect cache hits in real time.

Live Feed marks every cache hit with a CACHED pill.

Caching

On this page