Guide

Caching

Two layers that stack — a semantic response cache and provider-side prompt caching — to cut cost without changing your requests.

Overview

Two caching layers

Anoman caches at two independent layers that stack. A semantic cache can return a complete response with no provider call at all, and provider-side prompt caching makes the input portion of a live call far cheaper. Every completion tells you which (if any) was applied.

Semantic cache (Anoman)

Caches the complete response for an identical request. On a hit, the response is returned from Redis with zero provider call — a 100% saving. Opt-in per API key; auto-enabled on batch/overflow traffic.

Signalled by X-Anoman-Cache: semantic

Provider prompt cache

For large system prompts, Anoman auto-injects cache markers so the provider only re-processes the changed part. A live call is still made — the response is always fresh — but input processing is far cheaper. Applied transparently to paid tiers.

Signalled by X-Anoman-Cache: provider

Semantic cache

Per-customer, opt-in, zero-cost hits

The semantic cache is partitioned per customer — the cache key is hash(customer_id + model + messages), so you never see another customer's cached response. Enable it per key in Settings; TTL is set by tier (Pro ~5 min; Enterprise off by default).

A hit returns the stored response with no provider call — charged 0% weighted tokens.
Best for non-interactive, repeatable traffic: dev/test loops, repeated benchmarks, FAQ bots. Risk: stale responses.
Bypass the cache for a single request with the x-anoman-no-cache header.

# Enable the semantic cache on a key (Settings → API keys → cache toggle),
# or bypass the cache for a single request with a header:
curl https://api.anoman.io/v1/chat/completions \
  -H "Authorization: Bearer anm-sk-..." \
  -H "Content-Type: application/json" \
  -H "x-anoman-no-cache: 1" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [{"role": "user", "content": "Summarize the Q3 report."}]
  }'

Provider prompt cache

Cheaper input on live calls

When a system prompt exceeds the provider's minimum, Anoman injects a cache marker so the provider reuses the already-processed prefix. The response is always fresh — only input processing is discounted. Thresholds and discounts vary by provider:

Provider	Minimum	Input discount
Anthropic	2,048+ tokens	up to 90% off input (5-min cache)
OpenAI	automatic	up to 90% off the cached prefix
Google	32,768+ tokens	up to 97.5% off input

You don't configure anything — markers are injected transparently on paid tiers whenever a prompt qualifies.

Discounts & savings

How caching shows up in billing

Caching feeds directly into weighted-token billing. A semantic cache hit is charged 0%; a provider prompt-cache hit is charged at about 10% of normal weight — a ~90% weighted-token reduction (see Quota & overage).

# Cache + batch savings, aggregated for your account.
curl https://api.anoman.io/anoman/v1/usage/savings \
  -H "Authorization: Bearer anm-sk-..."

# {
#   "cache_savings_usd": "3.41",     # semantic + provider prompt cache
#   "batch_savings_usd": "12.08",
#   "window": "30d"
# }

Aggregate cache + batch savings appear on the Usage page and in the _anoman.cache block on every response.

Reference

The X-Anoman-Cache header

Every completion carries an X-Anoman-Cache header — none, provider, or semantic — plus a matching _anoman.cache block with the savings for that request.

HTTP/1.1 200 OK
x-anoman-cache:  semantic
x-anoman-tier:   pro
content-type:    application/json

# semantic  → served from Anoman's Redis cache, zero provider call (100% saved)
# provider  → live provider call, but cached input prefix (up to 90–97.5% off input)
# none      → no cache applied

Related: Batch routing (another ~50% saving on non-interactive work) and Quota & overage (how discounts map to your allowance).

See your savings

Cache and batch savings are tracked per account on the Usage page.

Open Usage Back to docs