Rate limits

Your spend boundary is a per-customer token cap by model tier class (weekly + monthly). RPM/TPM act as anti-flood burst guards, and token packs add headroom for planned spikes.

Token caps by model tier class

The spend boundary on flat tiers is a hard token cap per model tier class, enforced per customer over a weekly and a monthly window. A model's tier class (budget / mid / premium / ultra) is set by its price. Caps are shared across all keys on an account — N keys draw from one bucket, closing the multi-key gaming exploit older gateways have. Exceed a class's weekly or monthly cap and the request returns 402 token_quota_exceeded. A class with no row for a tier is not available on that tier.

Tokens count input + output. Starter runs budget models only. Pro adds mid + premium. Ultra-class models are Enterprise / PAYG territory. PAYG and Enterprise have no hard token cap — usage is governed by a soft internal cost-velocity alert plus your commercial terms.

A reserved Free tier (budget only, 1M wk / 3M mo) exists in schema but has no live signups today.

Custom per-customer overrides are available — contact [email protected].

RPM / TPM burst guards + inflight cap

Separate from the token cap (your spend boundary), every tier has RPM and TPM burst guards plus a per-key inflight cap. These are anti-flood limits — they protect upstream providers and your own tail latency, not your monthly budget. Tripping one returns 429 rate_limit_exceeded (or concurrent_limit_exceeded for the inflight cap).

RPM = requests / minute (rate). TPM = total tokens / minute (input + output). Max concurrent = simultaneous in-flight requests per key — headroom for concurrency, not a throughput cap.

Two different boundaries

• 402 token_quota_exceeded — you spent your weekly or monthly token cap for a model tier class. The response carries model_tier_class and window (weekly | monthly). Retrying won't help until the window rolls over — buy a token pack, upgrade your tier, or move the workload to a class you still have budget in.
• 429 rate_limit_exceeded — you tripped the RPM/TPM burst guard. Transient: back off and retry, respecting Retry-After.

Read these instead of hitting a limit

Every successful response reports your current RPM/TPM burst-guard budgets. Use them to throttle client-side before the guard fires. Token-cap exhaustion surfaces as a 402 token_quota_exceeded, distinct from the 429 rate_limit_exceeded flood guard.

200 — burst-guard headers on every response

HTTP/1.1 200 OK
x-anoman-rate-limit-rpm:    120
x-anoman-rate-remaining-rpm: 86
x-anoman-rate-limit-tpm:    500000
x-anoman-rate-remaining-tpm: 491200
x-anoman-tier:              pro
content-type:               application/json

402 — token quota exceeded (the spend boundary)

HTTP/1.1 402 Payment Required
x-anoman-tier:              pro
content-type:               application/json

{
  "error": {
    "type": "billing_error",
    "code": "token_quota_exceeded",
    "message": "Monthly token cap reached for the 'premium' model tier class.",
    "model_tier_class": "premium",
    "window": "monthly"
  }
}

429 — burst guard tripped (RPM/TPM flood)

HTTP/1.1 429 Too Many Requests
x-anoman-rate-limit-rpm:    120
x-anoman-rate-remaining-rpm: 0
retry-after:                24
x-anoman-tier:              pro
content-type:               application/json

{
  "error": {
    "type": "rate_limit_error",
    "code": "rate_limit_exceeded",
    "message": "120 RPM limit exceeded. Retry after 24s."
  }
}

429 — concurrent (inflight) cap

HTTP/1.1 429 Too Many Requests
x-anoman-inflight-cap:     150
x-anoman-inflight-current: 150
x-anoman-tier:             pro
content-type:              application/json

{
  "error": {
    "type": "rate_limit_error",
    "code": "concurrent_limit_exceeded",
    "message": "Per-key inflight cap (150) reached. Drop concurrency or upgrade tier."
  }
}

Respect Retry-After

The OpenAI / Anthropic SDKs retry automatically by default, but they ignore Retry-After in some versions. Here's an explicit pattern that picks server-suggested wait when present, exponential backoff otherwise:

Python — respect Retry-After

import time
from openai import OpenAI
from openai import RateLimitError, APIStatusError
 
client = OpenAI(base_url="https://api.anoman.io/v1", api_key="anm-sk-...")
 
def call_with_backoff(messages, model="gpt-4o-mini", max_attempts=5):
    for attempt in range(max_attempts):
        try:
            return client.chat.completions.create(model=model, messages=messages)
        except (RateLimitError, APIStatusError) as e:
            if attempt == max_attempts - 1:
                raise
            # Retry-After in seconds (server-suggested wait)
            retry_after = float(e.response.headers.get("retry-after", 0))
            # Fall back to exponential backoff with jitter
            sleep = retry_after or min(2 ** attempt, 30)
            time.sleep(sleep)

TypeScript — same pattern

import OpenAI, { APIError } from "openai";
 
const client = new OpenAI({
  baseURL: "https://api.anoman.io/v1",
  apiKey: process.env.ANOMAN_API_KEY,
});
 
async function callWithBackoff<T>(fn: () => Promise<T>, max = 5): Promise<T> {
  for (let i = 0; i < max; i++) {
    try {
      return await fn();
    } catch (err) {
      if (!(err instanceof APIError) || err.status !== 429) throw err;
      if (i === max - 1) throw err;
      const retryAfter = Number(err.headers?.["retry-after"] ?? 0);
      const sleep = retryAfter || Math.min(2 ** i, 30);
      await new Promise(r => setTimeout(r, sleep * 1000));
    }
  }
  throw new Error("unreachable");
}

Concurrent vs rate

The two 429 codes mean different things:

• rate_limit_exceeded — you tripped the RPM/TPM burst guard. Retry after the window rolls over.
• concurrent_limit_exceeded — too many in-flight at the same instant. Drop concurrency or wait for one to complete.

The inflight cap is independent of RPM/TPM. A Pro account at 120 RPM with 150 inflight cap can fire 120 sequential requests per minute or up to 150 simultaneous — but not 150 simultaneous of 5-second calls (that would burst RPM way over 120).

Why both? RPM protects upstream services from total volume. Inflight cap protects your own latency budget — too many in-flight requests waiting on the upstream queue makes everyone slower. Lower inflight = predictable tail latency.

Buy extra tokens when you need headroom

On a flat tier, when a model tier class is close to its token cap, buy a token pack to add temporary headroom to that class for a fixed duration. It's an additive bump to the class's cap — not a change to your RPM/TPM burst guards. Costs are deducted from your prepaid balance.

Python — buy a token pack

import requests
 
# Token packs add temporary headroom to a model tier class's token cap.
# Cost is deducted from prepaid balance. Pricing varies per tier — see the
# token-pack pricing table in your dashboard.
response = requests.post(
    "https://api.anoman.io/anoman/v1/burst/activate",
    headers={"Authorization": "Bearer anm-sk-..."},
    json={"model_tier_class": "premium", "extra_tokens": 5_000_000, "duration_hours": 24},
)
response.raise_for_status()
print(response.json())
# {
#   "active": true,
#   "model_tier_class": "premium",
#   "extra_tokens": 5000000,
#   "expires_at": 1740086400,
#   "cost_usd": "7.00",
#   "balance_remaining_usd": "23.00"
# }

PAYG and Enterprise don't need token packs — they have no hard token cap to begin with.

Watch your token caps in real time

The Rate Limits dashboard shows per-class token usage and RPM/TPM burst guards with sparklines.

• Open dashboard
• Back to docs