POST · /v1/chat/completions + GET · /anoman/v1/batch

Batch jobs.

Async routing for non-interactive workloads. ~50% cheaper than realtime with 5-30 minute SLA depending on tier.

When to use batch

Pick batch when…

Latency does not matter — overnight summarization, eval suites, data enrichment pipelines, batch RAG.
Volume is high — thousands of independent requests where ~50% savings adds up.
You can poll or wait — your code path is fine with 5-30 minute response times.

Pick realtime when…

You are streaming output to a user UI.
Latency is part of the product (chat, autocomplete, voice agents).
You need stream: true — batch does not support streaming.

SLA by tier

Tier	Batch SLA	Notes
Starter	30 min	Default; overflow + opt-in
Pro	15 min	Default for new accounts
Pay-As-You-Go	15 min	Same as Pro
Enterprise	5 min	Or custom contract SLA

When a queued job hits 80% of its SLA deadline without completing, Anoman auto-promotes it to realtime at full cost. You never see a breach — but you also stop saving.

Enqueue

Same endpoint, opt-in header

Batch is opt-in per request via x-anoman-prefer-batch: true (or set default_routing: batch on the API key in the dashboard for set-and-forget). The response is 202 with a poll_url.

# Opt-in via header; existing tools/SDKs work unchanged.
curl https://api.anoman.io/v1/chat/completions \
  -H "Authorization: Bearer anm-sk-..." \
  -H "Content-Type: application/json" \
  -H "x-anoman-prefer-batch: true" \
  -d '{
    "model": "deepseek-v3",
    "messages": [{"role": "user", "content": "Summarize this 50-page doc..."}]
  }'

Poll

GET /anoman/v1/batch/{job_id}

Hit the URL returned in the enqueue response. While processing, you get 202 with a hint poll_again_in_seconds. When complete, 200 with the full chat completion result plus savings_usd.

curl https://api.anoman.io/anoman/v1/batch/job_abc123def456 \
  -H "Authorization: Bearer anm-sk-..."

Python helper

Enqueue + poll in one call

import time
import httpx

API = "https://api.anoman.io"
KEY = "anm-sk-..."

def enqueue(model, messages):
    r = httpx.post(
        f"{API}/v1/chat/completions",
        headers={
            "Authorization": f"Bearer {KEY}",
            "x-anoman-prefer-batch": "true",
        },
        json={"model": model, "messages": messages},
    )
    r.raise_for_status()
    return r.json()  # 202 with poll_url

def poll(job_id, deadline_minutes=30):
    deadline = time.time() + deadline_minutes * 60
    while time.time() < deadline:
        r = httpx.get(
            f"{API}/anoman/v1/batch/{job_id}",
            headers={"Authorization": f"Bearer {KEY}"},
        )
        r.raise_for_status()
        if r.status_code == 200:
            return r.json()  # complete
        # 202: still processing — respect poll_again_in_seconds
        wait = r.json().get("poll_again_in_seconds", 30)
        time.sleep(wait)
    raise TimeoutError(f"Job {job_id} exceeded {deadline_minutes}m deadline")

# Usage
job = enqueue("deepseek-v3", [{"role": "user", "content": "..."}])
print(f"Queued: {job['id']}, SLA: {job['sla_minutes']}m")
result = poll(job["id"])
print(result["choices"][0]["message"]["content"])
print(f"Saved: $-{result.get('savings_usd', '0')}")

Equivalent helper exists in the official Anoman SDK as client.poll_batch(job_id).

Cancel

DELETE /anoman/v1/batch/{job_id}

Queued jobs can be cancelled with a single DELETE. Once execution starts upstream, cancellation returns 409 (you are still billed for whatever has already run).

curl -X DELETE \
  https://api.anoman.io/anoman/v1/batch/job_abc123def456 \
  -H "Authorization: Bearer anm-sk-..."

# 200 — { "id": "job_abc123def456", "status": "cancelled" }
# 409 — if job already started executing

What batch does not support

Constraints

No streaming — stream:true forces realtime regardless of headers.
No realtime guarantees — the Enterprise tier realtime SLA does not apply.
Auto-promotion to realtime at 80% SLA — when SF degrades or queue is hot, we promote rather than miss the SLA. You stop saving but the job still completes on time.
Same guardrail pipeline runs before enqueue — a batch request that fails injection detection returns 403 immediately and is never queued.

See batch economics in the dashboard.

Per-month savings broken down by model + provider.

Open Usage All endpoints