Concepts · Guardrails Pipeline

Defense-in-depth on every request.

6 guardrails run in a fixed order before and after the upstream call. Fail-fast on blocks. Pass-through results on every successful response.

The pipeline

Request flow

┌────────────────────────────────────────────────────────────────┐
│  Request arrives at /v1/chat/completions                       │
└────────────────────────────────────────────────────────────────┘
              │
              ▼
   1. Auth + rate limit       ← 401/402/403/429 fail fast (~1 ms)
              │
              ▼
   2. PRE-CALL GUARDRAILS     ← fail-fast on any block (~30-80 ms)
       ┌────────────────────┐
       │ a. Injection        │  DeBERTa v3 classifier
       │ b. PII              │  Presidio + spaCy NER
       │ c. Content          │  Keyword + classifier
       │ d. Policy (tools)   │  OPA-style allow/deny lists
       └────────────────────┘
              │
              ▼ (if all pass)
   3. Cache check             ← semantic cache hit returns 200 (~5 ms)
              │
              ▼ (miss)
   4. Routing decision        ← realtime or batch?
              │
              ▼
   5. Upstream LLM call       ← provider latency (~500ms-5s typical)
              │
              ▼
   6. POST-CALL GUARDRAILS    ← (~20-40 ms)
       ┌────────────────────┐
       │ e. Response content │  Filter unsafe completions
       │ f. Response PII     │  Mask leaked PII in output
       └────────────────────┘
              │
              ▼
   7. Token metering + trace  ← Redis Stream + ClickHouse
              │
              ▼
        Response to client

Every request runs through the pipeline in this fixed order. There is no way to skip it; the master toggle to bypass guardrails (rarely used, audit-logged) is a per-policy-group setting that an admin must explicitly enable.

Pre-call guardrails

What each one does

a. Prompt injection

DeBERTa v3 classifier (ProtectAI, Apache 2.0) scoring every user-role message. Score > 0.85 (configurable) returns 403. Catches jailbreak prompts, system prompt extraction, and tool-call hijacking. Runs in ~40 ms on CPU.

b. PII detection

Presidio + spaCy NER recognize 9 entity types (email, phone, credit card, SG NRIC, ID NIK, IP, URL, person, address). Behavior depends on the policy group’s PII mode: redact (replace with <EMAIL> placeholders), tokenize (replace with reversible tokens and restore in the response), synthetic (replace with realistic fake values), or block (return 403).

c. Content moderation

Keyword denylist (admin-configurable) plus a multilingual classifier covering EN + Bahasa Indonesia. Per-category sensitivity (sexual, hate, self-harm, etc.). Returns 403 on hit.

d. Policy / tool enforcement

When the request includes tools, every tool name is checked against the API key’s policy group (allow/deny lists). MCP tool RBAC also applies here for connected MCP servers.

Post-call guardrails

After the upstream returns

e. Response content filter — same classifier as pre-call but applied to the model’s output. Catches accidental unsafe completions that the model produced despite a safe prompt.
f. Response PII scanner — masks any PII in the response. Useful when the upstream might echo back something sensitive even though the prompt was clean.

If a post-call guardrail blocks, the response is replaced with a 403 — the customer pays for the tokens (upstream already ran) but doesn’t see the unsafe content.

Reading results

Pass vs block on the wire

On a 200 response, every guardrail’s result is surfaced in _anoman.guardrails and as headers prefixed x-anoman-guardrail-*. Useful for logging the false-positive rate without inferring from blocks.

{
  "choices": [...],
  "_anoman": {
    "guardrails": {
      "injection":        { "status": "pass", "score": 0.02 },
      "pii":              { "status": "pass" },
      "content":          { "status": "pass" },
      "policy":           { "status": "pass" },
      "response_content": { "status": "pass" }
    }
  }
}

On a block, the response is 403 with type: "guardrail_error" and a specific code identifying which guardrail fired:

// Response body
{
  "error": {
    "type": "guardrail_error",
    "code": "prompt_injection",
    "message": "Request blocked by prompt injection detector (score 0.94 > threshold 0.85)."
  }
}

// Headers
x-anoman-guardrail-injection: blocked score=0.94

Latency budget

What does it cost in time?

Pre-call guardrails (target p95):
  Injection (DeBERTa)        ~40 ms
  PII (Presidio + spaCy)     ~25 ms
  Content (keyword + ML)     ~5 ms
  Policy (allow/deny)        ~10 ms
                            ────────
  Total pre-call             <100 ms

Post-call guardrails (target p95):
  Response content           ~15 ms
  Response PII               ~20 ms
                            ────────
  Total post-call            <40 ms

Anoman total overhead:       <150 ms p95
  (vs typical LLM call latency of 500ms – 5s — negligible)

The pre-call pipeline runs sequentially with fail-fast — when injection blocks, we don’t bother running PII / content / policy. On a typical pass, the full pipeline adds <100 ms before the upstream call begins.

Configuration

Per-key + per-policy-group overrides

Each API key is assigned a policy group with: PII mode, injection threshold, content moderation severity, tool allow/deny list, and a master enable toggle. Per-key overrides can opt out of specific guardrails (audit-logged) — useful for trusted internal services where you don’t want PII redaction modifying the prompt.

See the Guardrails settings page in the dashboard, or the /docs/guardrails reference for the full config surface.

See every guardrail firing live.

Live Feed shows the pre/post-call result on every request as it happens.

Open Live Feed All concepts