GitHub PR review bot.
Claude Sonnet 4.6 + tool calling + a GitHub webhook. Posts inline comments with severity buckets. ~250 lines of Python.
Architecture — What we're building
- GitHub fires a
pull_requestwebhook on PR open / sync. - Our FastAPI handler verifies the GitHub signature + kicks off a background review.
- The review loop calls Claude Sonnet with the PR diff + two tools:
read_fileandpost_review_comment. - The model loops: read files, think, post comments, repeat until it's satisfied (or hits the 15-step cap).
- Anoman's guardrails inspect every prompt + response. Tool policy enforces that the bot can only call our two declared tools.
1. Tool definitions — What the bot can do
Two tools. The schemas are sent to the model on every call. Set tool_choice: "auto" and let it decide.
Tool definitions
2. Review loop — Agent loop with a step cap
Standard tool-using agent loop: keep calling until the model returns text without tool calls (or we hit the step cap to bound cost).
Review loop
3. GitHub webhook entrypoint — Fire the bot from a webhook
GitHub webhook entrypoint (FastAPI)
Same signature pattern as Anoman's own webhooks — see /docs/webhooks for the verification recipe.
4. Production hardening — Handle guardrail blocks + rate limits
Don't retry on guardrail blocks (they'll fail again the same way). Do retry on 503/504 with exponential backoff. Stop fast on budget exceeded.
Robust call with retry + guardrail awareness
Production deploy — Real-world tips
- Per-key budget — set a monthly USD cap on your bot's Anoman key. Runaway PRs hit 402 instead of draining you.
- Policy group for tools — create a dedicated policy group with
read_file+post_review_commentin the allowlist. Now if Anthropic ever returns an unexpected tool, it's blocked. - Agent ID tag — pass
anoman-agent-id: code-reviewerheader so the dashboard groups all calls under one agent. - Vision for screenshots — extend to call
gpt-4oon PRs containing image attachments — useful for design-system PRs.
More recipes for agentic patterns.
Streaming chat, batch document analysis, RAG with caching.