What is Antirion?
Antirion is an open-source AI gateway that sits between applications and model providers, routing LLM requests through a single API. It runs as a Rust binary with an admin console included, and can be deployed on one node or as a cluster. It supports clients using OpenAI, Anthropic, Gemini or Bedrock APIs, including official SDKs, LangChain and the Vercel AI SDK. It is designed for teams that want to manage model access, spending, safety and request records within their own infrastructure.
Antirion Features:
Provider Translation: Antirion translates between OpenAI, Anthropic, Gemini and Bedrock API formats so a client can call providers using different API dialects.
Routing Controls: Route requests with fallback chains, weighted splits, least-cost or least-latency rules, tiers and classifiers, with retries, hedging and circuit breakers.
Spend Management: Set rate limits and budgets in dollars or tokens for organizations, teams, projects, keys and models, with projections and a usage ledger.
Request Guardrails: Apply PII, secrets, regex, JSON schema, tool-permission, moderation and LLM-judge checks to requests, responses and streams, with options to block, redact, flag or divert.
Request Observability: Inspect routes, attempts, failure reasons, tokens, costs, cache status and guardrail verdicts, and send telemetry to Prometheus, OTLP or log sinks.
Deployment Options: Run a standalone node with an embedded store, or deploy a cluster using PostgreSQL and Valkey, with Docker, Compose, Kubernetes or a Linux binary.
Antirion Benefits:
One Client Setup: Teams can connect existing compatible SDKs and frameworks through one gateway endpoint instead of configuring each application separately for different provider APIs.
More Routing Choice: Fallbacks, retries and routing rules give platform teams ways to respond to provider failures and direct requests according to cost, latency or other criteria.
Clearer Spend: Budgets, projections and a ledger help teams track model usage and set spending limits at several organizational levels.
Centralized Oversight: Request details and guardrail verdicts give teams a shared place to review how calls were routed, what they cost and why a policy acted.
Infrastructure Control: Running Antirion in a team's own infrastructure keeps requests on their network between applications and providers, without adding a separate console deployment.
Use Cases:
Multi-Provider Access: Use one gateway endpoint when an application needs to call models from several providers or an OpenAI-compatible server such as vLLM.
Provider Failover: Configure fallback chains, retries or circuit breakers for applications that need defined routing behavior when a provider request fails.
Team Budgeting: Set and monitor model-use budgets for separate teams, projects or API keys when several groups share provider access.
Sensitive Data Checks: Apply PII or secret detection and redaction to model requests, responses or streams before they pass through the gateway.
Gateway Operations: Review request routes, costs and errors, then export telemetry for platform teams monitoring model traffic across their systems.

