What is GPT-Load?
GPT-Load is a self-hosted gateway for routing AI requests through multiple providers, channels, and credentials from one entry point. It is designed for developers and businesses that want to manage API keys and subscription accounts together while keeping their applications on supported native protocols. A management console handles provider setup, routing policy, request visibility, and usage tracking, with data stored in SQLite, MySQL, or PostgreSQL.
GPT-Load Features:
Unified Gateway: Give applications one base URL and AccessKey while configuring providers, accounts, models, credentials, and routing policies in the management console.
Protocol Support: Serve OpenAI, Anthropic, Gemini, and other supported interfaces, including chat, responses, images, embeddings, reranking, and audio endpoints.
Provider Channels: Connect official APIs, cloud platforms, model services, subscription accounts such as Codex and Claude, and custom OpenAI-compatible relays.
Credential Scheduling: Route requests across multiple credentials with configurable weights, session affinity, retries, cooldowns, and blacklisting.
Request Monitoring: Review request logs, channel and credential health, routes, usage statistics, token categories, cache hit rate, and cost estimates.
Self-Hosted Storage: Run the gateway with Docker Compose and store its data in application-managed SQLite or an external MySQL or PostgreSQL database, with local credential encryption.
GPT-Load Benefits:
Simpler Client Setup: A single base URL and AccessKey can spare application teams from configuring a separate gateway connection for each provider.
Fewer Disruptions: Retries, cooldowns, blacklisting, and alternate credentials help keep requests moving when a credential is overloaded or fails.
Flexible Routing: Teams can adjust how traffic is distributed without changing the application’s provider connection each time routing needs change.
Clearer Usage Picture: Centralized logs, health views, and usage statistics make it easier to investigate requests and understand where usage and estimated costs come from.
Infrastructure Control: Self-hosting gives operators control over deployment and data storage while keeping encrypted channel credentials within their own setup.
Use Cases:
Multi-Provider Apps: Route an application’s OpenAI, Anthropic, or Gemini requests through one gateway while retaining the client protocol it already uses.
Shared Credentials: Manage several API keys or subscription accounts for an application that needs a common access point for its model requests.
Traffic Distribution: Set weights and session affinity when sending requests across multiple credentials or channels for a service.
Failure Handling: Configure retries and cooldowns for an AI-backed product that needs to handle unavailable or unhealthy credentials.
Usage Review: Inspect request logs, token categories, cache hit rates, and cost estimates when operating or troubleshooting an AI service.