diegosouzapw/OmniRoute
OmniRoute is a free AI gateway that unifies access to over 170 AI providers, offering token compression, auto-fallback, and aggregating free tiers to provide billions of free tokens monthly.
Awesome Infra for AI › LLM Gateways & Proxies
tbphp/gpt-load
GPT-Load is a self-hosted AI gateway written in Go for managing multi-channel, multi-credential access to AI providers. It exposes a single base URL and access key to client applications while handling routing, credential management, and traffic scheduling internally. Users configure providers, accounts, credentials, models and routing policy through an embedded management UI. It supports native client protocols for OpenAI Chat Completions, OpenAI Responses, OpenAI Images, OpenAI Embeddings, Anthropic Messages and Gemini APIs, converting between supported capabilities without acting as a universal protocol translator. Built-in channels cover official providers (OpenAI, Anthropic, Gemini, xAI, Azure OpenAI, AWS Bedrock, Google Vertex AI), model services (DeepSeek, Moonshot AI, SiliconFlow, Zhipu AI, Alibaba, Volcengine, OpenRouter, Groq) and subscription-based accounts (Codex, Claude, Antigravity, Grok) with OAuth flows for the latter. The gateway shares one credential-management, scheduling and health-handling mechanism across API-key and subscription channels, applying multi-credential scheduling, automatic weighting, retries, cooldown periods, blacklisting and session affinity to reduce the impact of failing or rate-limited credentials. It records request logs, usage statistics and cost estimates, viewable through the embedded UI, alongside health status for routes and channels. Backing storage can be SQLite, MySQL or PostgreSQL, with credentials encrypted locally. Deployment is via Docker Compose, generating a management key on first start; by default the service binds only to loopback and is not exposed publicly. GPT-Load targets teams and individuals running self-hosted infrastructure who need to consolidate multiple AI provider accounts and subscriptions behind a single, observable entry point, rather than embedding provider-specific credentials and retry logic directly into client applications.
https://github.com/tbphp/gpt-load
Also in LLM Gateways & Proxies
Mintplex-Labs/anything-llm
AnythingLLM is an all-in-one local-first AI application for chatting with documents, managing AI agents, and integrating with various LLMs and vector databases, offering dynamic model routing and m...
BerriAI/litellm
LiteLLM is an open-source AI Gateway and Python SDK providing a unified interface to over 100 LLM providers, with features like cost tracking, guardrails, load balancing, and observability.
QuantumNous/new-api
new-api is a unified LLM gateway and AI asset management system that enables aggregation, distribution, and cross-conversion of various LLMs into OpenAI, Claude, or Gemini compatible formats, offer...
Kong/kong
Kong Gateway is a cloud-native API and AI gateway offering high performance, extensibility via plugins, and advanced AI traffic capabilities including multi-LLM support, semantic security, and cach...
decolua/9router
9Router is an AI router and token saver that connects various AI coding tools to over 40 AI providers, optimizing usage with auto-fallback, quota tracking, and token compression.
apache/apisix
Apache APISIX is a dynamic, real-time, high-performance API Gateway that can also function as an AI Gateway, providing AI proxying, load balancing for LLMs, and robust security for AI agents.
lidge-jun/opencodex
OpenCodex is a universal local proxy that enables OpenAI Codex, Claude Code, and Grok Build to utilize any LLM provider, offering advanced model routing, account pooling, and API key management.