diegosouzapw/OmniRoute
OmniRoute is a free AI gateway that unifies access to over 170 AI providers, offering token compression, auto-fallback, and aggregating free tiers to provide billions of free tokens monthly.
Awesome Infra for AI › LLM Gateways & Proxies
api7/aisix
AISIX is an open-source AI gateway written in Rust by the team behind Apache APISIX. It puts a single OpenAI- or Anthropic-compatible API in front of every LLM provider — OpenAI, Anthropic, Google Gemini and Vertex, AWS Bedrock, Azure OpenAI, DeepSeek and any OpenAI-compatible endpoint — so platform teams get one control point for routing, governance, security and observability of LLM and AI-agent traffic. It ships as one static binary with low per-request overhead, lock-free configuration reads and a fast cold start, and it streams server-sent events as a first-class case rather than an afterthought. Configuration is declarative: resources are declared in a single resources.yaml and reloaded on SIGHUP with no restart, or the gateway can read from etcd for a multi-replica cluster, which keeps deployment to one container with no database or separate configuration store. Between the client and the provider the gateway applies API-key authentication, rate and token limits, guardrails, response caching, routing rules and failover, and emits observability data for the traffic it handles. The same binary doubles as the data plane for AISIX Cloud, a commercial control plane hosted by API7 or on-premises that adds centralized management, team governance, budgets, audit and a dashboard; in both arrangements the gateway runs inside the user's own environment and calls providers directly, so live AI traffic never passes through the control plane. Run standalone it remains fully functional and free, licensed under Apache 2.0, and is aimed at organizations that want provider-agnostic access, cost control and a single audit point for everything their applications and agents send to a model.
Also in LLM Gateways & Proxies
Mintplex-Labs/anything-llm
AnythingLLM is an all-in-one local-first AI application for chatting with documents, managing AI agents, and integrating with various LLMs and vector databases, offering dynamic model routing and m...
BerriAI/litellm
LiteLLM is an open-source AI Gateway and Python SDK providing a unified interface to over 100 LLM providers, with features like cost tracking, guardrails, load balancing, and observability.
QuantumNous/new-api
new-api is a unified LLM gateway and AI asset management system that enables aggregation, distribution, and cross-conversion of various LLMs into OpenAI, Claude, or Gemini compatible formats, offer...
Kong/kong
Kong Gateway is a cloud-native API and AI gateway offering high performance, extensibility via plugins, and advanced AI traffic capabilities including multi-LLM support, semantic security, and cach...
decolua/9router
9Router is an AI router and token saver that connects various AI coding tools to over 40 AI providers, optimizing usage with auto-fallback, quota tracking, and token compression.
apache/apisix
Apache APISIX is a dynamic, real-time, high-performance API Gateway that can also function as an AI Gateway, providing AI proxying, load balancing for LLMs, and robust security for AI agents.
lidge-jun/opencodex
OpenCodex is a universal local proxy that enables OpenAI Codex, Claude Code, and Grok Build to utilize any LLM provider, offering advanced model routing, account pooling, and API key management.