diegosouzapw/OmniRoute
OmniRoute is a free AI gateway that unifies access to over 170 AI providers, offering token compression, auto-fallback, and aggregating free tiers to provide billions of free tokens monthly.
Awesome Infra for AI › LLM Gateways & Proxies
theopenco/llmgateway
LLM Gateway acts as a middleware between applications and diverse Large Language Model (LLM) providers like OpenAI, Anthropic, and Google Vertex AI. It offers a unified API interface, compatible with the OpenAI API format, simplifying migration and integration efforts. The gateway centralizes API key management, allowing users to handle multiple provider keys from a single point. It rigorously tracks token usage, costs, and response times across all LLM interactions, providing essential usage analytics and performance metrics. This enables users to optimize their LLM consumption and compare the performance and cost-effectiveness of different models. The project supports both hosted and self-hosted deployment options, ensuring flexibility for various operational needs. For self-hosting, it provides Docker-based deployment instructions, emphasizing best practices for volume management. Its architecture includes separate components for the UI dashboard, a playground, API backend, the gateway itself for routing requests, and documentation. Additionally, it offers enterprise features like advanced billing, extended data retention, custom provider key configurations, and team management, available under a commercial license, while its core functionality remains open source under AGPLv3. This tool is designed for operational teams managing production LLM deployments, focusing on the inference and runtime aspects.
https://github.com/theopenco/llmgateway
Also in LLM Gateways & Proxies
Mintplex-Labs/anything-llm
AnythingLLM is an all-in-one local-first AI application for chatting with documents, managing AI agents, and integrating with various LLMs and vector databases, offering dynamic model routing and m...
BerriAI/litellm
LiteLLM is an open-source AI Gateway and Python SDK providing a unified interface to over 100 LLM providers, with features like cost tracking, guardrails, load balancing, and observability.
QuantumNous/new-api
new-api is a unified LLM gateway and AI asset management system that enables aggregation, distribution, and cross-conversion of various LLMs into OpenAI, Claude, or Gemini compatible formats, offer...
Kong/kong
Kong Gateway is a cloud-native API and AI gateway offering high performance, extensibility via plugins, and advanced AI traffic capabilities including multi-LLM support, semantic security, and cach...
decolua/9router
9Router is an AI router and token saver that connects various AI coding tools to over 40 AI providers, optimizing usage with auto-fallback, quota tracking, and token compression.
apache/apisix
Apache APISIX is a dynamic, real-time, high-performance API Gateway that can also function as an AI Gateway, providing AI proxying, load balancing for LLMs, and robust security for AI agents.
lidge-jun/opencodex
OpenCodex is a universal local proxy that enables OpenAI Codex, Claude Code, and Grok Build to utilize any LLM provider, offering advanced model routing, account pooling, and API key management.