diegosouzapw/OmniRoute
OmniRoute is a free AI gateway that unifies access to over 170 AI providers, offering token compression, auto-fallback, and aggregating free tiers to provide billions of free tokens monthly.
Awesome Infra for AI › LLM Gateways & Proxies
Portkey-AI/gateway
Portkey AI Gateway provides a lightweight, open-source, and enterprise-ready solution for managing API calls to a wide range of Large Language Models (LLMs) and multi-modal AI models. It acts as a central proxy that routes requests to various AI providers, supporting over 1,600 models across more than 40 providers. Key features include blazing-fast performance with sub-millisecond latency and a tiny footprint, making it suitable for high-throughput AI applications. The gateway offers robust reliability features such as automatic retries and fallbacks to prevent downtimes, ensuring continuous service availability. For scaling AI applications, it supports load balancing and conditional routing based on predefined rules. A significant aspect of Portkey AI Gateway is its integrated AI safety and guardrails, which help protect AI deployments by filtering or modifying outputs based on configured policies. It also provides capabilities for multi-modal AI applications and integrates with agentic workflows. The project emphasizes ease of integration, allowing developers to connect with any language model quickly, and offers a console for local log visibility. Deployment options include local setup, Docker, Node.js, Cloudflare Workers, and various cloud platforms like AWS, Azure, and GCP for enterprise-grade private deployments.
https://github.com/Portkey-AI/gateway
Also in LLM Gateways & Proxies
Mintplex-Labs/anything-llm
AnythingLLM is an all-in-one local-first AI application for chatting with documents, managing AI agents, and integrating with various LLMs and vector databases, offering dynamic model routing and m...
BerriAI/litellm
LiteLLM is an open-source AI Gateway and Python SDK providing a unified interface to over 100 LLM providers, with features like cost tracking, guardrails, load balancing, and observability.
QuantumNous/new-api
new-api is a unified LLM gateway and AI asset management system that enables aggregation, distribution, and cross-conversion of various LLMs into OpenAI, Claude, or Gemini compatible formats, offer...
Kong/kong
Kong Gateway is a cloud-native API and AI gateway offering high performance, extensibility via plugins, and advanced AI traffic capabilities including multi-LLM support, semantic security, and cach...
decolua/9router
9Router is an AI router and token saver that connects various AI coding tools to over 40 AI providers, optimizing usage with auto-fallback, quota tracking, and token compression.
apache/apisix
Apache APISIX is a dynamic, real-time, high-performance API Gateway that can also function as an AI Gateway, providing AI proxying, load balancing for LLMs, and robust security for AI agents.
lidge-jun/opencodex
OpenCodex is a universal local proxy that enables OpenAI Codex, Claude Code, and Grok Build to utilize any LLM provider, offering advanced model routing, account pooling, and API key management.