diegosouzapw/OmniRoute
OmniRoute is a free AI gateway that unifies access to over 170 AI providers, offering token compression, auto-fallback, and aggregating free tiers to provide billions of free tokens monthly.
Awesome Infra for AI › LLM Gateways & Proxies
maximhq/bifrost
Bifrost is an enterprise-grade AI gateway designed to streamline and optimize access to various AI model providers. It acts as a single OpenAI-compatible API endpoint for over 23 different providers, including OpenAI, Anthropic, AWS Bedrock, and Google Vertex AI, simplifying integration for developers. The gateway provides core infrastructure capabilities such as automatic fallbacks for seamless failover, intelligent load balancing across multiple API keys and providers, and unified multimodal support for text, images, and audio. Beyond core routing, Bifrost offers advanced features crucial for production AI systems. These include semantic caching to reduce latency and costs by intelligently reusing responses based on similarity, and a Model Context Protocol (MCP) that enables AI models to interact with external tools like filesystems and databases. For enterprise deployments, it supports robust governance with virtual keys, budget management, and fine-grained access control, alongside OIDC user provisioning. Observability is built-in with native Prometheus metrics, distributed tracing, and comprehensive logging. It also provides secure secrets management for API keys. The gateway emphasizes developer experience with zero-config startup, acting as a drop-in replacement for existing GenAI APIs, and offering flexible configuration options via Web UI, API, or file-based methods.
https://github.com/maximhq/bifrost
Also in LLM Gateways & Proxies
Mintplex-Labs/anything-llm
AnythingLLM is an all-in-one local-first AI application for chatting with documents, managing AI agents, and integrating with various LLMs and vector databases, offering dynamic model routing and m...
BerriAI/litellm
LiteLLM is an open-source AI Gateway and Python SDK providing a unified interface to over 100 LLM providers, with features like cost tracking, guardrails, load balancing, and observability.
QuantumNous/new-api
new-api is a unified LLM gateway and AI asset management system that enables aggregation, distribution, and cross-conversion of various LLMs into OpenAI, Claude, or Gemini compatible formats, offer...
Kong/kong
Kong Gateway is a cloud-native API and AI gateway offering high performance, extensibility via plugins, and advanced AI traffic capabilities including multi-LLM support, semantic security, and cach...
decolua/9router
9Router is an AI router and token saver that connects various AI coding tools to over 40 AI providers, optimizing usage with auto-fallback, quota tracking, and token compression.
apache/apisix
Apache APISIX is a dynamic, real-time, high-performance API Gateway that can also function as an AI Gateway, providing AI proxying, load balancing for LLMs, and robust security for AI agents.
lidge-jun/opencodex
OpenCodex is a universal local proxy that enables OpenAI Codex, Claude Code, and Grok Build to utilize any LLM provider, offering advanced model routing, account pooling, and API key management.