diegosouzapw/OmniRoute
OmniRoute is a free AI gateway that unifies access to over 170 AI providers, offering token compression, auto-fallback, and aggregating free tiers to provide billions of free tokens monthly.
Awesome Infra for AI › LLM Gateways & Proxies
Mintplex-Labs/anything-llm
AnythingLLM provides a comprehensive local-first AI application designed for interacting with documents and deploying AI agents. It features a fully-featured chat interface, allowing users to connect their preferred local or cloud-based Large Language Models (LLMs) and ingest various document types for conversational AI. Key functionalities include dynamic model routing, which automatically directs chats to the most suitable LLM based on defined rules, and automatic/user-managed memories for enhanced conversational context. The platform supports scheduled tasks for recurring prompts and agent capabilities, alongside intelligent skill selection to optimize token usage for models with numerous tools. An integrated no-code AI Agent builder facilitates the creation of custom AI agents. Additionally, AnythingLLM offers multi-modal support for both closed and open-source LLMs, multi-user instance support with permissioning (in the Docker version), and an embeddable chat widget for websites. It is production-ready for cloud deployments, compatible with a wide array of LLM providers and vector databases, and includes optimizations for large document sets to reduce costs and improve response times. A full developer API is available for custom integrations.
https://github.com/Mintplex-Labs/anything-llm
Also in LLM Gateways & Proxies
BerriAI/litellm
LiteLLM is an open-source AI Gateway and Python SDK providing a unified interface to over 100 LLM providers, with features like cost tracking, guardrails, load balancing, and observability.
QuantumNous/new-api
new-api is a unified LLM gateway and AI asset management system that enables aggregation, distribution, and cross-conversion of various LLMs into OpenAI, Claude, or Gemini compatible formats, offer...
Kong/kong
Kong Gateway is a cloud-native API and AI gateway offering high performance, extensibility via plugins, and advanced AI traffic capabilities including multi-LLM support, semantic security, and cach...
decolua/9router
9Router is an AI router and token saver that connects various AI coding tools to over 40 AI providers, optimizing usage with auto-fallback, quota tracking, and token compression.
apache/apisix
Apache APISIX is a dynamic, real-time, high-performance API Gateway that can also function as an AI Gateway, providing AI proxying, load balancing for LLMs, and robust security for AI agents.
lidge-jun/opencodex
OpenCodex is a universal local proxy that enables OpenAI Codex, Claude Code, and Grok Build to utilize any LLM provider, offering advanced model routing, account pooling, and API key management.
casdoor/casdoor
Casdoor is an open-source, "AI-first" Identity and Access Management (IAM) and Model Context Protocol (MCP) gateway, providing authentication and authorization for AI applications and agents.