Mintplex-Labs/anything-llm
AnythingLLM is an all-in-one local-first AI application for chatting with documents, managing AI agents, and integrating with various LLMs and vector databases, offering dynamic model routing and m...
Awesome Infra for AI › LLM Gateways & Proxies
diegosouzapw/OmniRoute
OmniRoute is an open-source AI gateway designed to streamline interaction with a vast ecosystem of AI models and providers. It acts as a single endpoint, allowing users to connect various AI tools and agents (such as Claude Code, Codex, Cursor, Cline, and Copilot) to over 170 different providers, including more than 50 with free tiers. A key feature is its token compression technology (RTK+Caveman), which can reduce token usage by 15-95%, significantly extending the utility of available resources and potentially saving costs. The gateway also incorporates smart auto-fallback mechanisms for enhanced reliability and supports multimodal APIs. OmniRoute aggregates and tracks free token allowances from multiple providers, claiming to offer over 1.9 billion free tokens per month, and up to 2.5 billion with initial signup credits, without inflating numbers from pooled resources. It provides a dashboard to monitor free token usage. The project supports various routing strategies, making it a flexible solution for managing diverse AI inference workloads. It is designed to be private and local-first, emphasizing user control over their AI interactions. OmniRoute runs across multiple platforms including Desktop (Electron, PWA), CLI, npm, and Docker.
https://github.com/diegosouzapw/OmniRoute
Also in LLM Gateways & Proxies
BerriAI/litellm
LiteLLM is an open-source AI Gateway and Python SDK providing a unified interface to over 100 LLM providers, with features like cost tracking, guardrails, load balancing, and observability.
QuantumNous/new-api
new-api is a unified LLM gateway and AI asset management system that enables aggregation, distribution, and cross-conversion of various LLMs into OpenAI, Claude, or Gemini compatible formats, offer...
Kong/kong
Kong Gateway is a cloud-native API and AI gateway offering high performance, extensibility via plugins, and advanced AI traffic capabilities including multi-LLM support, semantic security, and cach...
decolua/9router
9Router is an AI router and token saver that connects various AI coding tools to over 40 AI providers, optimizing usage with auto-fallback, quota tracking, and token compression.
apache/apisix
Apache APISIX is a dynamic, real-time, high-performance API Gateway that can also function as an AI Gateway, providing AI proxying, load balancing for LLMs, and robust security for AI agents.
lidge-jun/opencodex
OpenCodex is a universal local proxy that enables OpenAI Codex, Claude Code, and Grok Build to utilize any LLM provider, offering advanced model routing, account pooling, and API key management.
casdoor/casdoor
Casdoor is an open-source, "AI-first" Identity and Access Management (IAM) and Model Context Protocol (MCP) gateway, providing authentication and authorization for AI applications and agents.