diegosouzapw/OmniRoute
OmniRoute is a free AI gateway that unifies access to over 170 AI providers, offering token compression, auto-fallback, and aggregating free tiers to provide billions of free tokens monthly.
Awesome Infra for AI › LLM Gateways & Proxies
bestruirui/octopus
Octopus is an open-source, self-hosted LLM API aggregation and load balancing service designed to simplify the management and operation of large language model (LLM) APIs for individuals and small teams. It acts as a unified gateway, allowing users to connect to multiple LLM providers (e.g., OpenAI, Anthropic) through a single interface. Key features include multi-channel aggregation for various LLM services, support for multiple API keys per channel, and smart selection capabilities to route requests to the endpoint with the shortest delay for optimized performance. The platform also offers automatic load balancing to ensure stable and efficient service delivery. Octopus includes protocol conversion for seamless interaction between different LLM API formats, such as OpenAI Chat/Responses and Anthropic API. It automatically synchronizes model lists and pricing information, providing up-to-date cost and availability data. For operational insights, the service offers comprehensive analytics, including request statistics, token consumption, and cost tracking. Users interact with Octopus through an elegant web management panel, and it supports multiple database backends like SQLite, MySQL, and PostgreSQL for flexible deployment. The project emphasizes ease of deployment via Docker or direct binary execution, making it accessible for personal and small-scale operational use of LLMs.
https://github.com/bestruirui/octopus
Also in LLM Gateways & Proxies
Mintplex-Labs/anything-llm
AnythingLLM is an all-in-one local-first AI application for chatting with documents, managing AI agents, and integrating with various LLMs and vector databases, offering dynamic model routing and m...
BerriAI/litellm
LiteLLM is an open-source AI Gateway and Python SDK providing a unified interface to over 100 LLM providers, with features like cost tracking, guardrails, load balancing, and observability.
QuantumNous/new-api
new-api is a unified LLM gateway and AI asset management system that enables aggregation, distribution, and cross-conversion of various LLMs into OpenAI, Claude, or Gemini compatible formats, offer...
Kong/kong
Kong Gateway is a cloud-native API and AI gateway offering high performance, extensibility via plugins, and advanced AI traffic capabilities including multi-LLM support, semantic security, and cach...
decolua/9router
9Router is an AI router and token saver that connects various AI coding tools to over 40 AI providers, optimizing usage with auto-fallback, quota tracking, and token compression.
apache/apisix
Apache APISIX is a dynamic, real-time, high-performance API Gateway that can also function as an AI Gateway, providing AI proxying, load balancing for LLMs, and robust security for AI agents.
lidge-jun/opencodex
OpenCodex is a universal local proxy that enables OpenAI Codex, Claude Code, and Grok Build to utilize any LLM provider, offering advanced model routing, account pooling, and API key management.