diegosouzapw/OmniRoute
OmniRoute is a free AI gateway that unifies access to over 170 AI providers, offering token compression, auto-fallback, and aggregating free tiers to provide billions of free tokens monthly.
Awesome Infra for AI › LLM Gateways & Proxies
Fast-Editor/Lynkr
Lynkr is a command-line interface (CLI) tool that functions as an HTTP proxy, specifically engineered to streamline and optimize interactions with large language models (LLMs) for AI coding assistants like Claude Code, Cursor IDE, and Codex CLI. Its primary purpose revolves around token compression, semantic caching, and intelligent request routing to enhance efficiency and reduce costs. The tool achieves significant token savings by: - Stripping unused tools from requests, leading to up to 53% fewer tokens on tool-heavy interactions. - Compressing large JSON tool results, offering up to 87.6% compression. Lynkr also implements a semantic cache that serves repeated queries in milliseconds, effectively reducing billed tokens. A key feature is its "tier routing" capability, which automatically directs different request types to appropriate models based on complexity. Simple queries can be sent to cheaper, faster local models like Ollama, while more complex tasks are escalated to powerful cloud-based LLMs from providers such as OpenRouter, AWS Bedrock, or Databricks. This dynamic routing ensures optimal resource utilization and cost management. Lynkr integrates with various LLM providers, including local options (Ollama, llama.cpp, LM Studio) and cloud services (OpenRouter, AWS Bedrock, Databricks, Azure OpenAI, Azure Anthropic, OpenAI, DeepSeek). It prides itself on requiring zero code changes in the client application; users only need to adjust an environment variable to point their AI tool to the Lynkr proxy. This makes it a non-intrusive solution for improving the operational aspects of AI-powered coding workflows, focusing on inference optimization and LLM gateway functionalities.
https://github.com/Fast-Editor/Lynkr
Also in LLM Gateways & Proxies
Mintplex-Labs/anything-llm
AnythingLLM is an all-in-one local-first AI application for chatting with documents, managing AI agents, and integrating with various LLMs and vector databases, offering dynamic model routing and m...
BerriAI/litellm
LiteLLM is an open-source AI Gateway and Python SDK providing a unified interface to over 100 LLM providers, with features like cost tracking, guardrails, load balancing, and observability.
QuantumNous/new-api
new-api is a unified LLM gateway and AI asset management system that enables aggregation, distribution, and cross-conversion of various LLMs into OpenAI, Claude, or Gemini compatible formats, offer...
Kong/kong
Kong Gateway is a cloud-native API and AI gateway offering high performance, extensibility via plugins, and advanced AI traffic capabilities including multi-LLM support, semantic security, and cach...
decolua/9router
9Router is an AI router and token saver that connects various AI coding tools to over 40 AI providers, optimizing usage with auto-fallback, quota tracking, and token compression.
apache/apisix
Apache APISIX is a dynamic, real-time, high-performance API Gateway that can also function as an AI Gateway, providing AI proxying, load balancing for LLMs, and robust security for AI agents.
lidge-jun/opencodex
OpenCodex is a universal local proxy that enables OpenAI Codex, Claude Code, and Grok Build to utilize any LLM provider, offering advanced model routing, account pooling, and API key management.