Awesome Infra for AI › LLM Gateways & Proxies

mydisha/keirouter

⭐ 146 Go added to this list on 2026-07-27 repository created 2026-05-31

KeiRouter is an open-source, self-hostable AI gateway designed to streamline and optimize interactions with various large language model (LLM) providers like OpenAI and Anthropic. It mitigates common pain points associated with LLM API usage, such as managing multiple API keys, handling rate limits, and controlling token costs. By directing all AI coding tools and applications to a single local endpoint, KeiRouter intelligently routes requests to the appropriate model, implements failover mechanisms for uninterrupted service, and employs a semantic cache to instantly serve repeat queries, significantly reducing API calls and costs. The gateway offers comprehensive cost control features, including multiple token-saving mechanisms, a budget engine for setting USD and token limits per key or organization, and rate limiting with configurable RPM, TPM, and concurrency caps. For security and governance, KeiRouter integrates guardrails that detect PII, prompt injection, toxicity, and bias, offering layered protection across global, provider, model, and key levels. It also features robust operational and user experience tools, such as activity monitoring, white-label branding, auto-configuration for various CLI tools, and a usage portal for teams to track their consumption and savings. Written in Go, it boasts a small memory footprint and fast startup times, delivering its capabilities through a single binary and an intuitive dashboard.

https://github.com/mydisha/keirouter

ai-gatewayllm-proxyai-routercost-controlrate-limitingguardrailssemantic-cacheself-hostablellm-ops