Awesome Infra for AI › LLM Gateways & Proxies

GetBusbar/busbar

⭐ 171 Rust added to this list on 2026-07-27 repository created 2026-05-29

Busbar is an open-source, self-hosted LLM gateway designed to sit between an application and various AI providers like OpenAI, Anthropic, Gemini, and Bedrock. It's built in Rust as a single, static binary, emphasizing high reliability and performance. The core functionality includes fault-attributed circuit breaking and streaming-safe in-flight failover, ensuring applications remain operational even if a provider experiences issues. Busbar translates seamlessly between different LLM wire protocols (chat, embeddings, images, audio, moderations), allowing applications to use one SDK (e.g., OpenAI's) while connecting to various backend providers. Key features include weighted pools with smooth round-robin load balancing, session affinity, and per-lane concurrency caps. It also offers flexible routing policies, either built-in, via dynamically loaded plugins, or through out-of-process sidecars, which can consider factors like cost, latency, concurrency, and rate limits. For security, Busbar provides native TLS/mTLS termination and hardening features like SSRF guards and constant-time authentication. Governance capabilities include signed expiring keys and hierarchical groups with configurable request, token, budget, and concurrency limits, along with spend tracking. Observability is provided through open standards like Prometheus metrics and OTLP traces. Busbar positions itself as a reliability-first alternative to other LLM gateways, focusing on robust operation and efficient multi-provider management for AI applications.

https://github.com/GetBusbar/busbar

ai-gatewayanthropicbedrockcircuit-breakerclaudefailovergeminilitellm-alternativellmllm-gatewayllm-proxyllmopsload-balanceropenaiopenrouter-alternativereverse-proxyrustsigv4streamingprotocol translationfault tolerancegovernanceobservability