Awesome Infra for AI › LLM Gateways & Proxies

tiylabs/tiygate

⭐ 146 Rust added to this list on 2026-07-06 repository created 2026-06-17

TiyGate is a lightweight, open-source AI gateway written in Rust designed to unify access and management of various LLM providers such as OpenAI, Anthropic, Bedrock, and Gemini. It serves as a control plane sitting between applications and upstream LLM services, enabling users to consolidate fragmented provider access into a single interface. Key functionalities include policy-based routing of virtual models across different providers, automatic failover and recovery strategies based on criteria like priority, weight, throughput, and latency to ensure high availability. The gateway also provides detailed request/response logging, capturing interactions between the client, gateway, and provider, aiding in debugging and operational visibility. Furthermore, TiyGate aggregates usage analytics across various providers, models, and API keys, offering a clear overview of LLM consumption. It supports multiple protocols (OpenAI-compatible, Responses, Messages, Gemini) with extensible N\u00d7N protocol translation. Deployment options include a zero-config desktop app for personal use and containerized modes for enterprise-scale deployments with data plane/control plane separation. Security features include encryption of provider API keys at rest and redaction of sensitive log fields. TiyGate addresses the challenges of managing multiple LLM subscriptions, ensuring service reliability, simplifying debugging, and providing comprehensive usage insights in multi-provider LLM environments.

https://github.com/tiylabs/tiygate

ai-gatewayllm-gatewaymulti-provideropenai-compatiblefailovertraffic-managementllm-observabilityroutinglogging