Awesome Infra for AI › LLM Gateways & Proxies

LeenHawk/gproxy

⭐ 248 Rust repository created 2026-02-01

GPROXY is a robust, Rust-implemented LLM proxy server designed for high performance and multi-provider, multi-tenant environments. It consolidates access to various LLM APIs, including OpenAI, Anthropic, Gemini, DeepSeek, and others, under a single, unified interface that is compatible with OpenAI, Anthropic, and Gemini protocols. Key features include two routing modes (aggregated and scoped), same-protocol passthrough for minimal overhead, and cross-protocol translation that allows clients using one API dialect to communicate with upstream providers employing another. The proxy provides comprehensive multi-tenant authorization with user and API key management, glob model permissions, rate limiting (RPM/RPD/tokens), and USD-denominated quotas. It also supports Claude prompt caching, request and message rewrite rules for JSON field manipulation and regex text substitution, and pluggable storage options like SQLite, PostgreSQL, and MySQL with optional encryption. An embedded React-based admin console simplifies configuration and management without requiring separate deployments. Additionally, GPROXY offers a Rust SDK for embedding its engine into other services. This tool is essential for operating, governing, and serving large language models by providing a centralized and controlled access point with advanced management capabilities.

https://github.com/LeenHawk/gproxy

LLM proxyAPI gatewaymulti-providermulti-tenantOpenAI-compatibleClaude-compatibleGemini-compatiblerate limitingquota managementAI governancemodel servingRust