Awesome Infra for AI › LLM Gateways & Proxies

caidaoli/ccLoad

⭐ 418 Go added to this list on 2026-06-14 repository created 2025-09-08

ccLoad is an specialized AI API gateway designed to simplify the operational complexities of managing multiple AI API upstreams like Claude Code, Codex, Gemini, and OpenAI. It acts as a single, stable entry point for clients, abstracting away the intricacies of API key management, rate limits, and provider-specific failures. The gateway features smart routing, which prioritizes channels and uses weighted round-robin for balanced load distribution among upstreams. It ensures high availability through automatic failover mechanisms, which detect and bypass failed keys or channels, and implements exponential cooldowns to prevent hammering unstable services. ccLoad also supports multi-URL scheduling, allowing a single channel to leverage multiple upstream URLs, with selection based on observed latency and health. A key feature is its ability to perform protocol transformations, enabling cross-protocol conversion between Anthropic, OpenAI, Gemini, and Codex APIs, thus allowing a single channel to serve multiple client protocols. For operational visibility, it offers live monitoring with a web dashboard displaying active requests, logs, token usage, time to first byte (TTFB), and costs. It also handles "soft errors" (HTTP 200 responses that contain error content) and provides robust cost control through per-channel daily limits and per-token cost limits. The system includes local token counting for accurate billing estimation, secure authentication, and supports single binary deployment with embedded SQLite.

https://github.com/caidaoli/ccLoad

aiai-gatewayanthropicapi-proxyclaude-apiclaude-codecodexcost-controlfailovergeminigolangllm-proxyload-balancermonitoringopenaiopenai-apiprotocol-transformreverse-proxysmart-routingtoken-counting