Awesome Infra for AI › LLM Gateways & Proxies

edgee-ai/edgee

⭐ 135 Rust added to this list on 2026-07-13 repository created 2026-03-16

Edgee is a lightweight, open-source LLM gateway written in Rust, designed to sit between an application and various AI providers. Its primary purpose is to route, observe, and compress AI traffic, offering a single control point for managing interactions with large language models. Key features include token compression, which analyzes request context to remove redundancy, leading to fewer tokens and reduced AI costs without altering model responses. Edgee provides a unified API for multiple LLM providers like Anthropic and OpenAI, enabling users to switch models, load-balance, or implement failover strategies without modifying application code. It also offers real-time observability into token consumption, compression savings, and per-request costs. Built with Rust, Edgee boasts fast startup times, a minimal memory footprint, and no runtime dependencies, making it highly efficient. It supports integration with AI coding assistants, allowing for automatic traffic compression, and offers shell shims for seamless routing of AI agent interactions through the gateway. Edgee also includes a statusline feature to display live token usage and compression savings directly within coding assistant interfaces, with tools for managing and resolving conflicts with project-level statuslines.

https://github.com/edgee-ai/edgee

llm-gatewaytoken-compressionobservabilitycost-optimizationmulti-provider routingrustai-traffic-management