Awesome Infra for AI › LLM Gateways & Proxies

mnfst/llm-gateway

⭐ 7553 TypeScript added to this list on 2026-09-14 repository created 2022-09-27

Manifest (mnfst/llm-gateway) is an open-source LLM gateway that gives AI agents and applications a single OpenAI-compatible endpoint for hundreds of models. It connects API keys, subscriptions, and locally hosted models through 35 built-in provider integrations plus custom OpenAI/Anthropic-compatible endpoints, exposing over 300 models behind one routing layer. Sending "model": "auto" lets Manifest pick the best model for a request, while custom routing rules can target a specific API key, subscription, or local model. The gateway logs full request and response bodies for both successful calls and errors, tracks spend down to the dollar with configurable notifications and limits, and can fall back to alternate models when a query fails, effectively self-healing bad requests before they reach the caller. It ships as a Docker image with one-command self-hosting, plus one-click deployment templates for Railway, Render, DigitalOcean, AWS, GCP, Fly.io, Coolify, Easypanel, Heroku, and Koyeb; a hosted cloud version is also available. Because every deployment path is designed for durable request recording, teams get an auditable history of gateway traffic without extra tooling. Manifest targets teams and individual developers who want to avoid single-provider lock-in for their AI agents or applications, giving them provider flexibility, budget control, and resilience against provider outages or failed generations without changing application code beyond pointing at the gateway's endpoint. It is written in TypeScript and actively maintained, with CI, code coverage tracking, and a Discord community for support.

https://github.com/mnfst/llm-gateway

LLM gatewaymulti-provider routingself-hostedcost trackingTypeScript