Awesome Infra for AI › LLM Gateways & Proxies

astaxie/TokenHub

⭐ 1350 Go added to this list on 2026-07-27 repository created 2026-06-10

TokenHub is a private enterprise AI gateway designed for secure and governed access to large language models (LLMs) and other AI services. It features role-based workspaces for users, team leaders, and administrators, enabling granular control over AI resource consumption and management. The gateway supports OpenAI-compatible APIs, Anthropic Messages APIs, and image generation, and can integrate with various providers such as OpenAI, Azure OpenAI, Anthropic, Gemini, DeepSeek, Qwen, local vLLM/Ollama, and custom upstreams. Key capabilities include a configurable model catalog, dynamic routing policies with priority, weight, and failover, and comprehensive project-scoped key management with quotas and concurrency controls. TokenHub provides detailed usage analytics and request logs attributed by user, project, team, model, and cost center, facilitating cost management and accountability. It also supports identity source configurations for OAuth/OIDC, robust RBAC, and audit trails for compliance. The platform is designed for private deployment, with SQLite-first local deployments and PostgreSQL support for multi-instance, horizontally scalable environments. TokenHub serves as a critical operational layer for enterprises seeking to manage, secure, and optimize their AI inference workloads.

https://github.com/astaxie/TokenHub

ai-gatewayllm-gatewayenterprise-aiapi-managementaccess-controlobservabilityllm-opsdeploymentroutingcost-management