Awesome Infra for AI › LLM Gateways & Proxies

khankamraan2006-crypto/fabric-router-core

⭐ 116 HTML added to this list on 2026-07-06 repository created 2026-06-28

CloudForge AI functions as a multi-node, horizontally scalable Large Language Model (LLM) gateway designed for production use cases requiring high availability, intelligent failover, and granular cost management. It abstracts away the complexities of integrating with diverse LLM providers by offering a unified, policy-driven interface. The platform decouples application logic from specific inference providers, treating LLM capabilities as a dynamic resource pool. Key features include a provider-agnostic routing engine that optimizes requests based on latency, cost, and rate limits; secure OAuth integration; a dynamic provider registry for hot-reloading configurations; and robust cost governance with budget caps. It also offers comprehensive request telemetry for observability and multilingual response normalization to ensure consistent output across different models. CloudForge AI is built with an operational support framework, including health checks and automated failover, to maintain service continuity. Its architecture emphasizes "request normalization before routing," converting client requests into a canonical internal format before dispatching them to the most optimal backend via dedicated provider adapters. This mediation layer ensures adaptability to changing model costs, performance, and new releases, making it ideal for organizations pursuing a multi-cloud AI strategy.

https://github.com/khankamraan2006-crypto/fabric-router-core

LLM gatewayAI proxyinference orchestrationmulti-provider LLMintelligent routingcost governanceAI securityLLM observabilityproduction AIfailoverAPI management for LLMs