diegosouzapw/OmniRoute
OmniRoute is a free AI gateway that unifies access to over 170 AI providers, offering token compression, auto-fallback, and aggregating free tiers to provide billions of free tokens monthly.
Awesome Infra for AI › LLM Gateways & Proxies
w8123/EnterpriseAgentFramework
EnterpriseAgentFramework (ReachAI) is a comprehensive platform designed for Java/Spring Boot enterprises to transform their existing APIs, domain methods, knowledge, models, and processes into governed, orchestrated, and exposed AI capabilities. It addresses critical challenges in enterprise AI adoption, such as managing which APIs an agent can invoke, handling permissioning, audit trails, and impact analysis of API changes on agents and workflows. The framework provides a capability registration center, where Spring Boot applications can declare their business logic as AI-ready capabilities using annotations. This includes a robust governance pipeline with field-level diffing and review processes before capabilities are added to the official catalog. Key features include an 'Agent Studio' for visually composing workflows from capabilities, LLMs, and external tools, supporting interactive nodes, human approval steps, and AI-driven workflow generation and modification. The runtime executes these GraphSpec-based workflows, with adapters for various agent frameworks. For production operations, ReachAI integrates RunOps and tracing, providing granular access control (ACL), pre-flight checks, and immutable invocation authorization for tools. It also offers a gateway for exposing governed agents and public capabilities via protocols like MCP and A2A, along with managing client credentials and call logs. The platform emphasizes enterprise identity mapping and embedded chat SDKs, allowing agents to securely interact within business application pages.
https://github.com/w8123/EnterpriseAgentFramework
Also in LLM Gateways & Proxies
Mintplex-Labs/anything-llm
AnythingLLM is an all-in-one local-first AI application for chatting with documents, managing AI agents, and integrating with various LLMs and vector databases, offering dynamic model routing and m...
BerriAI/litellm
LiteLLM is an open-source AI Gateway and Python SDK providing a unified interface to over 100 LLM providers, with features like cost tracking, guardrails, load balancing, and observability.
QuantumNous/new-api
new-api is a unified LLM gateway and AI asset management system that enables aggregation, distribution, and cross-conversion of various LLMs into OpenAI, Claude, or Gemini compatible formats, offer...
Kong/kong
Kong Gateway is a cloud-native API and AI gateway offering high performance, extensibility via plugins, and advanced AI traffic capabilities including multi-LLM support, semantic security, and cach...
decolua/9router
9Router is an AI router and token saver that connects various AI coding tools to over 40 AI providers, optimizing usage with auto-fallback, quota tracking, and token compression.
apache/apisix
Apache APISIX is a dynamic, real-time, high-performance API Gateway that can also function as an AI Gateway, providing AI proxying, load balancing for LLMs, and robust security for AI agents.
lidge-jun/opencodex
OpenCodex is a universal local proxy that enables OpenAI Codex, Claude Code, and Grok Build to utilize any LLM provider, offering advanced model routing, account pooling, and API key management.