Awesome Infra for AI › LLM Gateways & Proxies

nextlevelbuilder/goclaw

⭐ 3640 Go repository created 2026-02-22

GoClaw is an open-source, multi-tenant AI agent platform and gateway re-envisioned in Go for scalability, security, and native concurrency. It provides a robust infrastructure for deploying and orchestrating AI agent teams, featuring multi-tenant isolation, a 5-layer security model, and inherent concurrency. The platform supports over 20 LLM providers, including Anthropic and OpenAI, and integrates with 7 messaging channels like Telegram and Discord. Key architectural components include an 8-stage agent pipeline for comprehensive context, history, prompt processing, and memory management, a 4-mode prompt system for optimized prompt delivery, and a 3-tier memory system (working, episodic, semantic) with knowledge graph capabilities. It offers advanced agent orchestration with shared task boards, inter-agent delegation, and self-evolution features for agents to adapt their communication styles. The platform emphasizes production readiness with features like multi-tenant PostgreSQL for isolated workspaces, encrypted API keys, RBAC, prompt injection detection, and SSRF protection. Observability is integrated with LLM call tracing and OpenTelemetry export. GoClaw is distributed as a single, lightweight Go binary, ensuring fast startup and efficient deployment. A desktop version, GoClaw Lite, is also available for local AI agents without the need for Docker or PostgreSQL, offering a simplified local experience for chat with agents, agent management, and team tasks.

https://github.com/nextlevelbuilder/goclaw

AI agent orchestrationLLM gatewaymulti-tenantGosecurityAI agentsagent pipelineprompt managementmemory managementknowledge graphobservabilityOpenTelemetryPostgreSQLpgvectormulti-provider LLMconversational AI

Also in LLM Gateways & Proxies

diegosouzapw/OmniRoute

OmniRoute is a free AI gateway that unifies access to over 170 AI providers, offering token compression, auto-fallback, and aggregating free tiers to provide billions of free tokens monthly.

Mintplex-Labs/anything-llm

AnythingLLM is an all-in-one local-first AI application for chatting with documents, managing AI agents, and integrating with various LLMs and vector databases, offering dynamic model routing and m...

BerriAI/litellm

LiteLLM is an open-source AI Gateway and Python SDK providing a unified interface to over 100 LLM providers, with features like cost tracking, guardrails, load balancing, and observability.

QuantumNous/new-api

new-api is a unified LLM gateway and AI asset management system that enables aggregation, distribution, and cross-conversion of various LLMs into OpenAI, Claude, or Gemini compatible formats, offer...

Kong/kong

Kong Gateway is a cloud-native API and AI gateway offering high performance, extensibility via plugins, and advanced AI traffic capabilities including multi-LLM support, semantic security, and cach...

decolua/9router

9Router is an AI router and token saver that connects various AI coding tools to over 40 AI providers, optimizing usage with auto-fallback, quota tracking, and token compression.

apache/apisix

Apache APISIX is a dynamic, real-time, high-performance API Gateway that can also function as an AI Gateway, providing AI proxying, load balancing for LLMs, and robust security for AI agents.

lidge-jun/opencodex

OpenCodex is a universal local proxy that enables OpenAI Codex, Claude Code, and Grok Build to utilize any LLM provider, offering advanced model routing, account pooling, and API key management.