diegosouzapw/OmniRoute
OmniRoute is a free AI gateway that unifies access to over 170 AI providers, offering token compression, auto-fallback, and aggregating free tiers to provide billions of free tokens monthly.
Awesome Infra for AI › LLM Gateways & Proxies
dataiku/kiji-proxy
Kiji Privacy Proxy acts as an intelligent privacy layer that sits between your applications and AI API services like OpenAI or Anthropic. Its primary function is to protect sensitive data by automatically detecting and masking Personally Identifiable Information (PII) before it leaves your local environment and is sent to external AI services. The proxy uses an ML-powered DistilBERT transformer model, optimized with ONNX Runtime, to identify a broad range of PII types including emails, phone numbers, SSNs, credit cards, and addresses. Before forwarding requests, Kiji replaces detected PII with realistic dummy values and then transparently restores the original data in the AI service's response, ensuring the consuming application functions normally. Users can configure masking behavior by disabling specific entity types, tuning sensitivity, or adding custom regex rules for domain-specific PII. A UI provides the ability to review and delete masked value mappings. It offers zero-code-change integration through a transparent proxy with automatic configuration for macOS browsers and supports Linux as a standalone server or with manual proxy settings. Kiji also provides a Chrome extension for inline PII detection on popular AI chat sites. All PII detection and masking occurs via fast, local inference, ensuring data privacy and compliance with regulations like GDPR, HIPAA, and CCPA without relying on external API calls for the masking process itself.
https://github.com/dataiku/kiji-proxy
Also in LLM Gateways & Proxies
Mintplex-Labs/anything-llm
AnythingLLM is an all-in-one local-first AI application for chatting with documents, managing AI agents, and integrating with various LLMs and vector databases, offering dynamic model routing and m...
BerriAI/litellm
LiteLLM is an open-source AI Gateway and Python SDK providing a unified interface to over 100 LLM providers, with features like cost tracking, guardrails, load balancing, and observability.
QuantumNous/new-api
new-api is a unified LLM gateway and AI asset management system that enables aggregation, distribution, and cross-conversion of various LLMs into OpenAI, Claude, or Gemini compatible formats, offer...
Kong/kong
Kong Gateway is a cloud-native API and AI gateway offering high performance, extensibility via plugins, and advanced AI traffic capabilities including multi-LLM support, semantic security, and cach...
decolua/9router
9Router is an AI router and token saver that connects various AI coding tools to over 40 AI providers, optimizing usage with auto-fallback, quota tracking, and token compression.
apache/apisix
Apache APISIX is a dynamic, real-time, high-performance API Gateway that can also function as an AI Gateway, providing AI proxying, load balancing for LLMs, and robust security for AI agents.
lidge-jun/opencodex
OpenCodex is a universal local proxy that enables OpenAI Codex, Claude Code, and Grok Build to utilize any LLM provider, offering advanced model routing, account pooling, and API key management.