Awesome Infra for AI › LLM Gateways & Proxies

0xzr/freellmpool

⭐ 119 Python added to this list on 2026-09-14 repository created 2026-06-03

freellmpool catalogs 22 LLM providers as distinct groups covering recurring free tiers, keyless endpoints, finite trials, pin-only routes and explicitly disabled candidates, exposing 178 enabled chat routes and 431 cataloged chat models as of its 0.13.0 release. It automatically pools only the routes a user can access behind a single OpenAI-compatible endpoint, usable as a CLI, a Python library, or a local HTTP proxy, and can start answering requests with no credentials at all when a keyless provider such as Pollinations, OVHcloud, Kilo Gateway or LLM7 is available. The proxy fails over between providers when one is rate limited or down, tracks local per-day usage, and offers a spread routing mode across enabled routes. A freellmpool init wizard inspects local provider keys, installed agent CLIs, Tailscale state and existing proxy configuration, then prints a single copy-pastable next step; a companion tailnet mode serves the proxy over a Tailscale address with a generated API key for remote access. Local runtimes such as LM Studio, Ollama and llama.cpp can be discovered and imported from a fixed list of loopback endpoints, read-only and opt-in, without network scanning. The project ships ready-made integration commands for Claude Code, Codex, aider, Cline, Continue, Cursor, Hermes, OpenCode and a named Metaswarm multi-agent lane configuration that pairs a cheap worker model with a larger reviewer model and reserves paid frontier models for explicit escalation. It targets individual developers and small teams who want to run coding agents against free or low-cost model tiers without manually managing several providers' credentials and rate limits.

https://github.com/0xzr/freellmpool

LLM gatewayfree tier routingmulti-providerlocal proxycoding agents