Awesome Infra for AI › LLM Gateways & Proxies

MaySudo/Misceo

⭐ 403 added to this list on 2026-08-31 repository created 2026-07-18

Misceo is a local gateway that lowers the cost of running AI agents without breaking their conversations or tool loops. It exposes an Anthropic-compatible endpoint, so existing clients point at it instead of the provider, and it decides per request which backend should serve the answer. Unlike a static router that must commit before generation, Misceo can also inspect the first completed candidate and then choose what the client receives. A request flows through explicit precedence rules first: protected routes, open tool loops, active pins and user rules bypass the cascade. Eligible traffic goes to a configured lower-cost backend. A structural gate rejects upstream failures, empty output and malformed tool calls. Depending on the selected mode an optional judge scores the visible candidate against the latest user turn, and a rejected candidate is discarded while a stronger backend regenerates the served answer. Escalated conversations then stay on the stronger backend for a configurable number of turns. The project puts particular weight on agent state. Requests are grouped by structured session identity so bootstrap, title-generation and visible turns from one launch stay in the same conversation. At model-family boundaries it handles provider-specific thinking signatures, cache markers, model names and message invariants so a handoff does not corrupt the transcript. When a model opens a tool loop, that backend keeps ownership until the loop completes, and the first visible reply of a session is served by the strong backend. Three postures ship out of the box: quality-first, balanced and savings-first. The proxy, an embedded dashboard, the configuration and the traffic logs all run locally, though inference itself still reaches the configured providers. Misceo is distributed as prebuilt binaries through npm under FSL-1.1-ALv2; this repository holds documentation, issues and releases rather than source.

https://github.com/MaySudo/Misceo

llm-gatewayproxymodel-routingcost-optimizationanthropic-compatibleagentsself-hosted