kigner/audio.cpp-webui
High-performance C++ audio inference framework built on `ggml` for local AI models, supporting TTS, ASR, voice conversion, and more with a full-task WebUI and optimized CUDA performance.
Awesome Infra for AI › Weekly › 2026-07-27
2026-07-27
Inference Optimization
RightNow-AI/auto
Auto is an AGI compiler that records LLM agent behavior, identifies repeatable patterns, and compiles them into verified, sandboxed WebAssembly binaries for efficient execution.
LLM Evaluation & Testing
Ricky-7-Yan/intelligent-audit-system
AuditPilot is an enterprise AI agent workbench designed for auditable, evidence-grounded workflows, featuring governed tools, evaluation harnesses, human review, and remediation delivery for audit ...
ShenSeanChen/waku-agent
Waku Agent is a local-first, personal AI assistant emphasizing a transparent architecture for its harness, loop, memory, and evaluation, designed for clarity and customizability.
OpenBMB/UltraEval-Audio
UltraEval-Audio is a unified open-source framework for comprehensive and reproducible evaluation of audio foundation models across speech understanding and speech generation tasks.
LLM Gateways & Proxies
lidge-jun/opencodex
OpenCodex is a universal local proxy that enables OpenAI Codex, Claude Code, and Grok Build to utilize any LLM provider, offering advanced model routing, account pooling, and API key management.
astaxie/TokenHub
TokenHub is an enterprise AI gateway offering role-based access, API routing, usage analytics, and a model catalog for managing various LLM providers.
Francis1998/nexus-llm-router
Nexus is an intelligent multi-LLM router providing task-aware model selection, cost optimization, and production safety controls via a drop-in OpenAI-compatible API.
youssefvdel/opengate
OpenGate is a self-hosted, OpenAI-compatible API gateway that enables users to access Qwen AI models for free using their chat.qwen.ai accounts, supporting multi-account rotation, tool calling, and...
GetBusbar/busbar
Busbar is a self-hosted LLM gateway implemented in Rust that provides multi-vendor failover, load balancing, protocol translation, and governance for AI applications.
mydisha/keirouter
KeiRouter is a self-hostable, blazing-fast AI gateway that acts as a smart middleman for LLM API calls, providing intelligent routing, caching, cost control, and security guardrails.
LLM Observability & Tracing
douglasmonsky/codex-usage-tracker
Tracks and analyzes local Codex (OpenAI) token usage, costs, and thread patterns through a local-first dashboard and CLI, aiding in cost optimization and waste reduction for AI developers.
inferock/inferock-bench
inferock-bench is a local LLM cost-tracking proxy that provides independent, per-call receipts for OpenAI, Anthropic, Gemini, and OpenRouter, helping users audit bills, track token usage, and ident...
langchain-tracer/Axon
Axon is an OpenTelemetry-native CLI for local LLM observability, providing real-time dashboards to monitor and debug LLM/agent traces without cloud accounts.
Workflow Orchestration for AI
builderz-labs/mission-control
A self-hosted control plane for dispatching tasks, tracking runs, monitoring spend, and coordinating various AI agent runtimes through a local dashboard.