FareedKhan-dev/kimi-k3-in-c
kimi-k3-in-c is a portable C99 inference engine designed to run the Kimi K3 2.78-trillion-parameter LLM on a single CPU with minimal RAM, focusing on extreme memory efficiency without GPUs or exter...
Awesome Infra for AI › Weekly › 2026-08-03
2026-08-03
Inference Optimization
LLM Evaluation & Testing
MigoXLab/LMeterX
LMeterX is a professional, full-lifecycle platform for load testing and performance benchmarking of large language models and other AI inference services, offering real-time monitoring and AI-power...
huangyiminghappy/ai-eval-platform
An open-source, self-hosted AI evaluation platform offering a web UI for assessing RAG, AI Agents, and multi-turn conversations through dataset management, scenario presets, metrics, LLM-as-a-Judge...
LLM Gateways & Proxies
Paritok-official/paritok-4b-v1
Paritok is an open-source, non-destructive compression model and gateway purpose-built for coding agents, significantly reducing LLM input token costs while maintaining performance on SWE-bench.
youssefvdel/qwengate
Qwen Gate is a self-hosted, OpenAI-compatible API gateway that allows users to access Qwen AI models (via chat.qwen.ai) for free, featuring multi-account rotation, tool calling, and a web dashboard.
LLM Observability & Tracing
comet-ml/opik-openclaw
An official plugin for OpenClaw that exports agent traces to Opik for comprehensive LLM observability, monitoring, and evaluation.
mikehasa/agentacct
Agentacct is a local-first agent work intelligence tool for coding agents, providing a dashboard to track token usage, estimated costs, tasks, and work steps without external telemetry.
comet-ml/opik-mcp
Opik MCP (Model Context Protocol) server for integrating AI hosts (Claude Code, Cursor, VS Code Copilot) with Opik for LLM observability, evaluation, and prompt management.
Prompt Management
Jia-Ethan/grok-keysmith
A Python CLI tool for deploying versioned instruction Markdown files, such as AGENTS.md, to Grok Build, ensuring global configuration with compatibility and hook isolation and providing a robust un...
Vector Databases & Retrieval Infrastructure
FlowElement-ai/m_flow
M-flow is a bio-inspired cognitive memory engine that uses a four-layer cone graph for Retrieval Augmented Generation (RAG), scoring knowledge units by the strongest chain of reasoning paths.
NGT-labs/NGT
NGT is a high-speed approximate nearest neighbor search library for high-dimensional vector data, featuring graph and tree-based methods, quantization techniques, and multi-language support.