langfuse/langfuse
Langfuse is an open-source LLM engineering platform for developing, monitoring, evaluating, and debugging AI applications, offering observability, prompt management, and evaluation capabilities.
Awesome Infra for AI › LLM Observability & Tracing
vivekchand/clawmetry
ClawMetry is a locally run observability tool for AI agent runtimes. It installs as a single Python package, starts with no configuration, and auto-detects the agent runtimes present on the machine, then serves a dashboard on localhost that shows what each agent is doing in real time. Coverage spans a broad set of runtimes — OpenClaw, NVIDIA NemoClaw, Claude Code, OpenAI Codex, Cursor, Goose, Hermes, opencode, Qwen Code, Aider, Deep Agents, n8n, GitHub Copilot and others — each with its own log or session format that ClawMetry reads and normalizes into a common event model. The dashboard visualizes agent flow: which sessions ran, which tools were called and in what order, which model and provider served each request, and how many tokens the run consumed. Because every runtime is reduced to the same event shape, a team running several agents side by side gets one place to compare their behaviour and cost rather than one bespoke log reader per tool. Everything runs on the developer machine, so traces and prompts do not leave it. The project positions itself as fleet metering: instead of instrumenting a single agent, it treats all agent processes on a host as one population to be observed. Typical uses are spotting runaway token spend, understanding why an agent looped on a tool, comparing how two runtimes solved the same task, and keeping an audit trail of agent activity. It is distributed under the MIT licence with documentation translated into a dozen languages.
https://github.com/vivekchand/clawmetry
Also in LLM Observability & Tracing
comet-ml/opik
Opik is an open-source platform for comprehensive observability, evaluation, and optimization of LLM applications, RAG systems, and agentic workflows.
raga-ai-hub/RagaAI-Catalyst
RagaAI Catalyst is a Python SDK for comprehensive observability, monitoring, and evaluation of AI agents and LLM applications, offering tracing, debugging, and advanced analytics.
Arize-ai/phoenix
Phoenix is an open-source AI observability platform for LLM application experimentation, evaluation, and troubleshooting, providing tracing, evaluation, dataset management, prompt management, and a...
VoltAgent/voltagent
VoltAgent is an end-to-end AI Agent Engineering Platform offering an open-source TypeScript framework for building intelligent agents and a VoltOps Console for observability, automation, deployment...
traceloop/openllmetry
OpenLLMetry provides open-source observability for LLM applications by extending OpenTelemetry to capture traces and metrics from LLM providers, vector databases, and AI frameworks.
Helicone/helicone
Helicone is an open-source LLM observability platform and AI gateway that provides monitoring, evaluation, prompt management, and intelligent routing for large language models.
Agenta-AI/agenta
Agenta is an open-source LLMOps platform designed to accelerate the development of reliable LLM applications, offering integrated prompt management, evaluation, and observability features.