langfuse/langfuse
Langfuse is an open-source LLM engineering platform for developing, monitoring, evaluating, and debugging AI applications, offering observability, prompt management, and evaluation capabilities.
Awesome Infra for AI › LLM Observability & Tracing
msfirebird/claw-lens
Claw-lens is a comprehensive, open-source observability solution specifically designed for OpenClaw AI agents. It offers a local-first dashboard that runs on the user's machine, eliminating the need for external data uploads or complex configurations. The tool provides crucial insights into AI agent operations, focusing on cost analytics, session monitoring, and security. Key features include a detailed overview of KPIs, token usage breakdowns, per-agent statistics, and a live activity feed. Users can inspect individual sessions, track scheduled tasks, and view agent memory. A robust audit feature provides security event timelines, rule-based risk scoring for file access, shell commands, external HTTP calls, sensitive data exposure, and prompt injection detection, alongside behavioral anomaly detection. It also offers session timelines, profilers for token consumption, tools to detect repetitive behavior in 'deep turns,' context window fill visualizations, and cache trace replays. Designed for operators and developers, claw-lens uniquely emphasizes cost as a primary metric and treats sessions as atomic units, addressing the specific challenges of AI agent observability. All data remains local, ensuring privacy and ease of use, reading directly from OpenClaw's JSONL logs and storing its own data in a local SQLite database.
https://github.com/msfirebird/claw-lens
Also in LLM Observability & Tracing
comet-ml/opik
Opik is an open-source platform for comprehensive observability, evaluation, and optimization of LLM applications, RAG systems, and agentic workflows.
raga-ai-hub/RagaAI-Catalyst
RagaAI Catalyst is a Python SDK for comprehensive observability, monitoring, and evaluation of AI agents and LLM applications, offering tracing, debugging, and advanced analytics.
Arize-ai/phoenix
Phoenix is an open-source AI observability platform for LLM application experimentation, evaluation, and troubleshooting, providing tracing, evaluation, dataset management, prompt management, and a...
VoltAgent/voltagent
VoltAgent is an end-to-end AI Agent Engineering Platform offering an open-source TypeScript framework for building intelligent agents and a VoltOps Console for observability, automation, deployment...
traceloop/openllmetry
OpenLLMetry provides open-source observability for LLM applications by extending OpenTelemetry to capture traces and metrics from LLM providers, vector databases, and AI frameworks.
Helicone/helicone
Helicone is an open-source LLM observability platform and AI gateway that provides monitoring, evaluation, prompt management, and intelligent routing for large language models.
Agenta-AI/agenta
Agenta is an open-source LLMOps platform designed to accelerate the development of reliable LLM applications, offering integrated prompt management, evaluation, and observability features.