langfuse/langfuse
Langfuse is an open-source LLM engineering platform for developing, monitoring, evaluating, and debugging AI applications, offering observability, prompt management, and evaluation capabilities.
Awesome Infra for AI › LLM Observability & Tracing
traceroot-ai/traceroot
TraceRoot is an open-source observability platform specifically designed for AI agents. It captures LLM calls, agent actions, and tool usage through an OpenTelemetry-compatible SDK to provide detailed tracing within complex agent systems. A key feature is its 'Agentic Debugging' capability, where AI analyzes traces, connects to a sandbox with production source code, identifies exact failing lines, and correlates failures with GitHub history (commits, PRs, issues) to suggest fixes. This dramatically reduces the effort in debugging issues like agent hallucinations, tool call instabilities, and version changes. Additionally, TraceRoot includes 'Detectors' that use an LLM-as-judge evaluator to monitor incoming traces for issues such as hallucinations, tool/logic failures, safety violations, and intent drift, automatically triggering root cause analysis and alerts. The platform emphasizes being fully open-source with no vendor lock-in, supporting various model providers (OpenAI, Anthropic, Gemini, etc.) and agent frameworks (LangChain, AutoGen, LlamaIndex, etc.) through native SDKs and integrations. It can be deployed locally via Docker or in production using Terraform on Kubernetes.
https://github.com/traceroot-ai/traceroot
Also in LLM Observability & Tracing
comet-ml/opik
Opik is an open-source platform for comprehensive observability, evaluation, and optimization of LLM applications, RAG systems, and agentic workflows.
raga-ai-hub/RagaAI-Catalyst
RagaAI Catalyst is a Python SDK for comprehensive observability, monitoring, and evaluation of AI agents and LLM applications, offering tracing, debugging, and advanced analytics.
Arize-ai/phoenix
Phoenix is an open-source AI observability platform for LLM application experimentation, evaluation, and troubleshooting, providing tracing, evaluation, dataset management, prompt management, and a...
VoltAgent/voltagent
VoltAgent is an end-to-end AI Agent Engineering Platform offering an open-source TypeScript framework for building intelligent agents and a VoltOps Console for observability, automation, deployment...
traceloop/openllmetry
OpenLLMetry provides open-source observability for LLM applications by extending OpenTelemetry to capture traces and metrics from LLM providers, vector databases, and AI frameworks.
Helicone/helicone
Helicone is an open-source LLM observability platform and AI gateway that provides monitoring, evaluation, prompt management, and intelligent routing for large language models.
Agenta-AI/agenta
Agenta is an open-source LLMOps platform designed to accelerate the development of reliable LLM applications, offering integrated prompt management, evaluation, and observability features.