langfuse/langfuse
Langfuse is an open-source LLM engineering platform for developing, monitoring, evaluating, and debugging AI applications, offering observability, prompt management, and evaluation capabilities.
Awesome Infra for AI › LLM Observability & Tracing
comet-ml/opik-openclaw
This project provides an official plugin, `@opik/opik-openclaw`, for OpenClaw, enabling the export of agent traces to Opik, an open-source LLM observability and evaluation platform by Comet.ml. The plugin integrates seamlessly within the OpenClaw Gateway process, capturing detailed LLM request/response spans, sub-agent request/response spans, and tool call spans with inputs, outputs, and errors. It also records run-level finalize metadata, along with usage and cost metrics. This allows users to monitor agent behavior, costs, token usage, and identify errors in real-time. The plugin facilitates a deeper understanding of agent execution flows, crucial for debugging, optimizing, and ensuring the reliability of AI agents. Configuration is straightforward, with options for API keys, project/workspace names, and advanced settings like trace cleanup and flush retries. It supports environment variable fallbacks and includes CLI commands for installation, configuration, and status checks, making it a robust tool for LLM operational monitoring and evaluation.
https://github.com/comet-ml/opik-openclaw
Also in LLM Observability & Tracing
comet-ml/opik
Opik is an open-source platform for comprehensive observability, evaluation, and optimization of LLM applications, RAG systems, and agentic workflows.
raga-ai-hub/RagaAI-Catalyst
RagaAI Catalyst is a Python SDK for comprehensive observability, monitoring, and evaluation of AI agents and LLM applications, offering tracing, debugging, and advanced analytics.
Arize-ai/phoenix
Phoenix is an open-source AI observability platform for LLM application experimentation, evaluation, and troubleshooting, providing tracing, evaluation, dataset management, prompt management, and a...
VoltAgent/voltagent
VoltAgent is an end-to-end AI Agent Engineering Platform offering an open-source TypeScript framework for building intelligent agents and a VoltOps Console for observability, automation, deployment...
traceloop/openllmetry
OpenLLMetry provides open-source observability for LLM applications by extending OpenTelemetry to capture traces and metrics from LLM providers, vector databases, and AI frameworks.
Helicone/helicone
Helicone is an open-source LLM observability platform and AI gateway that provides monitoring, evaluation, prompt management, and intelligent routing for large language models.
Agenta-AI/agenta
Agenta is an open-source LLMOps platform designed to accelerate the development of reliable LLM applications, offering integrated prompt management, evaluation, and observability features.