langfuse/langfuse
Langfuse is an open-source LLM engineering platform for developing, monitoring, evaluating, and debugging AI applications, offering observability, prompt management, and evaluation capabilities.
Awesome Infra for AI › LLM Observability & Tracing
langchain-tracer/Axon
Axon is a local observability tool specifically designed for LangChain and OpenTelemetry-instrumented AI agents. It functions as a CLI that receives OTLP trace data from your application, stores it on your local machine, and presents it through a real-time dashboard. This enables users to monitor and debug every LLM call, tool invocation, and chain execution efficiently. A key feature is its complete offline operation, requiring no cloud account and ensuring all data remains within your environment. It offers various trace views, including a conversational transcript, a waterfall diagram for span visualization, a hierarchical tree view with type icons and token counts, and a raw JSON view of spans. The tool is distributed as three interconnected npm packages for CLI, backend (Express server and SQLite database), and a React-based web interface. Installation is straightforward via npm, allowing users to initialize a project, start Axon, and point their OpenTelemetry-instrumented applications or LangChain tracers to its local OTLP endpoint. It also supports Docker deployment for users who prefer not to install Node.js. Axon's focus on local, privacy-preserving LLM observability makes it a valuable tool for developers working on AI agents and large language model applications.
https://github.com/langchain-tracer/Axon
Also in LLM Observability & Tracing
comet-ml/opik
Opik is an open-source platform for comprehensive observability, evaluation, and optimization of LLM applications, RAG systems, and agentic workflows.
raga-ai-hub/RagaAI-Catalyst
RagaAI Catalyst is a Python SDK for comprehensive observability, monitoring, and evaluation of AI agents and LLM applications, offering tracing, debugging, and advanced analytics.
Arize-ai/phoenix
Phoenix is an open-source AI observability platform for LLM application experimentation, evaluation, and troubleshooting, providing tracing, evaluation, dataset management, prompt management, and a...
VoltAgent/voltagent
VoltAgent is an end-to-end AI Agent Engineering Platform offering an open-source TypeScript framework for building intelligent agents and a VoltOps Console for observability, automation, deployment...
traceloop/openllmetry
OpenLLMetry provides open-source observability for LLM applications by extending OpenTelemetry to capture traces and metrics from LLM providers, vector databases, and AI frameworks.
Helicone/helicone
Helicone is an open-source LLM observability platform and AI gateway that provides monitoring, evaluation, prompt management, and intelligent routing for large language models.
Agenta-AI/agenta
Agenta is an open-source LLMOps platform designed to accelerate the development of reliable LLM applications, offering integrated prompt management, evaluation, and observability features.