langfuse/langfuse
Langfuse is an open-source LLM engineering platform for developing, monitoring, evaluating, and debugging AI applications, offering observability, prompt management, and evaluation capabilities.
Awesome Infra for AI › LLM Observability & Tracing
Netis/heron
Heron is a passive observability tool for AI agents and LLM traffic, built in Rust, that reconstructs what happened during an agent run entirely from the traffic itself rather than from instrumentation inside the agent. It ingests a .pcap file, a live network interface, a ZMQ feed from the companion cloud-probe tool, or, experimentally on Linux, TLS-encrypted traffic read in-process via eBPF SSL_read/SSL_write uprobes, and replays it through an HTTP/SSE parser, wire-API detector and semantic extractor. The result is a dashboard, backed by DuckDB and a REST API, showing live time-to-first-token, end-to-end latency, throughput, error rate and per-agent traffic mix. Its signature feature is agent-turn reconstruction: stitching a sequence of planner, tool and result calls into a single addressable turn rather than leaving users to join raw HTTP calls by hand, with named profiles for Claude Code and OpenAI Codex CLI plus a generic profile for other agents. It also builds a service-topology graph of clients, LiteLLM-style proxies and inference backends such as vLLM, SGLang, Ollama and llama.cpp, classifying each endpoint purely from bytes on the wire rather than configuration, and can export any turn or session as OpenAI-style JSONL suitable for supervised fine-tuning. Because it never sits in the request path, an observer failure cannot break the calls being observed, and no client code, SDK or proxy re-pointing is required, at the cost of losing cross-cluster client-level tracing that instrumented approaches provide. It targets platform and ML infrastructure teams who want agent and LLM traffic visibility on inference hosts or fleets without modifying the workloads being observed.
https://github.com/Netis/heron
Also in LLM Observability & Tracing
comet-ml/opik
Opik is an open-source platform for comprehensive observability, evaluation, and optimization of LLM applications, RAG systems, and agentic workflows.
raga-ai-hub/RagaAI-Catalyst
RagaAI Catalyst is a Python SDK for comprehensive observability, monitoring, and evaluation of AI agents and LLM applications, offering tracing, debugging, and advanced analytics.
Arize-ai/phoenix
Phoenix is an open-source AI observability platform for LLM application experimentation, evaluation, and troubleshooting, providing tracing, evaluation, dataset management, prompt management, and a...
VoltAgent/voltagent
VoltAgent is an end-to-end AI Agent Engineering Platform offering an open-source TypeScript framework for building intelligent agents and a VoltOps Console for observability, automation, deployment...
traceloop/openllmetry
OpenLLMetry provides open-source observability for LLM applications by extending OpenTelemetry to capture traces and metrics from LLM providers, vector databases, and AI frameworks.
Helicone/helicone
Helicone is an open-source LLM observability platform and AI gateway that provides monitoring, evaluation, prompt management, and intelligent routing for large language models.
Agenta-AI/agenta
Agenta is an open-source LLMOps platform designed to accelerate the development of reliable LLM applications, offering integrated prompt management, evaluation, and observability features.