langfuse/langfuse
Langfuse is an open-source LLM engineering platform for developing, monitoring, evaluating, and debugging AI applications, offering observability, prompt management, and evaluation capabilities.
Awesome Infra for AI › LLM Observability & Tracing
splunk/token-meter
Token Meter is an open-source usage and cost dashboard for AI coding agents, built and published by Splunk. It parses the session transcripts already written to disk by Claude, Codex, Cursor, OpenCode, Kiro, and Pi, and presents them as a single view covering token usage, estimated cost, context-window pressure, elapsed time, tool calls, and execution status, intended to help a developer decide whether to keep a running session going, step in, compare runs, or investigate a problem. The trace-analysis component is implemented using only the Python standard library, requires no API keys, and sends no analytics or telemetry off the user's machine. The installer stages a per-user runtime and starts both a local web server and a native companion application (a macOS menu-bar app or Linux tray icon), configured to start automatically; a backend-only installation mode is available for users who only want the browser dashboard without the native companion, avoiding the Swift toolchain or Linux GTK/AppIndicator dependencies that the full install needs. A beta Windows extension installs via a PowerShell bootstrap script that uses WinGet to fetch missing dependencies. Once installed, the dashboard is served locally on a fixed port. The project was announced via a Splunk engineering blog post framing it as tooling for "tokenomics", understanding and managing the growing cost surface of AI coding agents. It targets individual developers and teams who use multiple AI coding CLIs and want local, private visibility into what those sessions are actually costing and consuming, without sending session data to a third-party service.
https://github.com/splunk/token-meter
Also in LLM Observability & Tracing
comet-ml/opik
Opik is an open-source platform for comprehensive observability, evaluation, and optimization of LLM applications, RAG systems, and agentic workflows.
raga-ai-hub/RagaAI-Catalyst
RagaAI Catalyst is a Python SDK for comprehensive observability, monitoring, and evaluation of AI agents and LLM applications, offering tracing, debugging, and advanced analytics.
Arize-ai/phoenix
Phoenix is an open-source AI observability platform for LLM application experimentation, evaluation, and troubleshooting, providing tracing, evaluation, dataset management, prompt management, and a...
VoltAgent/voltagent
VoltAgent is an end-to-end AI Agent Engineering Platform offering an open-source TypeScript framework for building intelligent agents and a VoltOps Console for observability, automation, deployment...
traceloop/openllmetry
OpenLLMetry provides open-source observability for LLM applications by extending OpenTelemetry to capture traces and metrics from LLM providers, vector databases, and AI frameworks.
Helicone/helicone
Helicone is an open-source LLM observability platform and AI gateway that provides monitoring, evaluation, prompt management, and intelligent routing for large language models.
Agenta-AI/agenta
Agenta is an open-source LLMOps platform designed to accelerate the development of reliable LLM applications, offering integrated prompt management, evaluation, and observability features.