langfuse/langfuse
Langfuse is an open-source LLM engineering platform for developing, monitoring, evaluating, and debugging AI applications, offering observability, prompt management, and evaluation capabilities.
Awesome Infra for AI › LLM Observability & Tracing
mikehasa/agentacct
Agentacct is designed to offer a transparent, local-first view into the operational aspects of AI coding agents like Claude Code and Codex. It processes the session logs generated by these agents directly on the user's machine, consolidating information on token consumption, estimated costs, and the work steps performed during each task. The tool provides a dashboard that visualizes this data, breaking down activities by agent, model, and day. A core principle of agentacct is privacy, ensuring that all data remains local, with no cloud sync, phone-home telemetry, or API key storage. It differentiates between client-reported usage and estimated costs, providing attribution confidence levels for linking usage to specific work. The tool focuses on "work meaning" by recording work steps and machine checks (e.g., passing tests) through MCP (Multi-Client Protocol) events. It aims to give users honest and attributable insights into their agent's performance and expenditure, particularly useful for developers and teams using AI coding assistants who need to monitor resource usage and understand agent behavior in detail. The project is currently in an early alpha stage, prioritizing accuracy and transparency over broad feature sets.
https://github.com/mikehasa/agentacct
Also in LLM Observability & Tracing
comet-ml/opik
Opik is an open-source platform for comprehensive observability, evaluation, and optimization of LLM applications, RAG systems, and agentic workflows.
raga-ai-hub/RagaAI-Catalyst
RagaAI Catalyst is a Python SDK for comprehensive observability, monitoring, and evaluation of AI agents and LLM applications, offering tracing, debugging, and advanced analytics.
Arize-ai/phoenix
Phoenix is an open-source AI observability platform for LLM application experimentation, evaluation, and troubleshooting, providing tracing, evaluation, dataset management, prompt management, and a...
VoltAgent/voltagent
VoltAgent is an end-to-end AI Agent Engineering Platform offering an open-source TypeScript framework for building intelligent agents and a VoltOps Console for observability, automation, deployment...
traceloop/openllmetry
OpenLLMetry provides open-source observability for LLM applications by extending OpenTelemetry to capture traces and metrics from LLM providers, vector databases, and AI frameworks.
Helicone/helicone
Helicone is an open-source LLM observability platform and AI gateway that provides monitoring, evaluation, prompt management, and intelligent routing for large language models.
Agenta-AI/agenta
Agenta is an open-source LLMOps platform designed to accelerate the development of reliable LLM applications, offering integrated prompt management, evaluation, and observability features.