langfuse/langfuse
Langfuse is an open-source LLM engineering platform for developing, monitoring, evaluating, and debugging AI applications, offering observability, prompt management, and evaluation capabilities.
Awesome Infra for AI › LLM Observability & Tracing
chirpz-ai/pandaprobe
PandaProbe is an open-source agent engineering platform designed to aid teams in the development and operation of AI agents. It provides capabilities for tracing the execution of AI agents, evaluating their performance, monitoring their behavior in production, and debugging issues that arise. The platform is built to support collaborative workflows, making it easier for multiple team members to work on and improve agent systems. Key features include comprehensive observability through tracing, allowing developers to understand the flow and decisions of their agents. It also offers tools for agent evaluation, helping to quantify agent performance against desired metrics and identify areas for improvement. The monitoring aspect provides insights into agent health and operational statistics once deployed. PandaProbe integrates with popular agent frameworks and SDKs such as LangGraph, CrewAI, Claude Agent SDK, and OpenAI Agents SDK, enabling seamless adoption into existing agent development pipelines. The platform can be deployed as a self-hosted solution using Docker, providing control over data and infrastructure, or utilized through the PandaProbe Cloud offering. Its architecture comprises a FastAPI application server, a Next.js dashboard, and background workers handled by Celery, leveraging PostgreSQL for data storage and Redis for message queuing and caching. This setup ensures scalability and robust functionality for intensive agent engineering tasks. PandaProbe's focus is squarely on the operational aspects of AI agents, from debugging and evaluation to continuous monitoring.
https://github.com/chirpz-ai/pandaprobe
Also in LLM Observability & Tracing
comet-ml/opik
Opik is an open-source platform for comprehensive observability, evaluation, and optimization of LLM applications, RAG systems, and agentic workflows.
raga-ai-hub/RagaAI-Catalyst
RagaAI Catalyst is a Python SDK for comprehensive observability, monitoring, and evaluation of AI agents and LLM applications, offering tracing, debugging, and advanced analytics.
Arize-ai/phoenix
Phoenix is an open-source AI observability platform for LLM application experimentation, evaluation, and troubleshooting, providing tracing, evaluation, dataset management, prompt management, and a...
VoltAgent/voltagent
VoltAgent is an end-to-end AI Agent Engineering Platform offering an open-source TypeScript framework for building intelligent agents and a VoltOps Console for observability, automation, deployment...
traceloop/openllmetry
OpenLLMetry provides open-source observability for LLM applications by extending OpenTelemetry to capture traces and metrics from LLM providers, vector databases, and AI frameworks.
Helicone/helicone
Helicone is an open-source LLM observability platform and AI gateway that provides monitoring, evaluation, prompt management, and intelligent routing for large language models.
Agenta-AI/agenta
Agenta is an open-source LLMOps platform designed to accelerate the development of reliable LLM applications, offering integrated prompt management, evaluation, and observability features.