langfuse/langfuse
Langfuse is an open-source LLM engineering platform for developing, monitoring, evaluating, and debugging AI applications, offering observability, prompt management, and evaluation capabilities.
Awesome Infra for AI › LLM Observability & Tracing
Scale3-Labs/langtrace
Langtrace is an open-source observability solution designed specifically for LLM-powered applications. It utilizes OpenTelemetry standards to provide comprehensive tracing, monitoring, and analytics capabilities. The tool allows developers to capture, debug, and analyze traces and metrics from various components of their LLM applications, including LLM API calls, vector database operations, and LLM framework usage. Key features include real-time monitoring of LLM API calls and vector operations, performance insights (latency, costs, usage patterns), robust debugging tools for LLM application workflows, and detailed analytics with visualizations. Langtrace generates OpenTelemetry-compliant traces and is actively developing semantic conventions for LLM-related data. It offers both a managed SaaS version and a self-hosting option, ensuring data privacy as no telemetry is collected when self-hosted. The project provides SDKs for TypeScript and Python and boasts extensive integrations with popular LLM providers (e.g., OpenAI, Anthropic, AWS Bedrock), LLM frameworks (e.g., Langchain, LlamaIndex, LiteLLM), and vector databases (e.g., Pinecone, ChromaDB, Qdrant, Weaviate, Milvus, PGVector). This makes Langtrace a highly relevant tool for developers and operations teams focused on the production deployment and maintenance of AI applications.
https://github.com/Scale3-Labs/langtrace
Also in LLM Observability & Tracing
comet-ml/opik
Opik is an open-source platform for comprehensive observability, evaluation, and optimization of LLM applications, RAG systems, and agentic workflows.
raga-ai-hub/RagaAI-Catalyst
RagaAI Catalyst is a Python SDK for comprehensive observability, monitoring, and evaluation of AI agents and LLM applications, offering tracing, debugging, and advanced analytics.
Arize-ai/phoenix
Phoenix is an open-source AI observability platform for LLM application experimentation, evaluation, and troubleshooting, providing tracing, evaluation, dataset management, prompt management, and a...
VoltAgent/voltagent
VoltAgent is an end-to-end AI Agent Engineering Platform offering an open-source TypeScript framework for building intelligent agents and a VoltOps Console for observability, automation, deployment...
traceloop/openllmetry
OpenLLMetry provides open-source observability for LLM applications by extending OpenTelemetry to capture traces and metrics from LLM providers, vector databases, and AI frameworks.
Helicone/helicone
Helicone is an open-source LLM observability platform and AI gateway that provides monitoring, evaluation, prompt management, and intelligent routing for large language models.
Agenta-AI/agenta
Agenta is an open-source LLMOps platform designed to accelerate the development of reliable LLM applications, offering integrated prompt management, evaluation, and observability features.