langfuse/langfuse
Langfuse is an open-source LLM engineering platform for developing, monitoring, evaluating, and debugging AI applications, offering observability, prompt management, and evaluation capabilities.
Awesome Infra for AI › LLM Observability & Tracing
langfuse/oss-llmops-stack
The `langfuse/oss-llmops-stack` project presents a comprehensive, modular, and open-source solution for LLM operations (LLMOps), designed to streamline the management and performance of AI applications. It integrates two key components: LiteLLM and Langfuse. LiteLLM serves as an LLM gateway, providing a unified API interface across various large language models, managing intelligent routing to optimize cost and performance, and ensuring high availability for AI inference. This is crucial for developers working with multiple LLM providers, as it abstracts away the complexities of different APIs and offers mechanisms for failover and load balancing. Langfuse complements this by focusing on the operational aspects post-inference. It delivers detailed observability into LLM calls, allowing developers to trace prompts, responses, and intermediate steps. Furthermore, Langfuse supports prompt versioning, enabling experimentation and tracking changes in prompts over time, which is essential for iterative development and reproducibility. It also provides tools for performance evaluation, helping to quantify the effectiveness and efficiency of LLM-powered features. Together, LiteLLM and Langfuse form a robust stack for building, deploying, monitoring, and refining AI applications, addressing critical needs in LLM observability, prompt management, and gateway functionalities.
https://github.com/langfuse/oss-llmops-stack
Also in LLM Observability & Tracing
comet-ml/opik
Opik is an open-source platform for comprehensive observability, evaluation, and optimization of LLM applications, RAG systems, and agentic workflows.
raga-ai-hub/RagaAI-Catalyst
RagaAI Catalyst is a Python SDK for comprehensive observability, monitoring, and evaluation of AI agents and LLM applications, offering tracing, debugging, and advanced analytics.
Arize-ai/phoenix
Phoenix is an open-source AI observability platform for LLM application experimentation, evaluation, and troubleshooting, providing tracing, evaluation, dataset management, prompt management, and a...
VoltAgent/voltagent
VoltAgent is an end-to-end AI Agent Engineering Platform offering an open-source TypeScript framework for building intelligent agents and a VoltOps Console for observability, automation, deployment...
traceloop/openllmetry
OpenLLMetry provides open-source observability for LLM applications by extending OpenTelemetry to capture traces and metrics from LLM providers, vector databases, and AI frameworks.
Helicone/helicone
Helicone is an open-source LLM observability platform and AI gateway that provides monitoring, evaluation, prompt management, and intelligent routing for large language models.
Agenta-AI/agenta
Agenta is an open-source LLMOps platform designed to accelerate the development of reliable LLM applications, offering integrated prompt management, evaluation, and observability features.