langfuse/langfuse
Langfuse is an open-source LLM engineering platform for developing, monitoring, evaluating, and debugging AI applications, offering observability, prompt management, and evaluation capabilities.
Awesome Infra for AI › LLM Observability & Tracing
Arize-ai/openinference
OpenInference is designed to bring observability to AI applications by extending OpenTelemetry with specific conventions and instrumentation. Its primary goal is to provide deep insights into the invocation of Large Language Models (LLMs) and their surrounding operational context. This includes crucial aspects such as retrieval processes from vector stores, and the interaction with external tools like search engines or various APIs. The project offers a rich specification, defined in markdown files, that is independent of transport mechanisms and file formats, making it versatile for use with JSON, ProtoBuf, and DataFrames. OpenInference distinguishes itself by offering a suite of instrumentations for popular machine learning SDKs and frameworks, with a strong focus on Python. This includes specific instrumentations for key libraries such as OpenAI, OpenAI Agents, Agno Agents, Claude Agent SDK, LlamaIndex, and DSPy, facilitating easy integration into existing AI development workflows. The project also provides foundational components like `openinference-semantic-conventions` for standardized LLM app tracing and `openinference-instrumentation` for reusable utilities and helpers. While natively supported by Arize-ai's Phoenix platform, OpenInference is designed to be compatible with any OpenTelemetry-compatible backend, ensuring broad applicability and flexibility for AI practitioners seeking to monitor and troubleshoot their deployed AI models and agents effectively. It specifically targets the inference and runtime side of AI, providing the necessary tools to understand and optimize the performance and behavior of live AI systems.
https://github.com/Arize-ai/openinference
Also in LLM Observability & Tracing
comet-ml/opik
Opik is an open-source platform for comprehensive observability, evaluation, and optimization of LLM applications, RAG systems, and agentic workflows.
raga-ai-hub/RagaAI-Catalyst
RagaAI Catalyst is a Python SDK for comprehensive observability, monitoring, and evaluation of AI agents and LLM applications, offering tracing, debugging, and advanced analytics.
Arize-ai/phoenix
Phoenix is an open-source AI observability platform for LLM application experimentation, evaluation, and troubleshooting, providing tracing, evaluation, dataset management, prompt management, and a...
VoltAgent/voltagent
VoltAgent is an end-to-end AI Agent Engineering Platform offering an open-source TypeScript framework for building intelligent agents and a VoltOps Console for observability, automation, deployment...
traceloop/openllmetry
OpenLLMetry provides open-source observability for LLM applications by extending OpenTelemetry to capture traces and metrics from LLM providers, vector databases, and AI frameworks.
Helicone/helicone
Helicone is an open-source LLM observability platform and AI gateway that provides monitoring, evaluation, prompt management, and intelligent routing for large language models.
Agenta-AI/agenta
Agenta is an open-source LLMOps platform designed to accelerate the development of reliable LLM applications, offering integrated prompt management, evaluation, and observability features.