langfuse/langfuse
Langfuse is an open-source LLM engineering platform for developing, monitoring, evaluating, and debugging AI applications, offering observability, prompt management, and evaluation capabilities.
Awesome Infra for AI › LLM Observability & Tracing
comet-ml/opik-mcp
Opik MCP Server provides a Model Context Protocol (MCP) server that acts as a bridge between various AI coding assistants (like Claude Code, Cursor, and VS Code Copilot) and the Opik LLM observability and evaluation platform by Comet. Its primary function is to enable developers to interact with Opik directly from their AI assistant's chat interface. This allows for reading LLM traces, logging evaluation scores, managing prompt versions, and querying Opik's AI assistant, Ollie, for investigative insights into model performance. The server facilitates operations such as reading project and trace data, listing experiments, test suites, and prompts, and writing scores, comments, and new prompt versions. It offers tools like 'read' for universal data retrieval, 'list' for browsing collections with filtering and pagination, 'ask_ollie' for complex investigations and cross-entity synthesis, and 'write' for comprehensive data logging and management within Opik. This tool is built for LLM engineers who use Opik and want to streamline their observability and evaluation workflows directly from their coding environment.
https://github.com/comet-ml/opik-mcp
Also in LLM Observability & Tracing
comet-ml/opik
Opik is an open-source platform for comprehensive observability, evaluation, and optimization of LLM applications, RAG systems, and agentic workflows.
raga-ai-hub/RagaAI-Catalyst
RagaAI Catalyst is a Python SDK for comprehensive observability, monitoring, and evaluation of AI agents and LLM applications, offering tracing, debugging, and advanced analytics.
Arize-ai/phoenix
Phoenix is an open-source AI observability platform for LLM application experimentation, evaluation, and troubleshooting, providing tracing, evaluation, dataset management, prompt management, and a...
VoltAgent/voltagent
VoltAgent is an end-to-end AI Agent Engineering Platform offering an open-source TypeScript framework for building intelligent agents and a VoltOps Console for observability, automation, deployment...
traceloop/openllmetry
OpenLLMetry provides open-source observability for LLM applications by extending OpenTelemetry to capture traces and metrics from LLM providers, vector databases, and AI frameworks.
Helicone/helicone
Helicone is an open-source LLM observability platform and AI gateway that provides monitoring, evaluation, prompt management, and intelligent routing for large language models.
Agenta-AI/agenta
Agenta is an open-source LLMOps platform designed to accelerate the development of reliable LLM applications, offering integrated prompt management, evaluation, and observability features.