langfuse/langfuse
Langfuse is an open-source LLM engineering platform for developing, monitoring, evaluating, and debugging AI applications, offering observability, prompt management, and evaluation capabilities.
Awesome Infra for AI › LLM Observability & Tracing
however-yir/knowledgeops-agent
KnowledgeOps Agent is a comprehensive enterprise AI platform built on Spring AI, providing a solid baseline for knowledge operations. It integrates an Agent Workflow Engine, hybrid retrieval (vector, keyword, graph, web), knowledge graph capabilities, and long/short-term memory. The platform supports deep research functionalities, tenant-isolated RAG with asynchronous PDF ingestion, and strong security measures including JWT/API key authentication and RBAC permissions. Observability is a core feature, offering audit trails and integration with Prometheus, Loki, and Tempo for metrics, logs, and traces. Unlike typical RAG demos, KnowledgeOps Agent emphasizes a production-ready, deployable full-stack solution with a Vue-based console, MySQL, Redis/RabbitMQ, and pgvector. It focuses on solving real-world challenges in integrating conversational AI into business workflows, managing PDF/document knowledge with traceable sources, enabling secure and auditable tool execution, and ensuring operational viability with continuous evaluation loops. This platform is suitable for building intelligent customer service, internal knowledge bases, or AI-powered operational assistants, serving as a robust AI platform engineering baseline that can be extended for various enterprise AI applications.
https://github.com/however-yir/knowledgeops-agent
Also in LLM Observability & Tracing
comet-ml/opik
Opik is an open-source platform for comprehensive observability, evaluation, and optimization of LLM applications, RAG systems, and agentic workflows.
raga-ai-hub/RagaAI-Catalyst
RagaAI Catalyst is a Python SDK for comprehensive observability, monitoring, and evaluation of AI agents and LLM applications, offering tracing, debugging, and advanced analytics.
Arize-ai/phoenix
Phoenix is an open-source AI observability platform for LLM application experimentation, evaluation, and troubleshooting, providing tracing, evaluation, dataset management, prompt management, and a...
VoltAgent/voltagent
VoltAgent is an end-to-end AI Agent Engineering Platform offering an open-source TypeScript framework for building intelligent agents and a VoltOps Console for observability, automation, deployment...
traceloop/openllmetry
OpenLLMetry provides open-source observability for LLM applications by extending OpenTelemetry to capture traces and metrics from LLM providers, vector databases, and AI frameworks.
Helicone/helicone
Helicone is an open-source LLM observability platform and AI gateway that provides monitoring, evaluation, prompt management, and intelligent routing for large language models.
Agenta-AI/agenta
Agenta is an open-source LLMOps platform designed to accelerate the development of reliable LLM applications, offering integrated prompt management, evaluation, and observability features.