Awesome Infra for AI › LLM Observability & Tracing

tma1-ai/tma1

⭐ 119 Go added to this list on 2026-06-29 repository created 2026-03-13

TMA1 is a self-hosted, local-first observability solution specifically designed for LLM agents. Its primary function is to record every LLM call made by an agent and then feed relevant information back into the agent's subsequent turns through custom hooks and a set of Machine Co-Pilot (MCP) tools. The project aims to provide both human-readable insights via a dashboard and actionable context for the agent itself. The dashboard offers comprehensive monitoring capabilities, including token usage, cost analysis, latency tracking, tool activity, conversation replay, anomaly detection, and prompt evaluation. This allows developers and operators to understand agent behavior and performance in detail. For agents, TMA1 injects a compact `` block before each turn, which can be configured to include session state, anomalies, build statuses, and other relevant environmental changes. The MCP tools, exposed via an integral server, allow agents to request specific contextual information on demand, such as `get_context_bundle`, `get_session_state`, `get_anomalies`, and `get_project_state`. TMA1 supports various LLM sources like Claude Code, Codex, and Copilot CLI, primarily through OpenTelemetry traces, metrics, and logs, as well as JSONL transcripts. All data is stored locally in GreptimeDB, which is bundled with the application. The tool also includes a build sensor to capture development and test output, feeding build failures back to the agent for proactive issue resolution. Its "one binary, no Docker, no Grafana, no cloud account" philosophy emphasizes ease of self-hosting and local development.

https://github.com/tma1-ai/tma1

agent-loopagent-observabilityai-agentsllm-observabilityllmopslocal-firstlogsmetricsobservabilityopentelemetryself-hostedtracingmcpprompt-evaluation

Also in LLM Observability & Tracing

langfuse/langfuse

Langfuse is an open-source LLM engineering platform for developing, monitoring, evaluating, and debugging AI applications, offering observability, prompt management, and evaluation capabilities.

comet-ml/opik

Opik is an open-source platform for comprehensive observability, evaluation, and optimization of LLM applications, RAG systems, and agentic workflows.

raga-ai-hub/RagaAI-Catalyst

RagaAI Catalyst is a Python SDK for comprehensive observability, monitoring, and evaluation of AI agents and LLM applications, offering tracing, debugging, and advanced analytics.

Arize-ai/phoenix

Phoenix is an open-source AI observability platform for LLM application experimentation, evaluation, and troubleshooting, providing tracing, evaluation, dataset management, prompt management, and a...

VoltAgent/voltagent

VoltAgent is an end-to-end AI Agent Engineering Platform offering an open-source TypeScript framework for building intelligent agents and a VoltOps Console for observability, automation, deployment...

traceloop/openllmetry

OpenLLMetry provides open-source observability for LLM applications by extending OpenTelemetry to capture traces and metrics from LLM providers, vector databases, and AI frameworks.

Helicone/helicone

Helicone is an open-source LLM observability platform and AI gateway that provides monitoring, evaluation, prompt management, and intelligent routing for large language models.

Agenta-AI/agenta

Agenta is an open-source LLMOps platform designed to accelerate the development of reliable LLM applications, offering integrated prompt management, evaluation, and observability features.