Awesome Infra for AI › Prompt Management

microsoft/prompty

⭐ 1276 TypeScript repository created 2024-04-22

Prompty offers a structured markdown file format (.prompty) designed to enhance the observability, understandability, and portability of LLM prompts. It allows developers to define prompts once and run them across various environments, including VS Code, Python, and TypeScript. The format supports defining model configurations, inputs, tools, and the prompt's body using role markers (system, user, assistant) and templating syntaxes like Jinja2 or Mustache. Prompty includes a VS Code extension that provides features such as prompt scaffolding, live preview of rendered prompts, an interactive chat mode for thread-enabled prompts, and a connections sidebar for managing LLM provider endpoints (OpenAI, Microsoft Foundry, Anthropic). A key feature is its tracing mechanism, which generates `.tracy` files to inspect the full prompt execution pipeline—render, parse, execute, and process—including timing and payloads, crucial for debugging and optimization. The project provides Python and TypeScript runtimes, enabling programmatic invocation and step-by-step control over prompt execution. Prompty focuses squarely on the operational aspects of prompt engineering, making it a dedicated tool for prompt management, development, and debugging in AI applications.

https://github.com/microsoft/prompty

generative-aillm-evaluationllmspromptengineeringpromptyprompt-managementprompt-templatingllm-opsprompt-debuggingllm-tracing

Also in Prompt Management

BoundaryML/baml

BAML (Basically a Made-up Language) is an AI framework that facilitates reliable AI workflows and agents by transforming prompt engineering into schema engineering for structured output generation.

genkit-ai/genkit

Genkit is an open-source framework by Google for building and operating AI-powered applications across multiple languages, featuring unified APIs for various models, structured outputs, multi-modal...

pezzolabs/pezzo

Pezzo is an open-source, cloud-native LLMOps platform for streamlined prompt design, version management, instant delivery, collaboration, troubleshooting, and observability of AI operations.

dot-agent/nextpy

Nextpy is a framework for building self-modifying software, focusing on guardrails, structured outputs, a powerful prompt engine for pre-compiling and session state, and optimized code generation f...

neuron-core/neuron-ai

Neuron is a PHP framework for building and orchestrating AI agents, supporting LLM integration, prompt management, RAG, multi-agent workflows, and observability.

microsoft/aici

AICI (Artificial Intelligence Controller Interface) allows building Wasm-based controllers to constrain and direct LLM output in real time, enabling advanced generation strategies.

patterns-ai-core/langchainrb

Langchain.rb provides a Ruby interface for building LLM-powered applications, offering a unified API for various LLM providers, prompt management, and RAG capabilities.

Open-Source-Legal/OpenContracts

OpenContracts is an open-source document intelligence platform that processes unstructured documents into a programmable citation graph using AI agents, structured extraction, and a Model Context P...