promptfoo/promptfoo
Promptfoo is a CLI and library for evaluating LLM applications, offering automated testing, red teaming, and vulnerability scanning for prompts, models, agents, and RAGs.
Awesome Infra for AI › LLM Evaluation & Testing
cyberark/FuzzyAI
FuzzyAI is a specialized LLM security tool focused on automated fuzzing to uncover jailbreaks and security vulnerabilities within large language model APIs. It enables developers and security researchers to proactively test their LLM implementations against various attack vectors. The tool supports comprehensive fuzzing techniques, including mutation-based, generation-based, and intelligent fuzzing, and offers built-in input generation capabilities for exhaustive testing. Its architecture is designed for seamless integration into existing development and testing workflows, while also being extensible to meet unique security requirements. FuzzyAI provides support for a wide range of LLMs across major providers such as Anthropic, OpenAI, Gemini, Azure, Bedrock, AI21, DeepSeek, and Ollama, with mechanisms to easily add support for newer models. The project includes interactive Jupyter notebooks and datasets to aid in testing and provides examples for attacking both prompts and custom REST APIs, detailing argument breakdowns and key features. FuzzyAI is crucial for ensuring the robustness and safety of LLM applications by identifying and addressing potential security flaws related to prompt injection and adversarial attacks.
https://github.com/cyberark/FuzzyAI
Also in LLM Evaluation & Testing
ifixai-ai/iFixAi
iFixAi is a diagnostic tool that evaluates AI models and agents for operational misalignment, including fabrication, manipulation, deception, unpredictability, and opacity, by running up to 45 insp...
confident-ai/deepeval
DeepEval is an open-source LLM evaluation framework, offering a variety of metrics and tools for assessing the performance of AI agents, RAG pipelines, and chatbots through unit testing.
vibrantlabsai/ragas
Ragas is an evaluation framework for LLM applications that provides objective metrics, test data generation, and feedback loops for continuous improvement.
NVIDIA/garak
Garak is an open-source LLM vulnerability scanner designed to red-team and assess generative AI models for weaknesses like hallucination, data leakage, prompt injection, and toxicity.
evidentlyai/evidently
Evidently is an open-source Python framework for evaluating, testing, and monitoring ML and LLM systems, providing comprehensive data and model quality checks from experiments to production.
open-compass/opencompass
Open-source LLM evaluation platform for running standardized benchmarks across models, with configurable datasets, prompt templates, and an official public leaderboard.
Giskard-AI/giskard-oss
Giskard is an open-source Python library for testing and evaluating agentic systems and LLM applications, offering tools for scenario-based testing, red teaming, and vulnerability scanning.