promptfoo/promptfoo
Promptfoo is a CLI and library for evaluating LLM applications, offering automated testing, red teaming, and vulnerability scanning for prompts, models, agents, and RAGs.
Awesome Infra for AI › LLM Evaluation & Testing
msoedov/agentic_security
Agentic Security is a comprehensive open-source tool for identifying and mitigating vulnerabilities in Large Language Models (LLMs) and AI agent workflows. It functions as an AI red teaming kit, enabling developers and security professionals to proactively assess the security posture of their AI systems. The tool supports various attack vectors, including multimodal attacks, which probe vulnerabilities across text, images, and audio inputs. It also specializes in multi-step jailbreaks, simulating sophisticated and iterative attack sequences to uncover weaknesses in LLM safety mechanisms. Additionally, Agentic Security offers robust fuzzing capabilities to stress-test LLMs with randomized inputs, identifying edge cases and unexpected behaviors. It provides API integration and stress testing features, allowing seamless connection to LLM APIs and pushing their limits with high-volume, real-world attack scenarios. Further, it incorporates RL-Based Attacks, leveraging reinforcement learning to craft adaptive, intelligent probes that evolve with the model’s defenses. The project allows users to integrate their own datasets for custom testing and provides a CI check functionality to embed security testing into continuous integration pipelines, ensuring safer and more reliable AI deployments. It comes with a user interface and command-line interface for easy interaction and configuration.
https://github.com/msoedov/agentic_security
Also in LLM Evaluation & Testing
ifixai-ai/iFixAi
iFixAi is a diagnostic tool that evaluates AI models and agents for operational misalignment, including fabrication, manipulation, deception, unpredictability, and opacity, by running up to 45 insp...
confident-ai/deepeval
DeepEval is an open-source LLM evaluation framework, offering a variety of metrics and tools for assessing the performance of AI agents, RAG pipelines, and chatbots through unit testing.
vibrantlabsai/ragas
Ragas is an evaluation framework for LLM applications that provides objective metrics, test data generation, and feedback loops for continuous improvement.
NVIDIA/garak
Garak is an open-source LLM vulnerability scanner designed to red-team and assess generative AI models for weaknesses like hallucination, data leakage, prompt injection, and toxicity.
evidentlyai/evidently
Evidently is an open-source Python framework for evaluating, testing, and monitoring ML and LLM systems, providing comprehensive data and model quality checks from experiments to production.
open-compass/opencompass
Open-source LLM evaluation platform for running standardized benchmarks across models, with configurable datasets, prompt templates, and an official public leaderboard.
Giskard-AI/giskard-oss
Giskard is an open-source Python library for testing and evaluating agentic systems and LLM applications, offering tools for scenario-based testing, red teaming, and vulnerability scanning.