Awesome Infra for AI › AI Safety & Guardrails

protectai/llm-guard

⭐ 3213 Python repository created 2023-07-27

LLM Guard is an open-source security toolkit for safeguarding interactions with Large Language Models (LLMs) in production. It provides a suite of features to ensure the security, privacy, and integrity of LLM applications. The toolkit focuses on preventing common vulnerabilities and exploits associated with LLMs, such as prompt injection, data leakage, and the generation of harmful or biased content. It offers various 'scanners' for both prompt input and LLM output. Prompt scanners include anonymization of sensitive data, detection of banned code or topics, identification of prompt injection attempts, gibberish detection, and sentiment analysis. Output scanners extend these capabilities to LLM responses, allowing for the detection of bias, malicious URLs, refusal to answer, and factual consistency checks. The tool is designed for easy integration into existing LLM-powered applications and aims to be deployment-ready for production environments. By offering sanitization, threat detection, and prevention mechanisms, LLM Guard enables developers to build more secure and reliable LLM-based systems, ensuring that interactions remain safe and compliant.

https://github.com/protectai/llm-guard

LLM securityguardrailsprompt injectiondata leakage preventioncontent moderationAI safetyLLMOpssecurity toolkit