Awesome Infra for AI › AI Safety & Guardrails

superagent-ai/superagent

⭐ 6766 TypeScript repository created 2023-05-10

Superagent is an open-source SDK designed to embed safety and compliance into AI applications. It offers several key features to protect AI systems and their users. The 'Guard' functionality detects and blocks prompt injections, malicious instructions, and unsafe tool calls at runtime, enhancing the resilience of AI agents against adversarial inputs. The 'Redact' feature automatically removes sensitive information such as PII (Personally Identifiable Information), PHI (Protected Health Information), and secrets from text, helping applications comply with data privacy regulations. 'Scan' analyzes repositories to identify AI agent-targeted attacks like repo poisoning and malicious instructions, thereby securing the development and deployment phases. Additionally, a 'Test' feature (coming soon) will enable users to run red team scenarios against production AI agents to discover vulnerabilities proactively. Superagent is designed to work with various LLMs from providers like OpenAI, Anthropic, Google, and Groq, and supports open-weight models for on-premise deployment with low latency. It provides SDKs for TypeScript and Python, a CLI, and an MCP server for integrations, ensuring flexibility and transparency through its MIT license.

https://github.com/superagent-ai/superagent

AI safetyguardrailsprompt injectionPII redactionAI securityLLM securityagent safetyred teamingcompliance

Also in AI Safety & Guardrails

data-privacy-stack/presidio

Presidio is an open-source framework for detecting, redacting, masking, and anonymizing sensitive data (PII) across text, images, and structured data, leveraging NLP and customizable pipelines.

NVIDIA-NeMo/Guardrails

NVIDIA NeMo Guardrails is an open-source toolkit for adding programmable guardrails to LLM-based conversational applications, focusing on safety, security, and controlled dialog.

Tencent/AI-Infra-Guard

AI-Infra-Guard is a full-stack AI red teaming platform providing comprehensive security analysis, vulnerability scanning, and jailbreak evaluation for AI ecosystems and LLMs.

microsoft/agent-governance-toolkit

AI Agent Governance Toolkit (AGT) provides policy enforcement, identity management, execution sandboxing, and reliability engineering to secure autonomous AI agents in production.

FailproofAI/failproofai

Observability and policy enforcement for AI agent harnesses, hooking twelve coding and chat harnesses to record every run and block dangerous tool calls before they execute.

protectai/llm-guard

LLM Guard is a comprehensive open-source security toolkit designed to fortify Large Language Model (LLM) interactions by providing robust sanitization, malicious content detection, data leakage pre...

lennney/stop-that-shit

Multi-platform hook and skill guard for AI coding agents that blocks unrequested work such as generated hashes, checksums and task-scope expansion at the harness hook boundary.

kenryu42/cc-safety-net

A PreToolUse hook that blocks destructive commands and secret access before AI coding agents run them, parsing command semantics so shell wrappers and flag reordering cannot bypass it.