LLM - Blue Team
Introduction
This page is built for blue teamers, AI researchers, and enthusiats of the defensive security side of LLM.
You can find here tools, resource, notes for protect Large Language Models (LLMs).
Open Source
| Date |
Repo |
Description |
Stars |
Watchers |
Link |
| 2026-04-03 |
guardrails |
Adding guardrails to large language models. |
⭐ 6630 |
👁️ 36 |
guardrails |
| 2026-04-02 |
Guardrails |
NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems. |
⭐ 5920 |
👁️ 40 |
Guardrails |
| 2026-03-31 |
PurpleLlama |
Set of tools to assess and improve LLM security. |
⭐ 4102 |
👁️ 68 |
PurpleLlama |
| 2026-03-27 |
openguardrails |
Protect every action your agent takes. |
⭐ 327 |
👁️ 4 |
openguardrails |
| 2025-12-11 |
alibi-detect |
Algorithms for outlier, adversarial and drift detection |
⭐ 2511 |
👁️ 35 |
alibi-detect |
| 2025-10-08 |
granite-guardian |
The Granite Guardian models are designed to detect risks in prompts and responses. |
⭐ 136 |
👁️ 9 |
granite-guardian |
| 2025-09-03 |
llm-guard |
The Security Toolkit for LLM Interactions |
⭐ 2776 |
👁️ 26 |
llm-guard |
| 2025-03-22 |
llm-defense |
An easy-to-use Python framework to defend against jailbreak prompts. |
⭐ 21 |
👁️ 1 |
llm-defense |
| 2024-11-22 |
langkit |
🔍 LangKit: An open-source toolkit for monitoring Large Language Models (LLMs). 📚 Extracts signals from prompts & responses, ensuring safety & security. 🛡️ Features include text quality, relevance metrics, & sentiment analysis. 📊 A comprehensive tool for LLM observability. 👀 |
⭐ 980 |
👁️ 15 |
langkit |
| 2024-11-22 |
langkit |
🔍 LangKit: An open-source toolkit for monitoring Large Language Models (LLMs). 📚 Extracts signals from prompts & responses, ensuring safety & security. 🛡️ Features include text quality, relevance metrics, & sentiment analysis. 📊 A comprehensive tool for LLM observability. 👀 |
⭐ 980 |
👁️ 15 |
langkit |
| 2024-07-16 |
HeimdaLLM |
Constrain LLM output |
⭐ 113 |
👁️ 3 |
HeimdaLLM |
| 2024-01-31 |
vigil-llm |
⚡ Vigil ⚡ Detect prompt injections, jailbreaks, and other potentially risky Large Language Model (LLM) inputs |
⭐ 470 |
👁️ 9 |
vigil-llm |
| 2024-01-31 |
vigil-llm |
⚡ Vigil ⚡ Detect prompt injections, jailbreaks, and other potentially risky Large Language Model (LLM) inputs |
⭐ 470 |
👁️ 9 |
vigil-llm |
| 2024-01-25 |
rebuff |
LLM Prompt Injection Detector |
⭐ 1455 |
👁️ 15 |
rebuff |
| 2024-01-25 |
rebuff |
LLM Prompt Injection Detector |
⭐ 1455 |
👁️ 15 |
rebuff |
Commercial
| Company |
Tool |
Description |
Country of origin |
Main shareholder's country |
Link |
| Lakera |
Lakera Guard |
Usage-based API. The most popular commercial runtime (in-line) firewall for LLM. |
Switzerland |
Israel (Check Point) |
Link |
| Cisco |
Cisco AI Defence |
Enterprise Subscription. Network layer protection integrated with infrastructure. |
USA |
USA |
Link |
| Coralogix |
Coralogix AI |
SaaS Enterprise. Monitoring (Observability) and guardrails. |
Israel |
USA / Israel |
Link |
| HiddenLayer |
AISPM Platform |
Annual Licence. Specialised protection of intellectual property and model integrity. |
USA |
USA |
Link |
| Cato Networks |
Cato AI Security |
Enterprise Licence. Protection of ‘Shadow AI’ and agents built into the SASE platform. |
Israel |
Israel / USA |
Link |
| DataSunrise |
Database AI Security |
Per Instance Licence. Data security (PII masking) in AI-integrated databases. |
USA / Israel |
USA |
Link |
| CrowdStrike |
Falcon AI Guard |
SaaS / Pay-as-you-go. Runtime protection and prompt filtering. |
USA |
USA |
Link |
| Promptfoo |
Promptfoo Guard |
Seat-based / Enterprise. Commercial guardrails |
USA |
USA (Insight Partners) |
Link |
Data privacy in LLMs
A comprehensive list of tools to ensure privacy in LLM:
| Date |
Repo |
Description |
Stars |
Watchers |
Link |
| 2026-03-25 |
LangBiTe |
A Bias Tester framework for LLMs |
⭐ 24 |
👁️ 3 |
LangBiTe |
| 2024-08-12 |
anonLLM |
anonLLM: Anonymize Personally Identifiable Information (PII) for Large Language Model APIs |
⭐ 77 |
👁️ 1 |
anonLLM |
Inspiration & Ideas
Prompt Shields