LLM - Red Team
Wprowadzenie
Ta strona została stworzona dla członków zespołów red team, badaczy sztucznej inteligencji oraz entuzjastów ofensywnych aspektów bezpieczeństwa LLM. Znajdziesz tu narzędzia, ładunki i uwagi dotyczące testowania granic bezpieczeństwa dużych modeli językowych (LLM).
Payloads
Repozytoria z payloadami do wykorzystania podczas testów penetracyjnych:
| Date | Repo | Description | Stars | Watchers | Link |
|---|---|---|---|---|---|
| N/A | Basic-ML-prompt-injections | No description | ⭐ 0 | 👁️ 0 | Basic-ML-prompt-injections |
| 2026-03-18 | BlackFriday-GPTs-Prompts | List of free GPTs that doesn't require plus subscription | ⭐ 9290 | 👁️ 137 | BlackFriday-GPTs-Prompts |
| 2026-03-02 | ChatGPT_DAN | ChatGPT DAN, Jailbreaks prompt | ⭐ 11606 | 👁️ 284 | ChatGPT_DAN |
| 2026-02-17 | L1B3RT4S | TOTALLY HARMLESS LIBERATION PROMPTS FOR GOOD LIL AI'S! |
⭐ 18157 | 👁️ 489 | L1B3RT4S |
| 2026-02-17 | CL4R1T4S | LEAKED SYSTEM PROMPTS FOR CHATGPT, GEMINI, GROK, CLAUDE, PERPLEXITY, CURSOR, DEVIN, REPLIT, AND MORE! - AI SYSTEMS TRANSPARENCY FOR ALL! 👐 | ⭐ 13999 | 👁️ 332 | CL4R1T4S |
| 2026-01-13 | pallms | Payloads for Attacking Large Language Models | ⭐ 130 | 👁️ 2 | pallms |
| 2025-10-29 | Open-Prompt-Injection | This repository provides a benchmark for prompt injection attacks and defenses in LLMs | ⭐ 421 | 👁️ 3 | Open-Prompt-Injection |
| 2024-12-24 | jailbreak_llms | [CCS'24] A dataset consists of 15,140 ChatGPT prompts from Reddit, Discord, websites, and open-source datasets (including 1,405 jailbreak prompts). | ⭐ 3626 | 👁️ 45 | jailbreak_llms |
| 2024-11-10 | Prompt-injection-payloads | These are prompt injection payloads you can use for AI Chatbots | ⭐ 3 | 👁️ 1 | Prompt-injection-payloads |
| 2024-10-23 | ai-exploits | A collection of real world AI/ML exploits for responsibly disclosed vulnerabilities | ⭐ 1704 | 👁️ 38 | ai-exploits |
| 2024-08-02 | Prompt-Injection-Everywhere | Prompt Injections Everywhere | ⭐ 197 | 👁️ 4 | Prompt-Injection-Everywhere |
| 2023-11-22 | prompt-injection | Official repo for Customized but Compromised: Assessing Prompt Injection Risks in User-Designed GPTs | ⭐ 31 | 👁️ 3 | prompt-injection |
Narzędzia
Open Source
| Date | Repo | Description | Stars | Watchers | Link |
|---|---|---|---|---|---|
| 2026-04-03 | giskard-oss | 🐢 Open-Source Evaluation & Testing library for LLM Agents | ⭐ 5216 | 👁️ 39 | giskard-oss |
| 2026-04-03 | garak | the LLM vulnerability scanner | ⭐ 7450 | 👁️ 52 | garak |
| 2026-04-02 | deepteam | DeepTeam is a framework to red team LLMs and LLM systems. | ⭐ 1440 | 👁️ 6 | deepteam |
| 2026-03-27 | spikee | Simple Prompt Injection Kit for Evaluation and Exploitation | ⭐ 164 | 👁️ 8 | spikee |
| 2026-03-25 | PyRIT | The Python Risk Identification Tool for generative AI (PyRIT) is an open source framework built to empower security professionals and engineers to proactively identify risks in generative AI systems. | ⭐ 2 | 👁️ 0 | PyRIT |
| 2026-02-27 | GPTFuzz | Official repo for GPTFUZZER : Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts | ⭐ 575 | 👁️ 5 | GPTFuzz |
| 2026-02-16 | ps-fuzz | Make your GenAI Apps Safe & Secure  Test & harden your system prompt |
⭐ 667 | 👁️ 11 | ps-fuzz |
| 2026-02-16 | ps-fuzz | Make your GenAI Apps Safe & Secure Test & harden your system prompt |
⭐ 667 | 👁️ 11 | ps-fuzz |
| 2026-02-06 | FuzzyAI | A powerful tool for automated LLM fuzzing. It is designed to help developers and security researchers identify and mitigate potential jailbreaks in their LLM APIs. | ⭐ 1295 | 👁️ 19 | FuzzyAI |
| 2026-02-06 | FuzzyAI | A powerful tool for automated LLM fuzzing. It is designed to help developers and security researchers identify and mitigate potential jailbreaks in their LLM APIs. | ⭐ 1295 | 👁️ 19 | FuzzyAI |
| 2026-02-04 | plexiglass | A toolkit for detecting and protecting against vulnerabilities in Large Language Models (LLMs). | ⭐ 154 | 👁️ 5 | plexiglass |
| 2026-02-03 | agentic_security | Agentic LLM Vulnerability Scanner / AI red teaming kit 🧪 | ⭐ 1836 | 👁️ 21 | agentic_security |
| 2026-02-03 | LLMart | LLM Adversarial Robustness Toolkit, a toolkit for evaluating LLM robustness through adversarial testing. | ⭐ 49 | 👁️ 1 | LLMart |
| 2026-01-02 | PentestGPT | Automated Penetration Testing Agentic Framework Powered by Large Language Models | ⭐ 12387 | 👁️ 273 | PentestGPT |
| 2025-12-01 | promptmap | a security scanner for custom LLM applications | ⭐ 1166 | 👁️ 12 | promptmap |
| 2025-11-13 | adversarial-robustness-toolbox | Adversarial Robustness Toolbox (ART) - Python Library for Machine Learning Security - Evasion, Poisoning, Extraction, Inference - Red and Blue Teams | ⭐ 5914 | 👁️ 97 | adversarial-robustness-toolbox |
| 2025-10-29 | Open-Prompt-Injection | This repository provides a benchmark for prompt injection attacks and defenses in LLMs | ⭐ 421 | 👁️ 3 | Open-Prompt-Injection |
| 2025-10-27 | whistleblower | Whistleblower is a offensive security tool for testing against system prompt leakage and capability discovery of an AI application exposed through API. Built for AI engineers, security researchers and folks who want to know what's going on inside the LLM-based app they use daily | ⭐ 151 | 👁️ 3 | whistleblower |
| 2025-02-18 | artkit | Automated prompt-based testing and evaluation of Gen AI applications | ⭐ 165 | 👁️ 6 | artkit |
| 2024-11-04 | jailbreak-evaluation | The jailbreak-evaluation is an easy-to-use Python package for language model jailbreak evaluation. | ⭐ 27 | 👁️ 0 | jailbreak-evaluation |
| 2024-10-23 | prompt-injection | Application which investigates defensive measures against prompt injection attacks on an LLM, with a focus on the exposure of external tools. | ⭐ 34 | 👁️ 2 | prompt-injection |
| 2024-02-12 | LLMFuzzer | 🧠 LLMFuzzer - Fuzzing Framework for Large Language Models 🧠 LLMFuzzer is the first open-source fuzzing framework specifically designed for Large Language Models (LLMs), especially for their integrations in applications via LLM APIs. 🚀💥 | ⭐ 348 | 👁️ 5 | LLMFuzzer |
| 2023-10-16 | haystack | A suite of red teaming and evaluation frameworks for language models | ⭐ 5 | 👁️ 1 | haystack |
| 2023-09-24 | cogsec | ⚡ Vigil ⚡ Detect prompt injections, jailbreaks, and other potentially risky Large Language Model (LLM) inputs | ⭐ 0 | 👁️ 0 | cogsec |
Komercyjne
| Firma | Narzędzie | Opis | Kraj (pochodzenie) | Kraj głównego udziałowca | Link |
|---|---|---|---|---|---|
| Giskard | Ciągła współpraca zespołu red team | Najlepsze rozwiązanie dla firm z UE; skuteczne wykrywanie halucynacji i stronniczości. | Francja | Francja/UE | Link |
| Promptfoo | Red Teaming dla aplikacji AI | Standard dla programistów; ponad 50 rodzajów testów, ogromna biblioteka podpowiedzi. | USA | USA | Link |
| CalypsoAI (F5) | Agentic Warfare | Skalowalny zespół Red Teaming dla agentów AI i systemów klasy korporacyjnej. | Irlandia | USA (F5, Inc.) | Link |
| Lakera | Lakera Red | Ochrona w czasie rzeczywistym. | Szwajcaria | Izrael (Check Point) | Link |
| HiddenLayer | Automated Red Teaming | Ochrona własności intelektualnej modeli i skanowanie artefaktów (skanowanie modeli). | USA | USA | Link |
| Mindgard | Platforma Mindgard | Automatyzacja DAST-AI. | Wielka Brytania | USA / Wielka Brytania | Link |
| Protect AI | Recon | Skalowany Red Teaming dla AI. | USA | USA | Link |
| Cisco | Cisco AI Defense | Dogłębne testy obciążeniowe przed wdrożeniem. | USA | USA (Cisco) | Link |
Security Testing Framework
- LLM Adversarial Testing - Dec 7, 2024
LLM Testing Guidelines
- Mohit0 - Prompt Injection Cheatsheet - Oct 4, 2024
- Offensive ML Playbook - Apr 17, 2025
- Red Teaming LLMs: The Ultimate Step-by-Step LLM Red Teaming Guide - April 8, 2025
Inspiration & Ideas
- Novel Universal Bypass for All Major LLMs - Apr 24, 2025
- Prompt Attack Scenarios (Gist) - Apr 22, 2025
- SpAIware - Apr 17, 2025
- Embrace The Red - Apr 6, 2025
- An Emoji is All You Need… To Hack your LLM - Feb 20, 2025
- Lessons from red teaming 100 generative AI products - January 13, 2025
General Resources
Must-know resources for any AI security enthusiast:
- Learn Prompting - Mar 25, 2025
- Dair AI - Prompt Engineering Guide – Apr 5, 2025
- PayloadsAllTheThings: Prompt Injection – Mar 17, 2025
- LLM Security 101 – Oct 13, 2023
- LLMSecurity.net – Oct 11, 2023
- PIPE: Prompt Injection Penetration Environment – Aug 25, 2023
- LLM Security (by greshake) – Jun 17, 2023
- Prompt Injection PoC (Joseph Thacker) – May 19, 2023
- HuggingFace Red Teaming Blog – Feb 24, 2023
Disclaimer
All content in this repository is for educational and research purposes only.
Use responsibly. Know the law. Stay ethical.