SoK: Robustness in Large Language Models against Jailbreak Attacks
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913095020969984 |
|---|---|
| author | Xu, Feiyue Hu, Hongsheng He, Chaoxiang Hang, Sheng Hu, Hanqing Liu, Xiuming Zhao, Yubo Zhou, Zhengyan Zhu, Bin Benjamin Sun, Shi-Feng Gu, Dawu Wang, Shuo |
| author_facet | Xu, Feiyue Hu, Hongsheng He, Chaoxiang Hang, Sheng Hu, Hanqing Liu, Xiuming Zhao, Yubo Zhou, Zhengyan Zhu, Bin Benjamin Sun, Shi-Feng Gu, Dawu Wang, Shuo |
| contents | Large Language Models (LLMs) have achieved remarkable success but remain highly susceptible to jailbreak attacks, in which adversarial prompts coerce models into generating harmful, unethical, or policy-violating outputs. Such attacks pose real-world risks, eroding safety, trust, and regulatory compliance in high-stakes applications. Although a variety of attack and defense methods have been proposed, existing evaluation practices are inadequate, often relying on narrow metrics like attack success rate that fail to capture the multidimensional nature of LLM security. In this paper, we present a systematic taxonomy of jailbreak attacks and defenses and introduce Security Cube, a unified, multi-dimensional framework for comprehensive evaluation of these techniques. We provide detailed comparison tables of existing attacks and defenses, highlighting key insights and open challenges across the literature. Leveraging Security Cube, we conduct benchmark studies on 13 representative attacks and 5 defenses, establishing a clear view of the current landscape encompassing jailbreak attacks, defenses, automated judges, and LLM vulnerabilities. Based on these evaluations, we distill critical findings, identify unresolved problems, and outline promising research directions for enhancing LLM robustness against jailbreak attacks. Our analysis aims to pave the way towards more robust, interpretable, and trustworthy LLM systems. Our code is available at Code. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_05058 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | SoK: Robustness in Large Language Models against Jailbreak Attacks Xu, Feiyue Hu, Hongsheng He, Chaoxiang Hang, Sheng Hu, Hanqing Liu, Xiuming Zhao, Yubo Zhou, Zhengyan Zhu, Bin Benjamin Sun, Shi-Feng Gu, Dawu Wang, Shuo Cryptography and Security Artificial Intelligence Large Language Models (LLMs) have achieved remarkable success but remain highly susceptible to jailbreak attacks, in which adversarial prompts coerce models into generating harmful, unethical, or policy-violating outputs. Such attacks pose real-world risks, eroding safety, trust, and regulatory compliance in high-stakes applications. Although a variety of attack and defense methods have been proposed, existing evaluation practices are inadequate, often relying on narrow metrics like attack success rate that fail to capture the multidimensional nature of LLM security. In this paper, we present a systematic taxonomy of jailbreak attacks and defenses and introduce Security Cube, a unified, multi-dimensional framework for comprehensive evaluation of these techniques. We provide detailed comparison tables of existing attacks and defenses, highlighting key insights and open challenges across the literature. Leveraging Security Cube, we conduct benchmark studies on 13 representative attacks and 5 defenses, establishing a clear view of the current landscape encompassing jailbreak attacks, defenses, automated judges, and LLM vulnerabilities. Based on these evaluations, we distill critical findings, identify unresolved problems, and outline promising research directions for enhancing LLM robustness against jailbreak attacks. Our analysis aims to pave the way towards more robust, interpretable, and trustworthy LLM systems. Our code is available at Code. |
| title | SoK: Robustness in Large Language Models against Jailbreak Attacks |
| topic | Cryptography and Security Artificial Intelligence |
| url | https://arxiv.org/abs/2605.05058 |