The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injections
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912640468516864 |
|---|---|
| author | Nasr, Milad Carlini, Nicholas Sitawarin, Chawin Schulhoff, Sander V. Hayes, Jamie Ilie, Michael Pluto, Juliette Song, Shuang Chaudhari, Harsh Shumailov, Ilia Thakurta, Abhradeep Xiao, Kai Yuanqing Terzis, Andreas Tramèr, Florian |
| author_facet | Nasr, Milad Carlini, Nicholas Sitawarin, Chawin Schulhoff, Sander V. Hayes, Jamie Ilie, Michael Pluto, Juliette Song, Shuang Chaudhari, Harsh Shumailov, Ilia Thakurta, Abhradeep Xiao, Kai Yuanqing Terzis, Andreas Tramèr, Florian |
| contents | How should we evaluate the robustness of language model defenses? Current defenses against jailbreaks and prompt injections (which aim to prevent an attacker from eliciting harmful knowledge or remotely triggering malicious actions, respectively) are typically evaluated either against a static set of harmful attack strings, or against computationally weak optimization methods that were not designed with the defense in mind. We argue that this evaluation process is flawed.
Instead, we should evaluate defenses against adaptive attackers who explicitly modify their attack strategy to counter a defense's design while spending considerable resources to optimize their objective. By systematically tuning and scaling general optimization techniques-gradient descent, reinforcement learning, random search, and human-guided exploration-we bypass 12 recent defenses (based on a diverse set of techniques) with attack success rate above 90% for most; importantly, the majority of defenses originally reported near-zero attack success rates. We believe that future defense work must consider stronger attacks, such as the ones we describe, in order to make reliable and convincing claims of robustness. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_09023 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injections Nasr, Milad Carlini, Nicholas Sitawarin, Chawin Schulhoff, Sander V. Hayes, Jamie Ilie, Michael Pluto, Juliette Song, Shuang Chaudhari, Harsh Shumailov, Ilia Thakurta, Abhradeep Xiao, Kai Yuanqing Terzis, Andreas Tramèr, Florian Machine Learning Cryptography and Security How should we evaluate the robustness of language model defenses? Current defenses against jailbreaks and prompt injections (which aim to prevent an attacker from eliciting harmful knowledge or remotely triggering malicious actions, respectively) are typically evaluated either against a static set of harmful attack strings, or against computationally weak optimization methods that were not designed with the defense in mind. We argue that this evaluation process is flawed. Instead, we should evaluate defenses against adaptive attackers who explicitly modify their attack strategy to counter a defense's design while spending considerable resources to optimize their objective. By systematically tuning and scaling general optimization techniques-gradient descent, reinforcement learning, random search, and human-guided exploration-we bypass 12 recent defenses (based on a diverse set of techniques) with attack success rate above 90% for most; importantly, the majority of defenses originally reported near-zero attack success rates. We believe that future defense work must consider stronger attacks, such as the ones we describe, in order to make reliable and convincing claims of robustness. |
| title | The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injections |
| topic | Machine Learning Cryptography and Security |
| url | https://arxiv.org/abs/2510.09023 |