A Representation Engineering Perspective on the Effectiveness of Multi-Turn Jailbreaks
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913926148521984 |
|---|---|
| author | Bullwinkel, Blake Russinovich, Mark Salem, Ahmed Zanella-Beguelin, Santiago Jones, Daniel Severi, Giorgio Kim, Eugenia Hines, Keegan Minnich, Amanda Zunger, Yonatan Kumar, Ram Shankar Siva |
| author_facet | Bullwinkel, Blake Russinovich, Mark Salem, Ahmed Zanella-Beguelin, Santiago Jones, Daniel Severi, Giorgio Kim, Eugenia Hines, Keegan Minnich, Amanda Zunger, Yonatan Kumar, Ram Shankar Siva |
| contents | Recent research has demonstrated that state-of-the-art LLMs and defenses remain susceptible to multi-turn jailbreak attacks. These attacks require only closed-box model access and are often easy to perform manually, posing a significant threat to the safe and secure deployment of LLM-based systems. We study the effectiveness of the Crescendo multi-turn jailbreak at the level of intermediate model representations and find that safety-aligned LMs often represent Crescendo responses as more benign than harmful, especially as the number of conversation turns increases. Our analysis indicates that at each turn, Crescendo prompts tend to keep model outputs in a "benign" region of representation space, effectively tricking the model into fulfilling harmful requests. Further, our results help explain why single-turn jailbreak defenses like circuit breakers are generally ineffective against multi-turn attacks, motivating the development of mitigations that address this generalization gap. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_02956 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | A Representation Engineering Perspective on the Effectiveness of Multi-Turn Jailbreaks Bullwinkel, Blake Russinovich, Mark Salem, Ahmed Zanella-Beguelin, Santiago Jones, Daniel Severi, Giorgio Kim, Eugenia Hines, Keegan Minnich, Amanda Zunger, Yonatan Kumar, Ram Shankar Siva Cryptography and Security Artificial Intelligence Recent research has demonstrated that state-of-the-art LLMs and defenses remain susceptible to multi-turn jailbreak attacks. These attacks require only closed-box model access and are often easy to perform manually, posing a significant threat to the safe and secure deployment of LLM-based systems. We study the effectiveness of the Crescendo multi-turn jailbreak at the level of intermediate model representations and find that safety-aligned LMs often represent Crescendo responses as more benign than harmful, especially as the number of conversation turns increases. Our analysis indicates that at each turn, Crescendo prompts tend to keep model outputs in a "benign" region of representation space, effectively tricking the model into fulfilling harmful requests. Further, our results help explain why single-turn jailbreak defenses like circuit breakers are generally ineffective against multi-turn attacks, motivating the development of mitigations that address this generalization gap. |
| title | A Representation Engineering Perspective on the Effectiveness of Multi-Turn Jailbreaks |
| topic | Cryptography and Security Artificial Intelligence |
| url | https://arxiv.org/abs/2507.02956 |