X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agents
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908499759333376 |
|---|---|
| author | Rahman, Salman Jiang, Liwei Shiffer, James Liu, Genglin Issaka, Sheriff Parvez, Md Rizwan Palangi, Hamid Chang, Kai-Wei Choi, Yejin Gabriel, Saadia |
| author_facet | Rahman, Salman Jiang, Liwei Shiffer, James Liu, Genglin Issaka, Sheriff Parvez, Md Rizwan Palangi, Hamid Chang, Kai-Wei Choi, Yejin Gabriel, Saadia |
| contents | Multi-turn interactions with language models (LMs) pose critical safety risks, as harmful intent can be strategically spread across exchanges. Yet, the vast majority of prior work has focused on single-turn safety, while adaptability and diversity remain among the key challenges of multi-turn red-teaming. To address these challenges, we present X-Teaming, a scalable framework that systematically explores how seemingly harmless interactions escalate into harmful outcomes and generates corresponding attack scenarios. X-Teaming employs collaborative agents for planning, attack optimization, and verification, achieving state-of-the-art multi-turn jailbreak effectiveness and diversity with success rates up to 98.1% across representative leading open-weight and closed-source models. In particular, X-Teaming achieves a 96.2% attack success rate against the latest Claude 3.7 Sonnet model, which has been considered nearly immune to single-turn attacks. Building on X-Teaming, we introduce XGuard-Train, an open-source multi-turn safety training dataset that is 20x larger than the previous best resource, comprising 30K interactive jailbreaks, designed to enable robust multi-turn safety alignment for LMs. Our work offers essential tools and insights for mitigating sophisticated conversational attacks, advancing the multi-turn safety of LMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_13203 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agents Rahman, Salman Jiang, Liwei Shiffer, James Liu, Genglin Issaka, Sheriff Parvez, Md Rizwan Palangi, Hamid Chang, Kai-Wei Choi, Yejin Gabriel, Saadia Cryptography and Security Artificial Intelligence Computation and Language Machine Learning Multiagent Systems Multi-turn interactions with language models (LMs) pose critical safety risks, as harmful intent can be strategically spread across exchanges. Yet, the vast majority of prior work has focused on single-turn safety, while adaptability and diversity remain among the key challenges of multi-turn red-teaming. To address these challenges, we present X-Teaming, a scalable framework that systematically explores how seemingly harmless interactions escalate into harmful outcomes and generates corresponding attack scenarios. X-Teaming employs collaborative agents for planning, attack optimization, and verification, achieving state-of-the-art multi-turn jailbreak effectiveness and diversity with success rates up to 98.1% across representative leading open-weight and closed-source models. In particular, X-Teaming achieves a 96.2% attack success rate against the latest Claude 3.7 Sonnet model, which has been considered nearly immune to single-turn attacks. Building on X-Teaming, we introduce XGuard-Train, an open-source multi-turn safety training dataset that is 20x larger than the previous best resource, comprising 30K interactive jailbreaks, designed to enable robust multi-turn safety alignment for LMs. Our work offers essential tools and insights for mitigating sophisticated conversational attacks, advancing the multi-turn safety of LMs. |
| title | X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agents |
| topic | Cryptography and Security Artificial Intelligence Computation and Language Machine Learning Multiagent Systems |
| url | https://arxiv.org/abs/2504.13203 |