Closing the Distribution Gap in Adversarial Training for LLMs
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911455094243328 |
|---|---|
| author | Hu, Chengzhi Dornbusch, Jonas Lüdke, David Günnemann, Stephan Schwinn, Leo |
| author_facet | Hu, Chengzhi Dornbusch, Jonas Lüdke, David Günnemann, Stephan Schwinn, Leo |
| contents | Adversarial training for LLMs is one of the most promising methods to reliably improve robustness against adversaries. However, despite significant progress, models remain vulnerable to simple in-distribution exploits, such as rewriting prompts in the past tense or translating them into other languages. We argue that this persistent fragility stems from a fundamental limitation in current adversarial training algorithms: they minimize adversarial loss on their training set but inadequately cover the data distribution, resulting in vulnerability to seemingly simple attacks. To bridge this gap, we propose Distributional Adversarial Training, DAT. We leverage Diffusion LLMs to approximate the true joint distribution of prompts and responses, enabling generation of diverse, high-likelihood samples that address generalization failures. By combining optimization over the data distribution provided by the diffusion model with continuous adversarial training, DAT achieves substantially higher adversarial robustness than previous methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_15238 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Closing the Distribution Gap in Adversarial Training for LLMs Hu, Chengzhi Dornbusch, Jonas Lüdke, David Günnemann, Stephan Schwinn, Leo Machine Learning Artificial Intelligence Cryptography and Security I.2.6; K.6.5 Adversarial training for LLMs is one of the most promising methods to reliably improve robustness against adversaries. However, despite significant progress, models remain vulnerable to simple in-distribution exploits, such as rewriting prompts in the past tense or translating them into other languages. We argue that this persistent fragility stems from a fundamental limitation in current adversarial training algorithms: they minimize adversarial loss on their training set but inadequately cover the data distribution, resulting in vulnerability to seemingly simple attacks. To bridge this gap, we propose Distributional Adversarial Training, DAT. We leverage Diffusion LLMs to approximate the true joint distribution of prompts and responses, enabling generation of diverse, high-likelihood samples that address generalization failures. By combining optimization over the data distribution provided by the diffusion model with continuous adversarial training, DAT achieves substantially higher adversarial robustness than previous methods. |
| title | Closing the Distribution Gap in Adversarial Training for LLMs |
| topic | Machine Learning Artificial Intelligence Cryptography and Security I.2.6; K.6.5 |
| url | https://arxiv.org/abs/2602.15238 |