Defending Jailbreak Prompts via In-Context Adversarial Game
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866929725195157504 |
|---|---|
| author | Zhou, Yujun Han, Yufei Zhuang, Haomin Guo, Kehan Liang, Zhenwen Bao, Hongyan Zhang, Xiangliang |
| author_facet | Zhou, Yujun Han, Yufei Zhuang, Haomin Guo, Kehan Liang, Zhenwen Bao, Hongyan Zhang, Xiangliang |
| contents | Large Language Models (LLMs) demonstrate remarkable capabilities across diverse applications. However, concerns regarding their security, particularly the vulnerability to jailbreak attacks, persist. Drawing inspiration from adversarial training in deep learning and LLM agent learning processes, we introduce the In-Context Adversarial Game (ICAG) for defending against jailbreaks without the need for fine-tuning. ICAG leverages agent learning to conduct an adversarial game, aiming to dynamically extend knowledge to defend against jailbreaks. Unlike traditional methods that rely on static datasets, ICAG employs an iterative process to enhance both the defense and attack agents. This continuous improvement process strengthens defenses against newly generated jailbreak prompts. Our empirical studies affirm ICAG's efficacy, where LLMs safeguarded by ICAG exhibit significantly reduced jailbreak success rates across various attack scenarios. Moreover, ICAG demonstrates remarkable transferability to other LLMs, indicating its potential as a versatile defense mechanism. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_13148 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Defending Jailbreak Prompts via In-Context Adversarial Game Zhou, Yujun Han, Yufei Zhuang, Haomin Guo, Kehan Liang, Zhenwen Bao, Hongyan Zhang, Xiangliang Machine Learning Cryptography and Security Large Language Models (LLMs) demonstrate remarkable capabilities across diverse applications. However, concerns regarding their security, particularly the vulnerability to jailbreak attacks, persist. Drawing inspiration from adversarial training in deep learning and LLM agent learning processes, we introduce the In-Context Adversarial Game (ICAG) for defending against jailbreaks without the need for fine-tuning. ICAG leverages agent learning to conduct an adversarial game, aiming to dynamically extend knowledge to defend against jailbreaks. Unlike traditional methods that rely on static datasets, ICAG employs an iterative process to enhance both the defense and attack agents. This continuous improvement process strengthens defenses against newly generated jailbreak prompts. Our empirical studies affirm ICAG's efficacy, where LLMs safeguarded by ICAG exhibit significantly reduced jailbreak success rates across various attack scenarios. Moreover, ICAG demonstrates remarkable transferability to other LLMs, indicating its potential as a versatile defense mechanism. |
| title | Defending Jailbreak Prompts via In-Context Adversarial Game |
| topic | Machine Learning Cryptography and Security |
| url | https://arxiv.org/abs/2402.13148 |