SafeAligner: Safety Alignment against Jailbreak Attacks via Response Disparity Guidance
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916540916432896 |
|---|---|
| author | Huang, Caishuang Zhao, Wanxu Zheng, Rui Lv, Huijie Zhan, Wenyu Dou, Shihan Li, Sixian Wang, Xiao Zhou, Enyu Ye, Junjie Yang, Yuming Gui, Tao Zhang, Qi Huang, Xuanjing |
| author_facet | Huang, Caishuang Zhao, Wanxu Zheng, Rui Lv, Huijie Zhan, Wenyu Dou, Shihan Li, Sixian Wang, Xiao Zhou, Enyu Ye, Junjie Yang, Yuming Gui, Tao Zhang, Qi Huang, Xuanjing |
| contents | As the development of large language models (LLMs) rapidly advances, securing these models effectively without compromising their utility has become a pivotal area of research. However, current defense strategies against jailbreak attacks (i.e., efforts to bypass security protocols) often suffer from limited adaptability, restricted general capability, and high cost. To address these challenges, we introduce SafeAligner, a methodology implemented at the decoding stage to fortify defenses against jailbreak attacks. We begin by developing two specialized models: the Sentinel Model, which is trained to foster safety, and the Intruder Model, designed to generate riskier responses. SafeAligner leverages the disparity in security levels between the responses from these models to differentiate between harmful and beneficial tokens, effectively guiding the safety alignment by altering the output token distribution of the target model. Extensive experiments show that SafeAligner can increase the likelihood of beneficial tokens, while reducing the occurrence of harmful ones, thereby ensuring secure alignment with minimal loss to generality. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_18118 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | SafeAligner: Safety Alignment against Jailbreak Attacks via Response Disparity Guidance Huang, Caishuang Zhao, Wanxu Zheng, Rui Lv, Huijie Zhan, Wenyu Dou, Shihan Li, Sixian Wang, Xiao Zhou, Enyu Ye, Junjie Yang, Yuming Gui, Tao Zhang, Qi Huang, Xuanjing Cryptography and Security Computation and Language As the development of large language models (LLMs) rapidly advances, securing these models effectively without compromising their utility has become a pivotal area of research. However, current defense strategies against jailbreak attacks (i.e., efforts to bypass security protocols) often suffer from limited adaptability, restricted general capability, and high cost. To address these challenges, we introduce SafeAligner, a methodology implemented at the decoding stage to fortify defenses against jailbreak attacks. We begin by developing two specialized models: the Sentinel Model, which is trained to foster safety, and the Intruder Model, designed to generate riskier responses. SafeAligner leverages the disparity in security levels between the responses from these models to differentiate between harmful and beneficial tokens, effectively guiding the safety alignment by altering the output token distribution of the target model. Extensive experiments show that SafeAligner can increase the likelihood of beneficial tokens, while reducing the occurrence of harmful ones, thereby ensuring secure alignment with minimal loss to generality. |
| title | SafeAligner: Safety Alignment against Jailbreak Attacks via Response Disparity Guidance |
| topic | Cryptography and Security Computation and Language |
| url | https://arxiv.org/abs/2406.18118 |