Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2404.16369 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915368088371200 |
|---|---|
| author | Zhou, Yukai Lou, Jian Huang, Zhijie Qin, Zhan Yang, Yibei Wang, Wenjie |
| author_facet | Zhou, Yukai Lou, Jian Huang, Zhijie Qin, Zhan Yang, Yibei Wang, Wenjie |
| contents | Ensuring the safety alignment of Large Language Models (LLMs) is critical for generating responses consistent with human values. However, LLMs remain vulnerable to jailbreaking attacks, where carefully crafted prompts manipulate them into producing toxic content. One category of such attacks reformulates the task as an optimization problem, aiming to elicit affirmative responses from the LLM. However, these methods heavily rely on predefined objectionable behaviors, limiting their effectiveness and adaptability to diverse harmful queries. In this study, we first identify why the vanilla target loss is suboptimal and then propose enhancements to the loss objective. We introduce DSN (Don't Say No) attack, which combines a cosine decay schedule method with refusal suppression to achieve higher success rates. Extensive experiments demonstrate that DSN outperforms baseline attacks and achieves state-of-the-art attack success rates (ASR). DSN also shows strong universality and transferability to unseen datasets and black-box models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_16369 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Don't Say No: Jailbreaking LLM by Suppressing Refusal Zhou, Yukai Lou, Jian Huang, Zhijie Qin, Zhan Yang, Yibei Wang, Wenjie Computation and Language Ensuring the safety alignment of Large Language Models (LLMs) is critical for generating responses consistent with human values. However, LLMs remain vulnerable to jailbreaking attacks, where carefully crafted prompts manipulate them into producing toxic content. One category of such attacks reformulates the task as an optimization problem, aiming to elicit affirmative responses from the LLM. However, these methods heavily rely on predefined objectionable behaviors, limiting their effectiveness and adaptability to diverse harmful queries. In this study, we first identify why the vanilla target loss is suboptimal and then propose enhancements to the loss objective. We introduce DSN (Don't Say No) attack, which combines a cosine decay schedule method with refusal suppression to achieve higher success rates. Extensive experiments demonstrate that DSN outperforms baseline attacks and achieves state-of-the-art attack success rates (ASR). DSN also shows strong universality and transferability to unseen datasets and black-box models. |
| title | Don't Say No: Jailbreaking LLM by Suppressing Refusal |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2404.16369 |