Jailbreaking Large Language Models Through Alignment Vulnerabilities in Out-of-Distribution Settings
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929688441520128 |
|---|---|
| author | Huang, Yue Tang, Jingyu Chen, Dongping Tang, Bingda Wan, Yao Sun, Lichao Yu, Philip S. Zhang, Xiangliang |
| author_facet | Huang, Yue Tang, Jingyu Chen, Dongping Tang, Bingda Wan, Yao Sun, Lichao Yu, Philip S. Zhang, Xiangliang |
| contents | Recently, Large Language Models (LLMs) have garnered significant attention for their exceptional natural language processing capabilities. However, concerns about their trustworthiness remain unresolved, particularly in addressing ``jailbreaking'' attacks on aligned LLMs. Previous research predominantly relies on scenarios involving white-box LLMs or specific, fixed prompt templates, which are often impractical and lack broad applicability. In this paper, we introduce a straightforward and novel method called ObscurePrompt for jailbreaking LLMs, inspired by the observed fragile alignments in Out-of-Distribution (OOD) data. Specifically, we first formulate the decision boundary in the jailbreaking process and then explore how obscure text affects LLM's ethical decision boundary. ObscurePrompt starts with constructing a base prompt that integrates well-known jailbreaking techniques. Powerful LLMs are then utilized to obscure the original prompt through iterative transformations, aiming to bolster the attack's robustness. Comprehensive experiments show that our approach substantially improves upon previous methods in terms of attack effectiveness, maintaining efficacy against two prevalent defense mechanisms. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_13662 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Jailbreaking Large Language Models Through Alignment Vulnerabilities in Out-of-Distribution Settings Huang, Yue Tang, Jingyu Chen, Dongping Tang, Bingda Wan, Yao Sun, Lichao Yu, Philip S. Zhang, Xiangliang Computation and Language Recently, Large Language Models (LLMs) have garnered significant attention for their exceptional natural language processing capabilities. However, concerns about their trustworthiness remain unresolved, particularly in addressing ``jailbreaking'' attacks on aligned LLMs. Previous research predominantly relies on scenarios involving white-box LLMs or specific, fixed prompt templates, which are often impractical and lack broad applicability. In this paper, we introduce a straightforward and novel method called ObscurePrompt for jailbreaking LLMs, inspired by the observed fragile alignments in Out-of-Distribution (OOD) data. Specifically, we first formulate the decision boundary in the jailbreaking process and then explore how obscure text affects LLM's ethical decision boundary. ObscurePrompt starts with constructing a base prompt that integrates well-known jailbreaking techniques. Powerful LLMs are then utilized to obscure the original prompt through iterative transformations, aiming to bolster the attack's robustness. Comprehensive experiments show that our approach substantially improves upon previous methods in terms of attack effectiveness, maintaining efficacy against two prevalent defense mechanisms. |
| title | Jailbreaking Large Language Models Through Alignment Vulnerabilities in Out-of-Distribution Settings |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2406.13662 |