Attention Eclipse: Manipulating Attention to Bypass LLM Safety-Alignment
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910838209642496 |
|---|---|
| author | Zaree, Pedram Mamun, Md Abdullah Al Alam, Quazi Mishkatul Dong, Yue Alouani, Ihsen Abu-Ghazaleh, Nael |
| author_facet | Zaree, Pedram Mamun, Md Abdullah Al Alam, Quazi Mishkatul Dong, Yue Alouani, Ihsen Abu-Ghazaleh, Nael |
| contents | Recent research has shown that carefully crafted jailbreak inputs can induce large language models to produce harmful outputs, despite safety measures such as alignment. It is important to anticipate the range of potential Jailbreak attacks to guide effective defenses and accurate assessment of model safety. In this paper, we present a new approach for generating highly effective Jailbreak attacks that manipulate the attention of the model to selectively strengthen or weaken attention among different parts of the prompt. By harnessing attention loss, we develop more effective jailbreak attacks, that are also transferrable. The attacks amplify the success rate of existing Jailbreak algorithms including GCG, AutoDAN, and ReNeLLM, while lowering their generation cost (for example, the amplified GCG attack achieves 91.2% ASR, vs. 67.9% for the original attack on Llama2-7B/AdvBench, using less than a third of the generation time). |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_15334 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Attention Eclipse: Manipulating Attention to Bypass LLM Safety-Alignment Zaree, Pedram Mamun, Md Abdullah Al Alam, Quazi Mishkatul Dong, Yue Alouani, Ihsen Abu-Ghazaleh, Nael Cryptography and Security Artificial Intelligence Machine Learning Recent research has shown that carefully crafted jailbreak inputs can induce large language models to produce harmful outputs, despite safety measures such as alignment. It is important to anticipate the range of potential Jailbreak attacks to guide effective defenses and accurate assessment of model safety. In this paper, we present a new approach for generating highly effective Jailbreak attacks that manipulate the attention of the model to selectively strengthen or weaken attention among different parts of the prompt. By harnessing attention loss, we develop more effective jailbreak attacks, that are also transferrable. The attacks amplify the success rate of existing Jailbreak algorithms including GCG, AutoDAN, and ReNeLLM, while lowering their generation cost (for example, the amplified GCG attack achieves 91.2% ASR, vs. 67.9% for the original attack on Llama2-7B/AdvBench, using less than a third of the generation time). |
| title | Attention Eclipse: Manipulating Attention to Bypass LLM Safety-Alignment |
| topic | Cryptography and Security Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2502.15334 |