Universal Adversarial Attack on Aligned Multimodal LLMs
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866908393964306432 |
|---|---|
| author | Rahmatullaev, Temurbek Druzhinina, Polina Kurdiukov, Nikita Mikhalchuk, Matvey Kuznetsov, Andrey Razzhigaev, Anton |
| author_facet | Rahmatullaev, Temurbek Druzhinina, Polina Kurdiukov, Nikita Mikhalchuk, Matvey Kuznetsov, Andrey Razzhigaev, Anton |
| contents | We propose a universal adversarial attack on multimodal Large Language Models (LLMs) that leverages a single optimized image to override alignment safeguards across diverse queries and even multiple models. By backpropagating through the vision encoder and language head, we craft a synthetic image that forces the model to respond with a targeted phrase (e.g., "Sure, here it is") or otherwise unsafe content -- even for harmful prompts. In experiments on the SafeBench and MM-SafetyBench benchmarks, our method achieves higher attack success rates than existing baselines, including text-only universal prompts (e.g., up to 81% on certain models). We further demonstrate cross-model universality by training on several multimodal LLMs simultaneously. Additionally, a multi-answer variant of our approach produces more natural-sounding (yet still malicious) responses. These findings underscore critical vulnerabilities in current multimodal alignment and call for more robust adversarial defenses. We will release code and datasets under the Apache-2.0 license. Warning: some content generated by Multimodal LLMs in this paper may be offensive. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_07987 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Universal Adversarial Attack on Aligned Multimodal LLMs Rahmatullaev, Temurbek Druzhinina, Polina Kurdiukov, Nikita Mikhalchuk, Matvey Kuznetsov, Andrey Razzhigaev, Anton Artificial Intelligence I.2.7; I.2.10 We propose a universal adversarial attack on multimodal Large Language Models (LLMs) that leverages a single optimized image to override alignment safeguards across diverse queries and even multiple models. By backpropagating through the vision encoder and language head, we craft a synthetic image that forces the model to respond with a targeted phrase (e.g., "Sure, here it is") or otherwise unsafe content -- even for harmful prompts. In experiments on the SafeBench and MM-SafetyBench benchmarks, our method achieves higher attack success rates than existing baselines, including text-only universal prompts (e.g., up to 81% on certain models). We further demonstrate cross-model universality by training on several multimodal LLMs simultaneously. Additionally, a multi-answer variant of our approach produces more natural-sounding (yet still malicious) responses. These findings underscore critical vulnerabilities in current multimodal alignment and call for more robust adversarial defenses. We will release code and datasets under the Apache-2.0 license. Warning: some content generated by Multimodal LLMs in this paper may be offensive. |
| title | Universal Adversarial Attack on Aligned Multimodal LLMs |
| topic | Artificial Intelligence I.2.7; I.2.10 |
| url | https://arxiv.org/abs/2502.07987 |