JPS: Jailbreak Multimodal Large Language Models with Collaborative Visual Perturbation and Textual Steering
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915433425141760 |
|---|---|
| author | Chen, Renmiao Cui, Shiyao Huang, Xuancheng Pan, Chengwei Huang, Victor Shea-Jay Zhang, QingLin Ouyang, Xuan Zhang, Zhexin Wang, Hongning Huang, Minlie |
| author_facet | Chen, Renmiao Cui, Shiyao Huang, Xuancheng Pan, Chengwei Huang, Victor Shea-Jay Zhang, QingLin Ouyang, Xuan Zhang, Zhexin Wang, Hongning Huang, Minlie |
| contents | Jailbreak attacks against multimodal large language Models (MLLMs) are a significant research focus. Current research predominantly focuses on maximizing attack success rate (ASR), often overlooking whether the generated responses actually fulfill the attacker's malicious intent. This oversight frequently leads to low-quality outputs that bypass safety filters but lack substantial harmful content. To address this gap, we propose JPS, \underline{J}ailbreak MLLMs with collaborative visual \underline{P}erturbation and textual \underline{S}teering, which achieves jailbreaks via corporation of visual image and textually steering prompt. Specifically, JPS utilizes target-guided adversarial image perturbations for effective safety bypass, complemented by "steering prompt" optimized via a multi-agent system to specifically guide LLM responses fulfilling the attackers' intent. These visual and textual components undergo iterative co-optimization for enhanced performance. To evaluate the quality of attack outcomes, we propose the Malicious Intent Fulfillment Rate (MIFR) metric, assessed using a Reasoning-LLM-based evaluator. Our experiments show JPS sets a new state-of-the-art in both ASR and MIFR across various MLLMs and benchmarks, with analyses confirming its efficacy. Codes are available at \href{https://github.com/thu-coai/JPS}{https://github.com/thu-coai/JPS}. \color{warningcolor}{Warning: This paper contains potentially sensitive contents.} |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_05087 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | JPS: Jailbreak Multimodal Large Language Models with Collaborative Visual Perturbation and Textual Steering Chen, Renmiao Cui, Shiyao Huang, Xuancheng Pan, Chengwei Huang, Victor Shea-Jay Zhang, QingLin Ouyang, Xuan Zhang, Zhexin Wang, Hongning Huang, Minlie Multimedia Artificial Intelligence Computation and Language Cryptography and Security I.2.7; K.4.1; K.6.5 Jailbreak attacks against multimodal large language Models (MLLMs) are a significant research focus. Current research predominantly focuses on maximizing attack success rate (ASR), often overlooking whether the generated responses actually fulfill the attacker's malicious intent. This oversight frequently leads to low-quality outputs that bypass safety filters but lack substantial harmful content. To address this gap, we propose JPS, \underline{J}ailbreak MLLMs with collaborative visual \underline{P}erturbation and textual \underline{S}teering, which achieves jailbreaks via corporation of visual image and textually steering prompt. Specifically, JPS utilizes target-guided adversarial image perturbations for effective safety bypass, complemented by "steering prompt" optimized via a multi-agent system to specifically guide LLM responses fulfilling the attackers' intent. These visual and textual components undergo iterative co-optimization for enhanced performance. To evaluate the quality of attack outcomes, we propose the Malicious Intent Fulfillment Rate (MIFR) metric, assessed using a Reasoning-LLM-based evaluator. Our experiments show JPS sets a new state-of-the-art in both ASR and MIFR across various MLLMs and benchmarks, with analyses confirming its efficacy. Codes are available at \href{https://github.com/thu-coai/JPS}{https://github.com/thu-coai/JPS}. \color{warningcolor}{Warning: This paper contains potentially sensitive contents.} |
| title | JPS: Jailbreak Multimodal Large Language Models with Collaborative Visual Perturbation and Textual Steering |
| topic | Multimedia Artificial Intelligence Computation and Language Cryptography and Security I.2.7; K.4.1; K.6.5 |
| url | https://arxiv.org/abs/2508.05087 |