PRISM: Programmatic Reasoning with Image Sequence Manipulation for LVLM Jailbreaking
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914456745803776 |
|---|---|
| author | Zou, Quanchen Ying, Zonghao Chen, Moyang Xu, Wenzhuo Xiao, Yisong Li, Yakai Zhang, Deyue Yang, Dongdong Liu, Zhao Zhang, Xiangzheng |
| author_facet | Zou, Quanchen Ying, Zonghao Chen, Moyang Xu, Wenzhuo Xiao, Yisong Li, Yakai Zhang, Deyue Yang, Dongdong Liu, Zhao Zhang, Xiangzheng |
| contents | The increasing sophistication of large vision-language models (LVLMs) has been accompanied by advances in safety alignment mechanisms designed to prevent harmful content generation. However, these defenses remain vulnerable to sophisticated adversarial attacks. Existing jailbreak methods typically rely on direct and semantically explicit prompts, overlooking subtle vulnerabilities in how LVLMs compose information over multiple reasoning steps. In this paper, we propose a novel and effective jailbreak framework inspired by Return-Oriented Programming (ROP) techniques from software security. Our approach decomposes a harmful instruction into a sequence of individually benign visual gadgets. A carefully engineered textual prompt directs the sequence of inputs, prompting the model to integrate the benign visual gadgets through its reasoning process to produce a coherent and harmful output. This makes the malicious intent emergent and difficult to detect from any single component. We validate our method through extensive experiments on established benchmarks including SafeBench and MM-SafetyBench, targeting popular LVLMs. Results show that our approach consistently and substantially outperforms existing baselines on state-of-the-art models, achieving near-perfect attack success rates (over 0.90 on SafeBench) and improving ASR by up to 0.39. Our findings reveal a critical and underexplored vulnerability that exploits the compositional reasoning abilities of LVLMs, highlighting the urgent need for defenses that secure the entire reasoning process. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_21540 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | PRISM: Programmatic Reasoning with Image Sequence Manipulation for LVLM Jailbreaking Zou, Quanchen Ying, Zonghao Chen, Moyang Xu, Wenzhuo Xiao, Yisong Li, Yakai Zhang, Deyue Yang, Dongdong Liu, Zhao Zhang, Xiangzheng Cryptography and Security Computer Vision and Pattern Recognition The increasing sophistication of large vision-language models (LVLMs) has been accompanied by advances in safety alignment mechanisms designed to prevent harmful content generation. However, these defenses remain vulnerable to sophisticated adversarial attacks. Existing jailbreak methods typically rely on direct and semantically explicit prompts, overlooking subtle vulnerabilities in how LVLMs compose information over multiple reasoning steps. In this paper, we propose a novel and effective jailbreak framework inspired by Return-Oriented Programming (ROP) techniques from software security. Our approach decomposes a harmful instruction into a sequence of individually benign visual gadgets. A carefully engineered textual prompt directs the sequence of inputs, prompting the model to integrate the benign visual gadgets through its reasoning process to produce a coherent and harmful output. This makes the malicious intent emergent and difficult to detect from any single component. We validate our method through extensive experiments on established benchmarks including SafeBench and MM-SafetyBench, targeting popular LVLMs. Results show that our approach consistently and substantially outperforms existing baselines on state-of-the-art models, achieving near-perfect attack success rates (over 0.90 on SafeBench) and improving ASR by up to 0.39. Our findings reveal a critical and underexplored vulnerability that exploits the compositional reasoning abilities of LVLMs, highlighting the urgent need for defenses that secure the entire reasoning process. |
| title | PRISM: Programmatic Reasoning with Image Sequence Manipulation for LVLM Jailbreaking |
| topic | Cryptography and Security Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2507.21540 |