Stealthy and Adjustable Text-Guided Backdoor Attacks on Multimodal Pretrained Models
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913011462045696 |
|---|---|
| author | Zhang, Yiyang Yu, Chaojian Hong, Ziming Shao, Yuanjie Peng, Qinmu Liu, Tongliang You, Xinge |
| author_facet | Zhang, Yiyang Yu, Chaojian Hong, Ziming Shao, Yuanjie Peng, Qinmu Liu, Tongliang You, Xinge |
| contents | Multimodal pretrained models are vulnerable to backdoor attacks, yet most existing methods rely on visual or multimodal triggers, which are impractical since visually embedded triggers rarely occur in real-world data. To overcome this limitation, we propose a novel Text-Guided Backdoor (TGB) attack on multimodal pretrained models, where commonly occurring words in textual descriptions serve as backdoor triggers, significantly improving stealthiness and practicality. Furthermore, we introduce visual adversarial perturbations on poisoned samples to modulate the model's learning of textual triggers, enabling a controllable and adjustable TGB attack. Extensive experiments on downstream tasks built upon multimodal pretrained models, including Composed Image Retrieval (CIR) and Visual Question Answering (VQA), demonstrate that TGB achieves practicality and stealthiness with adjustable attack success rates across diverse realistic settings, revealing critical security vulnerabilities in multimodal pretrained models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_05809 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Stealthy and Adjustable Text-Guided Backdoor Attacks on Multimodal Pretrained Models Zhang, Yiyang Yu, Chaojian Hong, Ziming Shao, Yuanjie Peng, Qinmu Liu, Tongliang You, Xinge Cryptography and Security Machine Learning Multimodal pretrained models are vulnerable to backdoor attacks, yet most existing methods rely on visual or multimodal triggers, which are impractical since visually embedded triggers rarely occur in real-world data. To overcome this limitation, we propose a novel Text-Guided Backdoor (TGB) attack on multimodal pretrained models, where commonly occurring words in textual descriptions serve as backdoor triggers, significantly improving stealthiness and practicality. Furthermore, we introduce visual adversarial perturbations on poisoned samples to modulate the model's learning of textual triggers, enabling a controllable and adjustable TGB attack. Extensive experiments on downstream tasks built upon multimodal pretrained models, including Composed Image Retrieval (CIR) and Visual Question Answering (VQA), demonstrate that TGB achieves practicality and stealthiness with adjustable attack success rates across diverse realistic settings, revealing critical security vulnerabilities in multimodal pretrained models. |
| title | Stealthy and Adjustable Text-Guided Backdoor Attacks on Multimodal Pretrained Models |
| topic | Cryptography and Security Machine Learning |
| url | https://arxiv.org/abs/2604.05809 |