Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | https://arxiv.org/abs/2507.11968 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866908451912810496 |
|---|---|
| author | Mustakim, Sahid Hossain Islam, S M Jishanul Muna, Ummay Maria Chowdhury, Montasir Islam, Mohammed Jawwadul Ahmmed, Sadia Sikder, Tashfia Dhrubo, Syed Tasdid Azam Shatabda, Swakkhar |
| author_facet | Mustakim, Sahid Hossain Islam, S M Jishanul Muna, Ummay Maria Chowdhury, Montasir Islam, Mohammed Jawwadul Ahmmed, Sadia Sikder, Tashfia Dhrubo, Syed Tasdid Azam Shatabda, Swakkhar |
| contents | Multimodal Large Language Models (MLLMs) are increasingly used for content moderation, yet their robustness in short-form video contexts remains underexplored. Current safety evaluations often rely on unimodal attacks, failing to address combined attack vulnerabilities. In this paper, we introduce a comprehensive framework for evaluating the tri-modal safety of MLLMs. First, we present the Short-Video Multimodal Adversarial (SVMA) dataset, comprising diverse short-form videos with human-guided synthetic adversarial attacks. Second, we propose ChimeraBreak, a novel tri-modal attack strategy that simultaneously challenges visual, auditory, and semantic reasoning pathways. Extensive experiments on state-of-the-art MLLMs reveal significant vulnerabilities with high Attack Success Rates (ASR). Our findings uncover distinct failure modes, showing model biases toward misclassifying benign or policy-violating content. We assess results using LLM-as-a-judge, demonstrating attack reasoning efficacy. Our dataset and findings provide crucial insights for developing more robust and safe MLLMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_11968 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Watch, Listen, Understand, Mislead: Tri-modal Adversarial Attacks on Short Videos for Content Appropriateness Evaluation Mustakim, Sahid Hossain Islam, S M Jishanul Muna, Ummay Maria Chowdhury, Montasir Islam, Mohammed Jawwadul Ahmmed, Sadia Sikder, Tashfia Dhrubo, Syed Tasdid Azam Shatabda, Swakkhar Computer Vision and Pattern Recognition Multimodal Large Language Models (MLLMs) are increasingly used for content moderation, yet their robustness in short-form video contexts remains underexplored. Current safety evaluations often rely on unimodal attacks, failing to address combined attack vulnerabilities. In this paper, we introduce a comprehensive framework for evaluating the tri-modal safety of MLLMs. First, we present the Short-Video Multimodal Adversarial (SVMA) dataset, comprising diverse short-form videos with human-guided synthetic adversarial attacks. Second, we propose ChimeraBreak, a novel tri-modal attack strategy that simultaneously challenges visual, auditory, and semantic reasoning pathways. Extensive experiments on state-of-the-art MLLMs reveal significant vulnerabilities with high Attack Success Rates (ASR). Our findings uncover distinct failure modes, showing model biases toward misclassifying benign or policy-violating content. We assess results using LLM-as-a-judge, demonstrating attack reasoning efficacy. Our dataset and findings provide crucial insights for developing more robust and safe MLLMs. |
| title | Watch, Listen, Understand, Mislead: Tri-modal Adversarial Attacks on Short Videos for Content Appropriateness Evaluation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2507.11968 |