Mitigating GenAI-powered Evidence Pollution for Out-of-Context Multimodal Misinformation Detection
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915120552083456 |
|---|---|
| author | Yan, Zehong Qi, Peng Hsu, Wynne Lee, Mong Li |
| author_facet | Yan, Zehong Qi, Peng Hsu, Wynne Lee, Mong Li |
| contents | While large generative artificial intelligence (GenAI) models have achieved significant success, they also raise growing concerns about online information security due to their potential misuse for generating deceptive content. Out-of-context (OOC) multimodal misinformation detection, which often retrieves Web evidence to identify the repurposing of images in false contexts, faces the issue of reasoning over GenAI-polluted evidence to derive accurate predictions. Existing works simulate GenAI-powered pollution at the claim level with stylistic rewriting to conceal linguistic cues, and ignore evidence-level pollution for such information-seeking applications. In this work, we investigate how polluted evidence affects the performance of existing OOC detectors, revealing a performance degradation of more than 9 percentage points. We propose two strategies, cross-modal evidence reranking and cross-modal claim-evidence reasoning, to address the challenges posed by polluted evidence. Extensive experiments on two benchmark datasets show that these strategies can effectively enhance the robustness of existing out-of-context detectors amidst polluted evidence. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_14728 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Mitigating GenAI-powered Evidence Pollution for Out-of-Context Multimodal Misinformation Detection Yan, Zehong Qi, Peng Hsu, Wynne Lee, Mong Li Multimedia Computation and Language Computer Vision and Pattern Recognition Computers and Society While large generative artificial intelligence (GenAI) models have achieved significant success, they also raise growing concerns about online information security due to their potential misuse for generating deceptive content. Out-of-context (OOC) multimodal misinformation detection, which often retrieves Web evidence to identify the repurposing of images in false contexts, faces the issue of reasoning over GenAI-polluted evidence to derive accurate predictions. Existing works simulate GenAI-powered pollution at the claim level with stylistic rewriting to conceal linguistic cues, and ignore evidence-level pollution for such information-seeking applications. In this work, we investigate how polluted evidence affects the performance of existing OOC detectors, revealing a performance degradation of more than 9 percentage points. We propose two strategies, cross-modal evidence reranking and cross-modal claim-evidence reasoning, to address the challenges posed by polluted evidence. Extensive experiments on two benchmark datasets show that these strategies can effectively enhance the robustness of existing out-of-context detectors amidst polluted evidence. |
| title | Mitigating GenAI-powered Evidence Pollution for Out-of-Context Multimodal Misinformation Detection |
| topic | Multimedia Computation and Language Computer Vision and Pattern Recognition Computers and Society |
| url | https://arxiv.org/abs/2501.14728 |