Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2410.20140 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915532948635648 |
|---|---|
| author | Lakara, Kumud Channing, Georgia Rupprecht, Christian Sock, Juil Torr, Philip Collomosse, John de Witt, Christian Schroeder |
| author_facet | Lakara, Kumud Channing, Georgia Rupprecht, Christian Sock, Juil Torr, Philip Collomosse, John de Witt, Christian Schroeder |
| contents | One of the most challenging forms of misinformation involves pairing images with misleading text to create false narratives. Existing AI-driven detection systems often require domain-specific finetuning, limiting generalizability, and offer little insight into their decisions, hindering trust and adoption. We introduce MAD-Sherlock, a multi-agent debate system for out-of-context misinformation detection. MAD-Sherlock frames detection as a multi-agent debate, reflecting the diverse and conflicting discourse found online. Multimodal agents collaborate to assess contextual consistency and retrieve external information to support cross-context reasoning. Our framework is domain- and time-agnostic, requiring no finetuning, yet achieves state-of-the-art accuracy with in-depth explanations. Evaluated on NewsCLIPpings, VERITE, and MMFakeBench, it outperforms prior methods by 2%, 3%, and 5%, respectively. Ablation and user studies show that the debate and resultant explanations significantly improve detection performance and improve trust for both experts and non-experts, positioning MAD-Sherlock as a robust tool for autonomous citizen intelligence. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_20140 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | MAD-Sherlock: Multi-Agent Debate for Visual Misinformation Detection Lakara, Kumud Channing, Georgia Rupprecht, Christian Sock, Juil Torr, Philip Collomosse, John de Witt, Christian Schroeder Artificial Intelligence One of the most challenging forms of misinformation involves pairing images with misleading text to create false narratives. Existing AI-driven detection systems often require domain-specific finetuning, limiting generalizability, and offer little insight into their decisions, hindering trust and adoption. We introduce MAD-Sherlock, a multi-agent debate system for out-of-context misinformation detection. MAD-Sherlock frames detection as a multi-agent debate, reflecting the diverse and conflicting discourse found online. Multimodal agents collaborate to assess contextual consistency and retrieve external information to support cross-context reasoning. Our framework is domain- and time-agnostic, requiring no finetuning, yet achieves state-of-the-art accuracy with in-depth explanations. Evaluated on NewsCLIPpings, VERITE, and MMFakeBench, it outperforms prior methods by 2%, 3%, and 5%, respectively. Ablation and user studies show that the debate and resultant explanations significantly improve detection performance and improve trust for both experts and non-experts, positioning MAD-Sherlock as a robust tool for autonomous citizen intelligence. |
| title | MAD-Sherlock: Multi-Agent Debate for Visual Misinformation Detection |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2410.20140 |