MemeIntel: Explainable Detection of Propagandistic and Hateful Memes
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915824929865728 |
|---|---|
| author | Kmainasi, Mohamed Bayan Hasnat, Abul Hasan, Md Arid Shahroor, Ali Ezzat Alam, Firoj |
| author_facet | Kmainasi, Mohamed Bayan Hasnat, Abul Hasan, Md Arid Shahroor, Ali Ezzat Alam, Firoj |
| contents | The proliferation of multimodal content on social media presents significant challenges in understanding and moderating complex, context-dependent issues such as misinformation, hate speech, and propaganda. While efforts have been made to develop resources and propose new methods for automatic detection, limited attention has been given to jointly modeling label detection and the generation of explanation-based rationales, which often leads to degraded classification performance when trained simultaneously. To address this challenge, we introduce MemeXplain, an explanation-enhanced dataset for propagandistic memes in Arabic and hateful memes in English, making it the first large-scale resource for these tasks. To solve these tasks, we propose a multi-stage optimization approach and train Vision-Language Models (VLMs). Our results show that this strategy significantly improves both label detection and explanation generation quality over the base model, outperforming the current state-of-the-art with an absolute improvement of ~1.4% (Acc) on ArMeme and ~2.2% (Acc) on Hateful Memes. For reproducibility and future research, we aim to make the MemeXplain dataset and scripts publicly available (https://github.com/MohamedBayan/MemeIntel). |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_16612 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MemeIntel: Explainable Detection of Propagandistic and Hateful Memes Kmainasi, Mohamed Bayan Hasnat, Abul Hasan, Md Arid Shahroor, Ali Ezzat Alam, Firoj Computation and Language Artificial Intelligence Computer Vision and Pattern Recognition 68T50 I.2.7 The proliferation of multimodal content on social media presents significant challenges in understanding and moderating complex, context-dependent issues such as misinformation, hate speech, and propaganda. While efforts have been made to develop resources and propose new methods for automatic detection, limited attention has been given to jointly modeling label detection and the generation of explanation-based rationales, which often leads to degraded classification performance when trained simultaneously. To address this challenge, we introduce MemeXplain, an explanation-enhanced dataset for propagandistic memes in Arabic and hateful memes in English, making it the first large-scale resource for these tasks. To solve these tasks, we propose a multi-stage optimization approach and train Vision-Language Models (VLMs). Our results show that this strategy significantly improves both label detection and explanation generation quality over the base model, outperforming the current state-of-the-art with an absolute improvement of ~1.4% (Acc) on ArMeme and ~2.2% (Acc) on Hateful Memes. For reproducibility and future research, we aim to make the MemeXplain dataset and scripts publicly available (https://github.com/MohamedBayan/MemeIntel). |
| title | MemeIntel: Explainable Detection of Propagandistic and Hateful Memes |
| topic | Computation and Language Artificial Intelligence Computer Vision and Pattern Recognition 68T50 I.2.7 |
| url | https://arxiv.org/abs/2502.16612 |