I Came, I Saw, I Explained: Benchmarking Multimodal LLMs on Figurative Meaning in Memes
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866908909779812352 |
|---|---|
| author | Zhou, Shijia Mohammad, Saif M. Plank, Barbara Frassinelli, Diego |
| author_facet | Zhou, Shijia Mohammad, Saif M. Plank, Barbara Frassinelli, Diego |
| contents | Internet memes represent a popular form of multimodal online communication and often use figurative elements to convey layered meaning through the combination of text and images. However, it remains largely unclear how multimodal large language models (MLLMs) combine and interpret visual and textual information to identify figurative meaning in memes. To address this gap, we evaluate eight state-of-the-art generative MLLMs across three datasets on their ability to detect and explain six types of figurative meaning. In addition, we conduct a human evaluation of the explanations generated by these MLLMs, assessing whether the provided reasoning supports the predicted label and whether it remains faithful to the original meme content. Our findings indicate that all models exhibit a strong bias to associate a meme with figurative meaning, even when no such meaning is present. Qualitative analysis further shows that correct predictions are not always accompanied by faithful explanations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_23229 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | I Came, I Saw, I Explained: Benchmarking Multimodal LLMs on Figurative Meaning in Memes Zhou, Shijia Mohammad, Saif M. Plank, Barbara Frassinelli, Diego Computation and Language Internet memes represent a popular form of multimodal online communication and often use figurative elements to convey layered meaning through the combination of text and images. However, it remains largely unclear how multimodal large language models (MLLMs) combine and interpret visual and textual information to identify figurative meaning in memes. To address this gap, we evaluate eight state-of-the-art generative MLLMs across three datasets on their ability to detect and explain six types of figurative meaning. In addition, we conduct a human evaluation of the explanations generated by these MLLMs, assessing whether the provided reasoning supports the predicted label and whether it remains faithful to the original meme content. Our findings indicate that all models exhibit a strong bias to associate a meme with figurative meaning, even when no such meaning is present. Qualitative analysis further shows that correct predictions are not always accompanied by faithful explanations. |
| title | I Came, I Saw, I Explained: Benchmarking Multimodal LLMs on Figurative Meaning in Memes |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2603.23229 |