AVMeme Exam: A Multimodal Multilingual Multicultural Benchmark for LLMs' Contextual and Cultural Knowledge and Thinking
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866908786268045312 |
|---|---|
| author | Jiang, Xilin Wang, Qiaolin Wu, Junkai He, Xiaomin Xu, Zhongweiyang Ma, Yinghao Piao, Minshuo Yang, Kaiyi Zheng, Xiuwen Shimizu, Riki Chen, Yicong Firoozi, Arsalan Mischler, Gavin Dindar, Sukru Samet Antonello, Richard He, Linyang Hsieh, Tsun-An Fan, Xulin Wu, Yulun Ma, Yuesheng Amballa, Chaitanya Chen, Weixiong Hai, Jiarui Li, Ruisi Choudhari, Vishal Han, Cong Li, Yinghao Aaron Flinker, Adeen Elhilali, Mounya Benetos, Emmanouil Hasegawa-Johnson, Mark Choudhury, Romit Roy Mesgarani, Nima |
| author_facet | Jiang, Xilin Wang, Qiaolin Wu, Junkai He, Xiaomin Xu, Zhongweiyang Ma, Yinghao Piao, Minshuo Yang, Kaiyi Zheng, Xiuwen Shimizu, Riki Chen, Yicong Firoozi, Arsalan Mischler, Gavin Dindar, Sukru Samet Antonello, Richard He, Linyang Hsieh, Tsun-An Fan, Xulin Wu, Yulun Ma, Yuesheng Amballa, Chaitanya Chen, Weixiong Hai, Jiarui Li, Ruisi Choudhari, Vishal Han, Cong Li, Yinghao Aaron Flinker, Adeen Elhilali, Mounya Benetos, Emmanouil Hasegawa-Johnson, Mark Choudhury, Romit Roy Mesgarani, Nima |
| contents | Internet audio-visual clips convey meaning through time-varying sound and motion, which extend beyond what text alone can represent. To examine whether AI models can understand such signals in human cultural contexts, we introduce AVMeme Exam, a human-curated benchmark of over one thousand iconic Internet sounds and videos spanning speech, songs, music, and sound effects. Each meme is paired with a unique Q&A assessing levels of understanding from surface content to context and emotion to usage and world knowledge, along with metadata such as original year, transcript, summary, and sensitivity. We systematically evaluate state-of-the-art multimodal large language models (MLLMs) alongside human participants using this benchmark. Our results reveal a consistent limitation: current models perform poorly on textless music and sound effects, and struggle to think in context and in culture compared to surface content. These findings highlight a key gap in human-aligned multimodal intelligence and call for models that can perceive contextually and culturally beyond the surface of what they hear and see. Project page: avmemeexam.github.io/public |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_17645 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | AVMeme Exam: A Multimodal Multilingual Multicultural Benchmark for LLMs' Contextual and Cultural Knowledge and Thinking Jiang, Xilin Wang, Qiaolin Wu, Junkai He, Xiaomin Xu, Zhongweiyang Ma, Yinghao Piao, Minshuo Yang, Kaiyi Zheng, Xiuwen Shimizu, Riki Chen, Yicong Firoozi, Arsalan Mischler, Gavin Dindar, Sukru Samet Antonello, Richard He, Linyang Hsieh, Tsun-An Fan, Xulin Wu, Yulun Ma, Yuesheng Amballa, Chaitanya Chen, Weixiong Hai, Jiarui Li, Ruisi Choudhari, Vishal Han, Cong Li, Yinghao Aaron Flinker, Adeen Elhilali, Mounya Benetos, Emmanouil Hasegawa-Johnson, Mark Choudhury, Romit Roy Mesgarani, Nima Sound Computation and Language Computer Vision and Pattern Recognition Multimedia Audio and Speech Processing Internet audio-visual clips convey meaning through time-varying sound and motion, which extend beyond what text alone can represent. To examine whether AI models can understand such signals in human cultural contexts, we introduce AVMeme Exam, a human-curated benchmark of over one thousand iconic Internet sounds and videos spanning speech, songs, music, and sound effects. Each meme is paired with a unique Q&A assessing levels of understanding from surface content to context and emotion to usage and world knowledge, along with metadata such as original year, transcript, summary, and sensitivity. We systematically evaluate state-of-the-art multimodal large language models (MLLMs) alongside human participants using this benchmark. Our results reveal a consistent limitation: current models perform poorly on textless music and sound effects, and struggle to think in context and in culture compared to surface content. These findings highlight a key gap in human-aligned multimodal intelligence and call for models that can perceive contextually and culturally beyond the surface of what they hear and see. Project page: avmemeexam.github.io/public |
| title | AVMeme Exam: A Multimodal Multilingual Multicultural Benchmark for LLMs' Contextual and Cultural Knowledge and Thinking |
| topic | Sound Computation and Language Computer Vision and Pattern Recognition Multimedia Audio and Speech Processing |
| url | https://arxiv.org/abs/2601.17645 |