Why We Feel: Breaking Boundaries in Emotional Reasoning with Multimodal Large Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908323103637504 |
|---|---|
| author | Lin, Yuxiang Sun, Jingdong Cheng, Zhi-Qi Wang, Jue Liang, Haomin Cheng, Zebang Dong, Yifei He, Jun-Yan Peng, Xiaojiang Hua, Xian-Sheng |
| author_facet | Lin, Yuxiang Sun, Jingdong Cheng, Zhi-Qi Wang, Jue Liang, Haomin Cheng, Zebang Dong, Yifei He, Jun-Yan Peng, Xiaojiang Hua, Xian-Sheng |
| contents | Most existing emotion analysis emphasizes which emotion arises (e.g., happy, sad, angry) but neglects the deeper why. We propose Emotion Interpretation (EI), focusing on causal factors-whether explicit (e.g., observable objects, interpersonal interactions) or implicit (e.g., cultural context, off-screen events)-that drive emotional responses. Unlike traditional emotion recognition, EI tasks require reasoning about triggers instead of mere labeling. To facilitate EI research, we present EIBench, a large-scale benchmark encompassing 1,615 basic EI samples and 50 complex EI samples featuring multifaceted emotions. Each instance demands rationale-based explanations rather than straightforward categorization. We further propose a Coarse-to-Fine Self-Ask (CFSA) annotation pipeline, which guides Vision-Language Models (VLLMs) through iterative question-answer rounds to yield high-quality labels at scale. Extensive evaluations on open-source and proprietary large language models under four experimental settings reveal consistent performance gaps-especially for more intricate scenarios-underscoring EI's potential to enrich empathetic, context-aware AI applications. Our benchmark and methods are publicly available at: https://github.com/Lum1104/EIBench, offering a foundation for advanced multimodal causal analysis and next-generation affective computing. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_07521 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Why We Feel: Breaking Boundaries in Emotional Reasoning with Multimodal Large Language Models Lin, Yuxiang Sun, Jingdong Cheng, Zhi-Qi Wang, Jue Liang, Haomin Cheng, Zebang Dong, Yifei He, Jun-Yan Peng, Xiaojiang Hua, Xian-Sheng Artificial Intelligence Multimedia Most existing emotion analysis emphasizes which emotion arises (e.g., happy, sad, angry) but neglects the deeper why. We propose Emotion Interpretation (EI), focusing on causal factors-whether explicit (e.g., observable objects, interpersonal interactions) or implicit (e.g., cultural context, off-screen events)-that drive emotional responses. Unlike traditional emotion recognition, EI tasks require reasoning about triggers instead of mere labeling. To facilitate EI research, we present EIBench, a large-scale benchmark encompassing 1,615 basic EI samples and 50 complex EI samples featuring multifaceted emotions. Each instance demands rationale-based explanations rather than straightforward categorization. We further propose a Coarse-to-Fine Self-Ask (CFSA) annotation pipeline, which guides Vision-Language Models (VLLMs) through iterative question-answer rounds to yield high-quality labels at scale. Extensive evaluations on open-source and proprietary large language models under four experimental settings reveal consistent performance gaps-especially for more intricate scenarios-underscoring EI's potential to enrich empathetic, context-aware AI applications. Our benchmark and methods are publicly available at: https://github.com/Lum1104/EIBench, offering a foundation for advanced multimodal causal analysis and next-generation affective computing. |
| title | Why We Feel: Breaking Boundaries in Emotional Reasoning with Multimodal Large Language Models |
| topic | Artificial Intelligence Multimedia |
| url | https://arxiv.org/abs/2504.07521 |