Multimodal Video Emotion Recognition with Reliable Reasoning Priors
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912521918611456 |
|---|---|
| author | Wang, Zhepeng Zhu, Yingjian Dong, Guanghao Yi, Hongzhu Chen, Feng Wang, Xinming Xie, Jun |
| author_facet | Wang, Zhepeng Zhu, Yingjian Dong, Guanghao Yi, Hongzhu Chen, Feng Wang, Xinming Xie, Jun |
| contents | This study investigates the integration of trustworthy prior reasoning knowledge from MLLMs into multimodal emotion recognition. We employ Gemini to generate fine-grained, modality-separable reasoning traces, which are injected as priors during the fusion stage to enrich cross-modal interactions. To mitigate the pronounced class-imbalance in multimodal emotion recognition, we introduce Balanced Dual-Contrastive Learning, a loss formulation that jointly balances inter-class and intra-class distributions. Applied to the MER2024 benchmark, our prior-enhanced framework yields substantial performance gains, demonstrating that the reliability of MLLM-derived reasoning can be synergistically combined with the domain adaptability of lightweight fusion networks for robust, scalable emotion recognition. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_03722 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Multimodal Video Emotion Recognition with Reliable Reasoning Priors Wang, Zhepeng Zhu, Yingjian Dong, Guanghao Yi, Hongzhu Chen, Feng Wang, Xinming Xie, Jun Computer Vision and Pattern Recognition Artificial Intelligence This study investigates the integration of trustworthy prior reasoning knowledge from MLLMs into multimodal emotion recognition. We employ Gemini to generate fine-grained, modality-separable reasoning traces, which are injected as priors during the fusion stage to enrich cross-modal interactions. To mitigate the pronounced class-imbalance in multimodal emotion recognition, we introduce Balanced Dual-Contrastive Learning, a loss formulation that jointly balances inter-class and intra-class distributions. Applied to the MER2024 benchmark, our prior-enhanced framework yields substantial performance gains, demonstrating that the reliability of MLLM-derived reasoning can be synergistically combined with the domain adaptability of lightweight fusion networks for robust, scalable emotion recognition. |
| title | Multimodal Video Emotion Recognition with Reliable Reasoning Priors |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2508.03722 |