Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2409.07388 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913008313171968 |
|---|---|
| author | Hu, Guimin Lyu, Weimin Sun, Chang Zhu, Zhihong Gui, Lin Cai, Ruichu Cambria, Erik Seifi, Hasti |
| author_facet | Hu, Guimin Lyu, Weimin Sun, Chang Zhu, Zhihong Gui, Lin Cai, Ruichu Cambria, Erik Seifi, Hasti |
| contents | Multimodal affective computing has gained increasing attention due to its broad applications in understanding human behavior and intentions, particularly in text-centric multimodal scenarios. Existing research spans diverse tasks, modalities, and modeling paradigms, yet lacks a unified perspective. In this survey, we systematically review recent advances from an NLP perspective, focusing on four representative tasks: multimodal sentiment analysis (MSA), multimodal emotion recognition in conversation (MERC), multimodal aspect-based sentiment analysis (MABSA), and multimodal multi-label emotion recognition (MMER). We present a unified view by comparing task formulations, benchmark datasets, and evaluation protocols, and by organizing representative methods into key paradigms, including multitask learning, pre-trained models, knowledge enhancement, and contextual modeling. We further extend the discussion to related directions, such as facial, acoustic, and physiological modalities, as well as emotion cause analysis. Finally, we highlight key challenges and outline promising future directions. To facilitate further research, we release a curated repository of relevant works and resources \footnote{https://anonymous.4open.science/r/Multimodal-Affective-Computing-Survey-9819}. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_07388 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Recent Advances in Multimodal Affective Computing: An NLP Perspective Hu, Guimin Lyu, Weimin Sun, Chang Zhu, Zhihong Gui, Lin Cai, Ruichu Cambria, Erik Seifi, Hasti Computation and Language Multimodal affective computing has gained increasing attention due to its broad applications in understanding human behavior and intentions, particularly in text-centric multimodal scenarios. Existing research spans diverse tasks, modalities, and modeling paradigms, yet lacks a unified perspective. In this survey, we systematically review recent advances from an NLP perspective, focusing on four representative tasks: multimodal sentiment analysis (MSA), multimodal emotion recognition in conversation (MERC), multimodal aspect-based sentiment analysis (MABSA), and multimodal multi-label emotion recognition (MMER). We present a unified view by comparing task formulations, benchmark datasets, and evaluation protocols, and by organizing representative methods into key paradigms, including multitask learning, pre-trained models, knowledge enhancement, and contextual modeling. We further extend the discussion to related directions, such as facial, acoustic, and physiological modalities, as well as emotion cause analysis. Finally, we highlight key challenges and outline promising future directions. To facilitate further research, we release a curated repository of relevant works and resources \footnote{https://anonymous.4open.science/r/Multimodal-Affective-Computing-Survey-9819}. |
| title | Recent Advances in Multimodal Affective Computing: An NLP Perspective |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2409.07388 |