OV-MER: Towards Open-Vocabulary Multimodal Emotion Recognition
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866912364207538176 |
|---|---|
| author | Lian, Zheng Sun, Haiyang Sun, Licai Chen, Haoyu Chen, Lan Gu, Hao Wen, Zhuofan Chen, Shun Zhang, Siyuan Yao, Hailiang Liu, Bin Liu, Rui Liang, Shan Li, Ya Yi, Jiangyan Tao, Jianhua |
| author_facet | Lian, Zheng Sun, Haiyang Sun, Licai Chen, Haoyu Chen, Lan Gu, Hao Wen, Zhuofan Chen, Shun Zhang, Siyuan Yao, Hailiang Liu, Bin Liu, Rui Liang, Shan Li, Ya Yi, Jiangyan Tao, Jianhua |
| contents | Multimodal Emotion Recognition (MER) is a critical research area that seeks to decode human emotions from diverse data modalities. However, existing machine learning methods predominantly rely on predefined emotion taxonomies, which fail to capture the inherent complexity, subtlety, and multi-appraisal nature of human emotional experiences, as demonstrated by studies in psychology and cognitive science. To overcome this limitation, we advocate for introducing the concept of open vocabulary into MER. This paradigm shift aims to enable models to predict emotions beyond a fixed label space, accommodating a flexible set of categories to better reflect the nuanced spectrum of human emotions. To achieve this, we propose a novel paradigm: Open-Vocabulary MER (OV-MER), which enables emotion prediction without being confined to predefined spaces. However, constructing a dataset that encompasses the full range of emotions for OV-MER is practically infeasible; hence, we present a comprehensive solution including a newly curated database, novel evaluation metrics, and a preliminary benchmark. By advancing MER from basic emotions to more nuanced and diverse emotional states, we hope this work can inspire the next generation of MER, enhancing its generalizability and applicability in real-world scenarios. Code and dataset are available at: https://github.com/zeroQiaoba/AffectGPT. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_01495 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | OV-MER: Towards Open-Vocabulary Multimodal Emotion Recognition Lian, Zheng Sun, Haiyang Sun, Licai Chen, Haoyu Chen, Lan Gu, Hao Wen, Zhuofan Chen, Shun Zhang, Siyuan Yao, Hailiang Liu, Bin Liu, Rui Liang, Shan Li, Ya Yi, Jiangyan Tao, Jianhua Human-Computer Interaction Multimodal Emotion Recognition (MER) is a critical research area that seeks to decode human emotions from diverse data modalities. However, existing machine learning methods predominantly rely on predefined emotion taxonomies, which fail to capture the inherent complexity, subtlety, and multi-appraisal nature of human emotional experiences, as demonstrated by studies in psychology and cognitive science. To overcome this limitation, we advocate for introducing the concept of open vocabulary into MER. This paradigm shift aims to enable models to predict emotions beyond a fixed label space, accommodating a flexible set of categories to better reflect the nuanced spectrum of human emotions. To achieve this, we propose a novel paradigm: Open-Vocabulary MER (OV-MER), which enables emotion prediction without being confined to predefined spaces. However, constructing a dataset that encompasses the full range of emotions for OV-MER is practically infeasible; hence, we present a comprehensive solution including a newly curated database, novel evaluation metrics, and a preliminary benchmark. By advancing MER from basic emotions to more nuanced and diverse emotional states, we hope this work can inspire the next generation of MER, enhancing its generalizability and applicability in real-world scenarios. Code and dataset are available at: https://github.com/zeroQiaoba/AffectGPT. |
| title | OV-MER: Towards Open-Vocabulary Multimodal Emotion Recognition |
| topic | Human-Computer Interaction |
| url | https://arxiv.org/abs/2410.01495 |