VCEMO: Multi-Modal Emotion Recognition for Chinese Voiceprints
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909294142685184 |
|---|---|
| author | Tang, Jinghua Zhang, Liyun Lu, Yu Ding, Dian Yang, Lanqing Chen, YiChao Bian, Minjie Li, Xiaoshan Xue, Guangtao |
| author_facet | Tang, Jinghua Zhang, Liyun Lu, Yu Ding, Dian Yang, Lanqing Chen, YiChao Bian, Minjie Li, Xiaoshan Xue, Guangtao |
| contents | Emotion recognition can enhance humanized machine responses to user commands, while voiceprint-based perception systems can be easily integrated into commonly used devices like smartphones and stereos. Despite having the largest number of speakers, there is a noticeable absence of high-quality corpus datasets for emotion recognition using Chinese voiceprints. Hence, this paper introduces the VCEMO dataset to address this deficiency. The proposed dataset is constructed from everyday conversations and comprises over 100 users and 7,747 textual samples. Furthermore, this paper proposes a multimodal-based model as a benchmark, which effectively fuses speech, text, and external knowledge using a co-attention structure. The system employs contrastive learning-based regulation for the uneven distribution of the dataset and the diversity of emotional expressions. The experiments demonstrate the significant improvement of the proposed model over SOTA on the VCEMO and IEMOCAP datasets. Code and dataset will be released for research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_13019 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | VCEMO: Multi-Modal Emotion Recognition for Chinese Voiceprints Tang, Jinghua Zhang, Liyun Lu, Yu Ding, Dian Yang, Lanqing Chen, YiChao Bian, Minjie Li, Xiaoshan Xue, Guangtao Multimedia Human-Computer Interaction Emotion recognition can enhance humanized machine responses to user commands, while voiceprint-based perception systems can be easily integrated into commonly used devices like smartphones and stereos. Despite having the largest number of speakers, there is a noticeable absence of high-quality corpus datasets for emotion recognition using Chinese voiceprints. Hence, this paper introduces the VCEMO dataset to address this deficiency. The proposed dataset is constructed from everyday conversations and comprises over 100 users and 7,747 textual samples. Furthermore, this paper proposes a multimodal-based model as a benchmark, which effectively fuses speech, text, and external knowledge using a co-attention structure. The system employs contrastive learning-based regulation for the uneven distribution of the dataset and the diversity of emotional expressions. The experiments demonstrate the significant improvement of the proposed model over SOTA on the VCEMO and IEMOCAP datasets. Code and dataset will be released for research. |
| title | VCEMO: Multi-Modal Emotion Recognition for Chinese Voiceprints |
| topic | Multimedia Human-Computer Interaction |
| url | https://arxiv.org/abs/2408.13019 |