EmoBox: Multilingual Multi-corpus Speech Emotion Recognition Toolkit and Benchmark
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914831648423936 |
|---|---|
| author | Ma, Ziyang Chen, Mingjie Zhang, Hezhao Zheng, Zhisheng Chen, Wenxi Li, Xiquan Ye, Jiaxin Chen, Xie Hain, Thomas |
| author_facet | Ma, Ziyang Chen, Mingjie Zhang, Hezhao Zheng, Zhisheng Chen, Wenxi Li, Xiquan Ye, Jiaxin Chen, Xie Hain, Thomas |
| contents | Speech emotion recognition (SER) is an important part of human-computer interaction, receiving extensive attention from both industry and academia. However, the current research field of SER has long suffered from the following problems: 1) There are few reasonable and universal splits of the datasets, making comparing different models and methods difficult. 2) No commonly used benchmark covers numerous corpus and languages for researchers to refer to, making reproduction a burden. In this paper, we propose EmoBox, an out-of-the-box multilingual multi-corpus speech emotion recognition toolkit, along with a benchmark for both intra-corpus and cross-corpus settings. For intra-corpus settings, we carefully designed the data partitioning for different datasets. For cross-corpus settings, we employ a foundation SER model, emotion2vec, to mitigate annotation errors and obtain a test set that is fully balanced in speakers and emotions distributions. Based on EmoBox, we present the intra-corpus SER results of 10 pre-trained speech models on 32 emotion datasets with 14 languages, and the cross-corpus SER results on 4 datasets with the fully balanced test sets. To the best of our knowledge, this is the largest SER benchmark, across language scopes and quantity scales. We hope that our toolkit and benchmark can facilitate the research of SER in the community. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_07162 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | EmoBox: Multilingual Multi-corpus Speech Emotion Recognition Toolkit and Benchmark Ma, Ziyang Chen, Mingjie Zhang, Hezhao Zheng, Zhisheng Chen, Wenxi Li, Xiquan Ye, Jiaxin Chen, Xie Hain, Thomas Sound Artificial Intelligence Computation and Language Multimedia Audio and Speech Processing Speech emotion recognition (SER) is an important part of human-computer interaction, receiving extensive attention from both industry and academia. However, the current research field of SER has long suffered from the following problems: 1) There are few reasonable and universal splits of the datasets, making comparing different models and methods difficult. 2) No commonly used benchmark covers numerous corpus and languages for researchers to refer to, making reproduction a burden. In this paper, we propose EmoBox, an out-of-the-box multilingual multi-corpus speech emotion recognition toolkit, along with a benchmark for both intra-corpus and cross-corpus settings. For intra-corpus settings, we carefully designed the data partitioning for different datasets. For cross-corpus settings, we employ a foundation SER model, emotion2vec, to mitigate annotation errors and obtain a test set that is fully balanced in speakers and emotions distributions. Based on EmoBox, we present the intra-corpus SER results of 10 pre-trained speech models on 32 emotion datasets with 14 languages, and the cross-corpus SER results on 4 datasets with the fully balanced test sets. To the best of our knowledge, this is the largest SER benchmark, across language scopes and quantity scales. We hope that our toolkit and benchmark can facilitate the research of SER in the community. |
| title | EmoBox: Multilingual Multi-corpus Speech Emotion Recognition Toolkit and Benchmark |
| topic | Sound Artificial Intelligence Computation and Language Multimedia Audio and Speech Processing |
| url | https://arxiv.org/abs/2406.07162 |