EMO-SUPERB: An In-depth Look at Speech Emotion Recognition
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866911793316626432 |
|---|---|
| author | Wu, Haibin Chou, Huang-Cheng Chang, Kai-Wei Goncalves, Lucas Du, Jiawei Jang, Jyh-Shing Roger Lee, Chi-Chun Lee, Hung-Yi |
| author_facet | Wu, Haibin Chou, Huang-Cheng Chang, Kai-Wei Goncalves, Lucas Du, Jiawei Jang, Jyh-Shing Roger Lee, Chi-Chun Lee, Hung-Yi |
| contents | Speech emotion recognition (SER) is a pivotal technology for human-computer interaction systems. However, 80.77% of SER papers yield results that cannot be reproduced. We develop EMO-SUPERB, short for EMOtion Speech Universal PERformance Benchmark, which aims to enhance open-source initiatives for SER. EMO-SUPERB includes a user-friendly codebase to leverage 15 state-of-the-art speech self-supervised learning models (SSLMs) for exhaustive evaluation across six open-source SER datasets. EMO-SUPERB streamlines result sharing via an online leaderboard, fostering collaboration within a community-driven benchmark and thereby enhancing the development of SER. On average, 2.58% of annotations are annotated using natural language. SER relies on classification models and is unable to process natural languages, leading to the discarding of these valuable annotations. We prompt ChatGPT to mimic annotators, comprehend natural language annotations, and subsequently re-label the data. By utilizing labels generated by ChatGPT, we consistently achieve an average relative gain of 3.08% across all settings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_13018 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | EMO-SUPERB: An In-depth Look at Speech Emotion Recognition Wu, Haibin Chou, Huang-Cheng Chang, Kai-Wei Goncalves, Lucas Du, Jiawei Jang, Jyh-Shing Roger Lee, Chi-Chun Lee, Hung-Yi Audio and Speech Processing Sound Speech emotion recognition (SER) is a pivotal technology for human-computer interaction systems. However, 80.77% of SER papers yield results that cannot be reproduced. We develop EMO-SUPERB, short for EMOtion Speech Universal PERformance Benchmark, which aims to enhance open-source initiatives for SER. EMO-SUPERB includes a user-friendly codebase to leverage 15 state-of-the-art speech self-supervised learning models (SSLMs) for exhaustive evaluation across six open-source SER datasets. EMO-SUPERB streamlines result sharing via an online leaderboard, fostering collaboration within a community-driven benchmark and thereby enhancing the development of SER. On average, 2.58% of annotations are annotated using natural language. SER relies on classification models and is unable to process natural languages, leading to the discarding of these valuable annotations. We prompt ChatGPT to mimic annotators, comprehend natural language annotations, and subsequently re-label the data. By utilizing labels generated by ChatGPT, we consistently achieve an average relative gain of 3.08% across all settings. |
| title | EMO-SUPERB: An In-depth Look at Speech Emotion Recognition |
| topic | Audio and Speech Processing Sound |
| url | https://arxiv.org/abs/2402.13018 |