Are Paralinguistic Representations all that is needed for Speech Emotion Recognition?
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866911951587639296 |
|---|---|
| author | Phukan, Orchid Chetia Kashyap, Gautam Siddharth Buduru, Arun Balaji Sharma, Rajesh |
| author_facet | Phukan, Orchid Chetia Kashyap, Gautam Siddharth Buduru, Arun Balaji Sharma, Rajesh |
| contents | Availability of representations from pre-trained models (PTMs) have facilitated substantial progress in speech emotion recognition (SER). Particularly, representations from PTM trained for paralinguistic speech processing have shown state-of-the-art (SOTA) performance for SER. However, such paralinguistic PTM representations haven't been evaluated for SER in linguistic environments other than English. Also, paralinguistic PTM representations haven't been investigated in benchmarks such as SUPERB, EMO-SUPERB, ML-SUPERB for SER. This makes it difficult to access the efficacy of paralinguistic PTM representations for SER in multiple languages. To fill this gap, we perform a comprehensive comparative study of five SOTA PTM representations. Our results shows that paralinguistic PTM (TRILLsson) representations performs the best and this performance can be attributed to its effectiveness in capturing pitch, tone and other speech characteristics more effectively than other PTM representations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_01579 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Are Paralinguistic Representations all that is needed for Speech Emotion Recognition? Phukan, Orchid Chetia Kashyap, Gautam Siddharth Buduru, Arun Balaji Sharma, Rajesh Audio and Speech Processing Computation and Language Sound Availability of representations from pre-trained models (PTMs) have facilitated substantial progress in speech emotion recognition (SER). Particularly, representations from PTM trained for paralinguistic speech processing have shown state-of-the-art (SOTA) performance for SER. However, such paralinguistic PTM representations haven't been evaluated for SER in linguistic environments other than English. Also, paralinguistic PTM representations haven't been investigated in benchmarks such as SUPERB, EMO-SUPERB, ML-SUPERB for SER. This makes it difficult to access the efficacy of paralinguistic PTM representations for SER in multiple languages. To fill this gap, we perform a comprehensive comparative study of five SOTA PTM representations. Our results shows that paralinguistic PTM (TRILLsson) representations performs the best and this performance can be attributed to its effectiveness in capturing pitch, tone and other speech characteristics more effectively than other PTM representations. |
| title | Are Paralinguistic Representations all that is needed for Speech Emotion Recognition? |
| topic | Audio and Speech Processing Computation and Language Sound |
| url | https://arxiv.org/abs/2402.01579 |