A Comprehensive Study on the Effectiveness of ASR Representations for Noise-Robust Speech Emotion Recognition
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2023
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866918279975534592 |
|---|---|
| author | Shi, Xiaohan He, Jiajun Li, Xingfeng Toda, Tomoki |
| author_facet | Shi, Xiaohan He, Jiajun Li, Xingfeng Toda, Tomoki |
| contents | This paper proposes an efficient attempt to noisy speech emotion recognition (NSER). Conventional NSER approaches have proven effective in mitigating the impact of artificial noise sources, such as white Gaussian noise, but are limited to non-stationary noises in real-world environments due to their complexity and uncertainty. To overcome this limitation, we introduce a new method for NSER by adopting the automatic speech recognition (ASR) model as a noise-robust feature extractor to eliminate non-vocal information in noisy speech. We first obtain intermediate layer information from the ASR model as a feature representation for emotional speech and then apply this representation for the downstream NSER task. Our experimental results show that 1) the proposed method achieves better NSER performance compared with the conventional noise reduction method, 2) outperforms self-supervised learning approaches, and 3) even outperforms text-based approaches using ASR transcription or the ground truth transcription of noisy speech. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2311_07093 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | A Comprehensive Study on the Effectiveness of ASR Representations for Noise-Robust Speech Emotion Recognition Shi, Xiaohan He, Jiajun Li, Xingfeng Toda, Tomoki Sound Computation and Language Audio and Speech Processing This paper proposes an efficient attempt to noisy speech emotion recognition (NSER). Conventional NSER approaches have proven effective in mitigating the impact of artificial noise sources, such as white Gaussian noise, but are limited to non-stationary noises in real-world environments due to their complexity and uncertainty. To overcome this limitation, we introduce a new method for NSER by adopting the automatic speech recognition (ASR) model as a noise-robust feature extractor to eliminate non-vocal information in noisy speech. We first obtain intermediate layer information from the ASR model as a feature representation for emotional speech and then apply this representation for the downstream NSER task. Our experimental results show that 1) the proposed method achieves better NSER performance compared with the conventional noise reduction method, 2) outperforms self-supervised learning approaches, and 3) even outperforms text-based approaches using ASR transcription or the ground truth transcription of noisy speech. |
| title | A Comprehensive Study on the Effectiveness of ASR Representations for Noise-Robust Speech Emotion Recognition |
| topic | Sound Computation and Language Audio and Speech Processing |
| url | https://arxiv.org/abs/2311.07093 |