Parameter Efficient Finetuning for Speech Emotion Recognition and Domain Adaptation
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910392139120640 |
|---|---|
| author | Lashkarashvili, Nineli Wu, Wen Sun, Guangzhi Woodland, Philip C. |
| author_facet | Lashkarashvili, Nineli Wu, Wen Sun, Guangzhi Woodland, Philip C. |
| contents | Foundation models have shown superior performance for speech emotion recognition (SER). However, given the limited data in emotion corpora, finetuning all parameters of large pre-trained models for SER can be both resource-intensive and susceptible to overfitting. This paper investigates parameter-efficient finetuning (PEFT) for SER. Various PEFT adaptors are systematically studied for both classification of discrete emotion categories and prediction of dimensional emotional attributes. The results demonstrate that the combination of PEFT methods surpasses full finetuning with a significant reduction in the number of trainable parameters. Furthermore, a two-stage adaptation strategy is proposed to adapt models trained on acted emotion data, which is more readily available, to make the model more adept at capturing natural emotional expressions. Both intra- and cross-corpus experiments validate the efficacy of the proposed approach in enhancing the performance on both the source and target domains. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_11747 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Parameter Efficient Finetuning for Speech Emotion Recognition and Domain Adaptation Lashkarashvili, Nineli Wu, Wen Sun, Guangzhi Woodland, Philip C. Audio and Speech Processing Machine Learning Sound Foundation models have shown superior performance for speech emotion recognition (SER). However, given the limited data in emotion corpora, finetuning all parameters of large pre-trained models for SER can be both resource-intensive and susceptible to overfitting. This paper investigates parameter-efficient finetuning (PEFT) for SER. Various PEFT adaptors are systematically studied for both classification of discrete emotion categories and prediction of dimensional emotional attributes. The results demonstrate that the combination of PEFT methods surpasses full finetuning with a significant reduction in the number of trainable parameters. Furthermore, a two-stage adaptation strategy is proposed to adapt models trained on acted emotion data, which is more readily available, to make the model more adept at capturing natural emotional expressions. Both intra- and cross-corpus experiments validate the efficacy of the proposed approach in enhancing the performance on both the source and target domains. |
| title | Parameter Efficient Finetuning for Speech Emotion Recognition and Domain Adaptation |
| topic | Audio and Speech Processing Machine Learning Sound |
| url | https://arxiv.org/abs/2402.11747 |