A Layer-Anchoring Strategy for Enhancing Cross-Lingual Speech Emotion Recognition
Fuente:
arXiv
Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909245511827456 |
|---|---|
| author | Upadhyay, Shreya G. Busso, Carlos Lee, Chi-Chun |
| author_facet | Upadhyay, Shreya G. Busso, Carlos Lee, Chi-Chun |
| contents | Cross-lingual speech emotion recognition (SER) is important for a wide range of everyday applications. While recent SER research relies heavily on large pretrained models for emotion training, existing studies often concentrate solely on the final transformer layer of these models. However, given the task-specific nature and hierarchical architecture of these models, each transformer layer encapsulates different levels of information. Leveraging this hierarchical structure, our study focuses on the information embedded across different layers. Through an examination of layer feature similarity across different languages, we propose a novel strategy called a layer-anchoring mechanism to facilitate emotion transfer in cross-lingual SER tasks. Our approach is evaluated using two distinct language affective corpora (MSP-Podcast and BIIC-Podcast), achieving a best UAR performance of 60.21% on the BIIC-podcast corpus. The analysis uncovers interesting insights into the behavior of popular pretrained models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_04966 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A Layer-Anchoring Strategy for Enhancing Cross-Lingual Speech Emotion Recognition Upadhyay, Shreya G. Busso, Carlos Lee, Chi-Chun Sound Machine Learning Audio and Speech Processing Cross-lingual speech emotion recognition (SER) is important for a wide range of everyday applications. While recent SER research relies heavily on large pretrained models for emotion training, existing studies often concentrate solely on the final transformer layer of these models. However, given the task-specific nature and hierarchical architecture of these models, each transformer layer encapsulates different levels of information. Leveraging this hierarchical structure, our study focuses on the information embedded across different layers. Through an examination of layer feature similarity across different languages, we propose a novel strategy called a layer-anchoring mechanism to facilitate emotion transfer in cross-lingual SER tasks. Our approach is evaluated using two distinct language affective corpora (MSP-Podcast and BIIC-Podcast), achieving a best UAR performance of 60.21% on the BIIC-podcast corpus. The analysis uncovers interesting insights into the behavior of popular pretrained models. |
| title | A Layer-Anchoring Strategy for Enhancing Cross-Lingual Speech Emotion Recognition |
| topic | Sound Machine Learning Audio and Speech Processing |
| url | https://arxiv.org/abs/2407.04966 |