Large Language Models Meet Contrastive Learning: Zero-Shot Emotion Recognition Across Languages
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915217461477376 |
|---|---|
| author | Zou, Heqing Lv, Fengmao Zheng, Desheng Chng, Eng Siong Rajan, Deepu |
| author_facet | Zou, Heqing Lv, Fengmao Zheng, Desheng Chng, Eng Siong Rajan, Deepu |
| contents | Multilingual speech emotion recognition aims to estimate a speaker's emotional state using a contactless method across different languages. However, variability in voice characteristics and linguistic diversity poses significant challenges for zero-shot speech emotion recognition, especially with multilingual datasets. In this paper, we propose leveraging contrastive learning to refine multilingual speech features and extend large language models for zero-shot multilingual speech emotion estimation. Specifically, we employ a novel two-stage training framework to align speech signals with linguistic features in the emotional space, capturing both emotion-aware and language-agnostic speech representations. To advance research in this field, we introduce a large-scale synthetic multilingual speech emotion dataset, M5SER. Our experiments demonstrate the effectiveness of the proposed method in both speech emotion recognition and zero-shot multilingual speech emotion recognition, including previously unseen datasets and languages. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_21806 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Large Language Models Meet Contrastive Learning: Zero-Shot Emotion Recognition Across Languages Zou, Heqing Lv, Fengmao Zheng, Desheng Chng, Eng Siong Rajan, Deepu Computation and Language Artificial Intelligence Multilingual speech emotion recognition aims to estimate a speaker's emotional state using a contactless method across different languages. However, variability in voice characteristics and linguistic diversity poses significant challenges for zero-shot speech emotion recognition, especially with multilingual datasets. In this paper, we propose leveraging contrastive learning to refine multilingual speech features and extend large language models for zero-shot multilingual speech emotion estimation. Specifically, we employ a novel two-stage training framework to align speech signals with linguistic features in the emotional space, capturing both emotion-aware and language-agnostic speech representations. To advance research in this field, we introduce a large-scale synthetic multilingual speech emotion dataset, M5SER. Our experiments demonstrate the effectiveness of the proposed method in both speech emotion recognition and zero-shot multilingual speech emotion recognition, including previously unseen datasets and languages. |
| title | Large Language Models Meet Contrastive Learning: Zero-Shot Emotion Recognition Across Languages |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2503.21806 |