Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2508.17878 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912552901935104 |
|---|---|
| author | Wang, Honghong Deng, Jing Meng, Fanqin Zheng, Rong |
| author_facet | Wang, Honghong Deng, Jing Meng, Fanqin Zheng, Rong |
| contents | This study investigates fine-tuning self-supervised learn ing (SSL) models using multi-task learning (MTL) to enhance
speech emotion recognition (SER). The framework simultane ously handles four related tasks: emotion recognition, gender
recognition, speaker verification, and automatic speech recog nition. An innovative co-attention module is introduced to dy namically capture the interactions between features from the
primary emotion classification task and auxiliary tasks, en abling context-aware fusion. Moreover, We introduce the Sam ple Weighted Focal Contrastive (SWFC) loss function to ad dress class imbalance and semantic confusion by adjusting sam ple weights for difficult and minority samples. The method is
validated on the Categorical Emotion Recognition task of the
Speech Emotion Recognition in Naturalistic Conditions Chal lenge, showing significant performance improvements. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_17878 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Enhancing Speech Emotion Recognition with Multi-Task Learning and Dynamic Feature Fusion Wang, Honghong Deng, Jing Meng, Fanqin Zheng, Rong Sound This study investigates fine-tuning self-supervised learn ing (SSL) models using multi-task learning (MTL) to enhance speech emotion recognition (SER). The framework simultane ously handles four related tasks: emotion recognition, gender recognition, speaker verification, and automatic speech recog nition. An innovative co-attention module is introduced to dy namically capture the interactions between features from the primary emotion classification task and auxiliary tasks, en abling context-aware fusion. Moreover, We introduce the Sam ple Weighted Focal Contrastive (SWFC) loss function to ad dress class imbalance and semantic confusion by adjusting sam ple weights for difficult and minority samples. The method is validated on the Categorical Emotion Recognition task of the Speech Emotion Recognition in Naturalistic Conditions Chal lenge, showing significant performance improvements. |
| title | Enhancing Speech Emotion Recognition with Multi-Task Learning and Dynamic Feature Fusion |
| topic | Sound |
| url | https://arxiv.org/abs/2508.17878 |