Bimodal Connection Attention Fusion for Speech Emotion Recognition
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910889369665536 |
|---|---|
| author | Luo, Jiachen Phan, Huy Wang, Lin Reiss, Joshua D. |
| author_facet | Luo, Jiachen Phan, Huy Wang, Lin Reiss, Joshua D. |
| contents | Multi-modal emotion recognition is challenging due to the difficulty of extracting features that capture subtle emotional differences. Understanding multi-modal interactions and connections is key to building effective bimodal speech emotion recognition systems. In this work, we propose Bimodal Connection Attention Fusion (BCAF) method, which includes three main modules: the interactive connection network, the bimodal attention network, and the correlative attention network. The interactive connection network uses an encoder-decoder architecture to model modality connections between audio and text while leveraging modality-specific features. The bimodal attention network enhances semantic complementation and exploits intra- and inter-modal interactions. The correlative attention network reduces cross-modal noise and captures correlations between audio and text. Experiments on the MELD and IEMOCAP datasets demonstrate that the proposed BCAF method outperforms existing state-of-the-art baselines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_05858 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Bimodal Connection Attention Fusion for Speech Emotion Recognition Luo, Jiachen Phan, Huy Wang, Lin Reiss, Joshua D. Sound Artificial Intelligence Computation and Language Multimedia Audio and Speech Processing Multi-modal emotion recognition is challenging due to the difficulty of extracting features that capture subtle emotional differences. Understanding multi-modal interactions and connections is key to building effective bimodal speech emotion recognition systems. In this work, we propose Bimodal Connection Attention Fusion (BCAF) method, which includes three main modules: the interactive connection network, the bimodal attention network, and the correlative attention network. The interactive connection network uses an encoder-decoder architecture to model modality connections between audio and text while leveraging modality-specific features. The bimodal attention network enhances semantic complementation and exploits intra- and inter-modal interactions. The correlative attention network reduces cross-modal noise and captures correlations between audio and text. Experiments on the MELD and IEMOCAP datasets demonstrate that the proposed BCAF method outperforms existing state-of-the-art baselines. |
| title | Bimodal Connection Attention Fusion for Speech Emotion Recognition |
| topic | Sound Artificial Intelligence Computation and Language Multimedia Audio and Speech Processing |
| url | https://arxiv.org/abs/2503.05858 |