Temporal-Frequency State Space Duality: An Efficient Paradigm for Speech Emotion Recognition
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909438298816512 |
|---|---|
| author | Zhao, Jiaqi Wang, Fei Li, Kun Wei, Yanyan Tang, Shengeng Zhao, Shu Sun, Xiao |
| author_facet | Zhao, Jiaqi Wang, Fei Li, Kun Wei, Yanyan Tang, Shengeng Zhao, Shu Sun, Xiao |
| contents | Speech Emotion Recognition (SER) plays a critical role in enhancing user experience within human-computer interaction. However, existing methods are overwhelmed by temporal domain analysis, overlooking the valuable envelope structures of the frequency domain that are equally important for robust emotion recognition. To overcome this limitation, we propose TF-Mamba, a novel multi-domain framework that captures emotional expressions in both temporal and frequency dimensions.Concretely, we propose a temporal-frequency mamba block to extract temporal- and frequency-aware emotional features, achieving an optimal balance between computational efficiency and model expressiveness. Besides, we design a Complex Metric-Distance Triplet (CMDT) loss to enable the model to capture representative emotional clues for SER. Extensive experiments on the IEMOCAP and MELD datasets show that TF-Mamba surpasses existing methods in terms of model size and latency, providing a more practical solution for future SER applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_16904 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Temporal-Frequency State Space Duality: An Efficient Paradigm for Speech Emotion Recognition Zhao, Jiaqi Wang, Fei Li, Kun Wei, Yanyan Tang, Shengeng Zhao, Shu Sun, Xiao Sound Audio and Speech Processing Speech Emotion Recognition (SER) plays a critical role in enhancing user experience within human-computer interaction. However, existing methods are overwhelmed by temporal domain analysis, overlooking the valuable envelope structures of the frequency domain that are equally important for robust emotion recognition. To overcome this limitation, we propose TF-Mamba, a novel multi-domain framework that captures emotional expressions in both temporal and frequency dimensions.Concretely, we propose a temporal-frequency mamba block to extract temporal- and frequency-aware emotional features, achieving an optimal balance between computational efficiency and model expressiveness. Besides, we design a Complex Metric-Distance Triplet (CMDT) loss to enable the model to capture representative emotional clues for SER. Extensive experiments on the IEMOCAP and MELD datasets show that TF-Mamba surpasses existing methods in terms of model size and latency, providing a more practical solution for future SER applications. |
| title | Temporal-Frequency State Space Duality: An Efficient Paradigm for Speech Emotion Recognition |
| topic | Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2412.16904 |