MATER: Multi-level Acoustic and Textual Emotion Representation for Interpretable Speech Emotion Recognition
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908590563917824 |
|---|---|
| author | Jon, Hyo Jin Jin, Longbin Jung, Hyuntaek Kim, Hyunseo Min, Donghun Kim, Eun Yi |
| author_facet | Jon, Hyo Jin Jin, Longbin Jung, Hyuntaek Kim, Hyunseo Min, Donghun Kim, Eun Yi |
| contents | This paper presents our contributions to the Speech Emotion Recognition in Naturalistic Conditions (SERNC) Challenge, where we address categorical emotion recognition and emotional attribute prediction. To handle the complexities of natural speech, including intra- and inter-subject variability, we propose Multi-level Acoustic-Textual Emotion Representation (MATER), a novel hierarchical framework that integrates acoustic and textual features at the word, utterance, and embedding levels. By fusing low-level lexical and acoustic cues with high-level contextualized representations, MATER effectively captures both fine-grained prosodic variations and semantic nuances. Additionally, we introduce an uncertainty-aware ensemble strategy to mitigate annotator inconsistencies, improving robustness in ambiguous emotional expressions. MATER ranks fourth in both tasks with a Macro-F1 of 41.01% and an average CCC of 0.5928, securing second place in valence prediction with an impressive CCC of 0.6941. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_19887 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MATER: Multi-level Acoustic and Textual Emotion Representation for Interpretable Speech Emotion Recognition Jon, Hyo Jin Jin, Longbin Jung, Hyuntaek Kim, Hyunseo Min, Donghun Kim, Eun Yi Audio and Speech Processing Artificial Intelligence Sound 68T10 This paper presents our contributions to the Speech Emotion Recognition in Naturalistic Conditions (SERNC) Challenge, where we address categorical emotion recognition and emotional attribute prediction. To handle the complexities of natural speech, including intra- and inter-subject variability, we propose Multi-level Acoustic-Textual Emotion Representation (MATER), a novel hierarchical framework that integrates acoustic and textual features at the word, utterance, and embedding levels. By fusing low-level lexical and acoustic cues with high-level contextualized representations, MATER effectively captures both fine-grained prosodic variations and semantic nuances. Additionally, we introduce an uncertainty-aware ensemble strategy to mitigate annotator inconsistencies, improving robustness in ambiguous emotional expressions. MATER ranks fourth in both tasks with a Macro-F1 of 41.01% and an average CCC of 0.5928, securing second place in valence prediction with an impressive CCC of 0.6941. |
| title | MATER: Multi-level Acoustic and Textual Emotion Representation for Interpretable Speech Emotion Recognition |
| topic | Audio and Speech Processing Artificial Intelligence Sound 68T10 |
| url | https://arxiv.org/abs/2506.19887 |