On the Emotion Understanding of Synthesized Speech
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915869870784512 |
|---|---|
| author | Ge, Yuan Zhao, Haishu Hao, Aokai Zhang, Junxiang Li, Bei Liu, Xiaoqian Wang, Chenglong Wang, Jianjin Zhou, Bingsen Liu, Bingyu Zhu, Jingbo Yu, Zhengtao Xiao, Tong |
| author_facet | Ge, Yuan Zhao, Haishu Hao, Aokai Zhang, Junxiang Li, Bei Liu, Xiaoqian Wang, Chenglong Wang, Jianjin Zhou, Bingsen Liu, Bingyu Zhu, Jingbo Yu, Zhengtao Xiao, Tong |
| contents | Emotion is a core paralinguistic feature in voice interaction. It is widely believed that emotion understanding models learn fundamental representations that transfer to synthesized speech, making emotion understanding results a plausible reward or evaluation metric for assessing emotional expressiveness in speech synthesis. In this work, we critically examine this assumption by systematically evaluating Speech Emotion Recognition (SER) on synthesized speech across datasets, discriminative and generative SER models, and diverse synthesis models. We find that current SER models can not generalize to synthesized speech, largely because speech token prediction during synthesis induces a representation mismatch between synthesized and human speech. Moreover, generative Speech Language Models (SLMs) tend to infer emotion from textual semantics while ignoring paralinguistic cues. Overall, our findings suggest that existing SER models often exploit non-robust shortcuts rather than capturing fundamental features, and paralinguistic understanding in SLMs remains challenging. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_16483 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | On the Emotion Understanding of Synthesized Speech Ge, Yuan Zhao, Haishu Hao, Aokai Zhang, Junxiang Li, Bei Liu, Xiaoqian Wang, Chenglong Wang, Jianjin Zhou, Bingsen Liu, Bingyu Zhu, Jingbo Yu, Zhengtao Xiao, Tong Computation and Language Emotion is a core paralinguistic feature in voice interaction. It is widely believed that emotion understanding models learn fundamental representations that transfer to synthesized speech, making emotion understanding results a plausible reward or evaluation metric for assessing emotional expressiveness in speech synthesis. In this work, we critically examine this assumption by systematically evaluating Speech Emotion Recognition (SER) on synthesized speech across datasets, discriminative and generative SER models, and diverse synthesis models. We find that current SER models can not generalize to synthesized speech, largely because speech token prediction during synthesis induces a representation mismatch between synthesized and human speech. Moreover, generative Speech Language Models (SLMs) tend to infer emotion from textual semantics while ignoring paralinguistic cues. Overall, our findings suggest that existing SER models often exploit non-robust shortcuts rather than capturing fundamental features, and paralinguistic understanding in SLMs remains challenging. |
| title | On the Emotion Understanding of Synthesized Speech |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2603.16483 |