SDiaReward: Modeling and Benchmarking Spoken Dialogue Rewards with Modality and Colloquialness
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915999678201856 |
|---|---|
| author | Lu, Jingyu Wang, Yuhan Zhuo, Fan Cheng, Xize Pan, Changhao Pu, Xueyi Chen, Yifu Wen, Chenyuhao Liang, Tianle Zhao, Zhou |
| author_facet | Lu, Jingyu Wang, Yuhan Zhuo, Fan Cheng, Xize Pan, Changhao Pu, Xueyi Chen, Yifu Wen, Chenyuhao Liang, Tianle Zhao, Zhou |
| contents | The rapid evolution of end-to-end spoken dialogue systems demands transcending mere textual semantics to incorporate paralinguistic nuances and the spontaneous nature of human conversation. However, current methods struggle with two critical gaps: the modality gap, involving prosody and emotion, and the colloquialness gap, distinguishing written scripts from natural speech. To address these challenges, we introduce SDiaReward, an end-to-end multi-turn reward model trained on SDiaReward-Dataset, a novel collection of episode-level preference pairs explicitly targeting these gaps. It operates directly on full multi-turn speech episodes and is optimized with pairwise preference supervision, enabling joint assessment of modality and colloquialness in a single evaluator. We further establish ESDR-Bench, a stratified benchmark for robust episode-level evaluation. Experiments demonstrate that SDiaReward achieves state-of-the-art pairwise preference accuracy, significantly outperforming general-purpose audio LLMs. Further analysis suggests that SDiaReward captures relative conversational expressiveness beyond superficial synthesis cues, improving generalization across domains and recording conditions. Code, data, and demos are available at https://github.com/MM-Speech/SDiaReward/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_14889 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | SDiaReward: Modeling and Benchmarking Spoken Dialogue Rewards with Modality and Colloquialness Lu, Jingyu Wang, Yuhan Zhuo, Fan Cheng, Xize Pan, Changhao Pu, Xueyi Chen, Yifu Wen, Chenyuhao Liang, Tianle Zhao, Zhou Audio and Speech Processing Computation and Language Machine Learning The rapid evolution of end-to-end spoken dialogue systems demands transcending mere textual semantics to incorporate paralinguistic nuances and the spontaneous nature of human conversation. However, current methods struggle with two critical gaps: the modality gap, involving prosody and emotion, and the colloquialness gap, distinguishing written scripts from natural speech. To address these challenges, we introduce SDiaReward, an end-to-end multi-turn reward model trained on SDiaReward-Dataset, a novel collection of episode-level preference pairs explicitly targeting these gaps. It operates directly on full multi-turn speech episodes and is optimized with pairwise preference supervision, enabling joint assessment of modality and colloquialness in a single evaluator. We further establish ESDR-Bench, a stratified benchmark for robust episode-level evaluation. Experiments demonstrate that SDiaReward achieves state-of-the-art pairwise preference accuracy, significantly outperforming general-purpose audio LLMs. Further analysis suggests that SDiaReward captures relative conversational expressiveness beyond superficial synthesis cues, improving generalization across domains and recording conditions. Code, data, and demos are available at https://github.com/MM-Speech/SDiaReward/. |
| title | SDiaReward: Modeling and Benchmarking Spoken Dialogue Rewards with Modality and Colloquialness |
| topic | Audio and Speech Processing Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2603.14889 |