SpeechParaling-Bench: A Comprehensive Benchmark for Paralinguistic-Aware Speech Generation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917429966274560 |
|---|---|
| author | Liu, Ruohan Yin, Shukang Wang, Tao Zhang, Dong Zhuang, Weiji Ren, Shuhuai He, Ran Shan, Caifeng Fu, Chaoyou |
| author_facet | Liu, Ruohan Yin, Shukang Wang, Tao Zhang, Dong Zhuang, Weiji Ren, Shuhuai He, Ran Shan, Caifeng Fu, Chaoyou |
| contents | Paralinguistic cues are essential for natural human-computer interaction, yet their evaluation in Large Audio-Language Models (LALMs) remains limited by coarse feature coverage and the inherent subjectivity of assessment. To address these challenges, we introduce SpeechParaling-Bench, a comprehensive benchmark for paralinguistic-aware speech generation. It expands existing coverage from fewer than 50 to over 100 fine-grained features, supported by more than 1,000 English-Chinese parallel speech queries, and is organized into three progressively challenging tasks: fine-grained control, intra-utterance variation, and context-aware adaptation. To enable reliable evaluation, we further develop a pairwise comparison pipeline, in which candidate responses are evaluated against a fixed baseline by an LALM-based judge. By framing evaluation as relative preference rather than absolute scoring, this approach mitigates subjectivity and yields more stable and scalable assessments without costly human annotation. Extensive experiments reveal substantial limitations in current LALMs. Even leading proprietary models struggle with comprehensive static control and dynamic modulation of paralinguistic features, while failure to correctly interpret paralinguistic cues accounts for 43.3% of errors in situational dialogue. These findings underscore the need for more robust paralinguistic modeling toward human-aligned voice assistants. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_20842 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | SpeechParaling-Bench: A Comprehensive Benchmark for Paralinguistic-Aware Speech Generation Liu, Ruohan Yin, Shukang Wang, Tao Zhang, Dong Zhuang, Weiji Ren, Shuhuai He, Ran Shan, Caifeng Fu, Chaoyou Computation and Language Artificial Intelligence Sound Paralinguistic cues are essential for natural human-computer interaction, yet their evaluation in Large Audio-Language Models (LALMs) remains limited by coarse feature coverage and the inherent subjectivity of assessment. To address these challenges, we introduce SpeechParaling-Bench, a comprehensive benchmark for paralinguistic-aware speech generation. It expands existing coverage from fewer than 50 to over 100 fine-grained features, supported by more than 1,000 English-Chinese parallel speech queries, and is organized into three progressively challenging tasks: fine-grained control, intra-utterance variation, and context-aware adaptation. To enable reliable evaluation, we further develop a pairwise comparison pipeline, in which candidate responses are evaluated against a fixed baseline by an LALM-based judge. By framing evaluation as relative preference rather than absolute scoring, this approach mitigates subjectivity and yields more stable and scalable assessments without costly human annotation. Extensive experiments reveal substantial limitations in current LALMs. Even leading proprietary models struggle with comprehensive static control and dynamic modulation of paralinguistic features, while failure to correctly interpret paralinguistic cues accounts for 43.3% of errors in situational dialogue. These findings underscore the need for more robust paralinguistic modeling toward human-aligned voice assistants. |
| title | SpeechParaling-Bench: A Comprehensive Benchmark for Paralinguistic-Aware Speech Generation |
| topic | Computation and Language Artificial Intelligence Sound |
| url | https://arxiv.org/abs/2604.20842 |