SpeechLLM-as-Judges: Towards General and Interpretable Speech Quality Evaluation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908968901672960 |
|---|---|
| author | Wang, Hui Zhao, Jinghua Yang, Yifan Liu, Shujie Chen, Junyang Zhang, Yanzhe Zhao, Shiwan Li, Jinyu Zhou, Jiaming Sun, Haoqin Lu, Yan Qin, Yong |
| author_facet | Wang, Hui Zhao, Jinghua Yang, Yifan Liu, Shujie Chen, Junyang Zhang, Yanzhe Zhao, Shiwan Li, Jinyu Zhou, Jiaming Sun, Haoqin Lu, Yan Qin, Yong |
| contents | Generative speech technologies are progressing rapidly, but evaluating the perceptual quality of synthetic speech remains a core challenge. Existing methods typically rely on scalar scores or binary decisions, which lack interpretability and generalization across tasks and languages. We present SpeechLLM-as-Judges, a new paradigm for enabling large language models (LLMs) to conduct structured and explanation-based speech quality evaluation. To support this direction, we introduce SpeechEval, a large-scale dataset containing 32,207 multilingual speech clips and 128,754 annotations spanning four tasks: quality assessment, pairwise comparison, improvement suggestion, and deepfake detection. Based on this resource, we develop SQ-LLM, a speech-quality-aware LLM trained with chain-of-thought reasoning and reward optimization to improve capability. Experimental results show that SQ-LLM delivers strong performance across tasks and languages, revealing the potential of this paradigm for advancing speech quality evaluation. The relevant code, models, and data are publicly available at https://github.com/NKU-HLT/SpeechLLM-as-Judges. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_14664 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | SpeechLLM-as-Judges: Towards General and Interpretable Speech Quality Evaluation Wang, Hui Zhao, Jinghua Yang, Yifan Liu, Shujie Chen, Junyang Zhang, Yanzhe Zhao, Shiwan Li, Jinyu Zhou, Jiaming Sun, Haoqin Lu, Yan Qin, Yong Sound Audio and Speech Processing Generative speech technologies are progressing rapidly, but evaluating the perceptual quality of synthetic speech remains a core challenge. Existing methods typically rely on scalar scores or binary decisions, which lack interpretability and generalization across tasks and languages. We present SpeechLLM-as-Judges, a new paradigm for enabling large language models (LLMs) to conduct structured and explanation-based speech quality evaluation. To support this direction, we introduce SpeechEval, a large-scale dataset containing 32,207 multilingual speech clips and 128,754 annotations spanning four tasks: quality assessment, pairwise comparison, improvement suggestion, and deepfake detection. Based on this resource, we develop SQ-LLM, a speech-quality-aware LLM trained with chain-of-thought reasoning and reward optimization to improve capability. Experimental results show that SQ-LLM delivers strong performance across tasks and languages, revealing the potential of this paradigm for advancing speech quality evaluation. The relevant code, models, and data are publicly available at https://github.com/NKU-HLT/SpeechLLM-as-Judges. |
| title | SpeechLLM-as-Judges: Towards General and Interpretable Speech Quality Evaluation |
| topic | Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2510.14664 |