SageLM: A Multi-aspect and Explainable Large Language Model for Speech Judgement
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918192851451904 |
|---|---|
| author | Ge, Yuan Zhang, Junxiang Liu, Xiaoqian Li, Bei Ma, Xiangnan Wang, Chenglong Ye, Kaiyang Du, Yangfan Zhang, Linfeng Huang, Yuxin Xiao, Tong Yu, Zhengtao Zhu, JingBo |
| author_facet | Ge, Yuan Zhang, Junxiang Liu, Xiaoqian Li, Bei Ma, Xiangnan Wang, Chenglong Ye, Kaiyang Du, Yangfan Zhang, Linfeng Huang, Yuxin Xiao, Tong Yu, Zhengtao Zhu, JingBo |
| contents | Speech-to-Speech (S2S) Large Language Models (LLMs) are foundational to natural human-computer interaction, enabling end-to-end spoken dialogue systems. However, evaluating these models remains a fundamental challenge. We propose \texttt{SageLM}, an end-to-end, multi-aspect, and explainable speech LLM for comprehensive S2S LLMs evaluation. First, unlike cascaded approaches that disregard acoustic features, SageLM jointly assesses both semantic and acoustic dimensions. Second, it leverages rationale-based supervision to enhance explainability and guide model learning, achieving superior alignment with evaluation outcomes compared to rule-based reinforcement learning methods. Third, we introduce \textit{SpeechFeedback}, a synthetic preference dataset, and employ a two-stage training paradigm to mitigate the scarcity of speech preference data. Trained on both semantic and acoustic dimensions, SageLM achieves an 82.79\% agreement rate with human evaluators, outperforming cascaded and SLM-based baselines by at least 7.42\% and 26.20\%, respectively. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_20916 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | SageLM: A Multi-aspect and Explainable Large Language Model for Speech Judgement Ge, Yuan Zhang, Junxiang Liu, Xiaoqian Li, Bei Ma, Xiangnan Wang, Chenglong Ye, Kaiyang Du, Yangfan Zhang, Linfeng Huang, Yuxin Xiao, Tong Yu, Zhengtao Zhu, JingBo Computation and Language Speech-to-Speech (S2S) Large Language Models (LLMs) are foundational to natural human-computer interaction, enabling end-to-end spoken dialogue systems. However, evaluating these models remains a fundamental challenge. We propose \texttt{SageLM}, an end-to-end, multi-aspect, and explainable speech LLM for comprehensive S2S LLMs evaluation. First, unlike cascaded approaches that disregard acoustic features, SageLM jointly assesses both semantic and acoustic dimensions. Second, it leverages rationale-based supervision to enhance explainability and guide model learning, achieving superior alignment with evaluation outcomes compared to rule-based reinforcement learning methods. Third, we introduce \textit{SpeechFeedback}, a synthetic preference dataset, and employ a two-stage training paradigm to mitigate the scarcity of speech preference data. Trained on both semantic and acoustic dimensions, SageLM achieves an 82.79\% agreement rate with human evaluators, outperforming cascaded and SLM-based baselines by at least 7.42\% and 26.20\%, respectively. |
| title | SageLM: A Multi-aspect and Explainable Large Language Model for Speech Judgement |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2508.20916 |