Saved in:
| Main Authors: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2509.26514 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912618500849664 |
|---|---|
| author | Wang, Yue Ma, Ruotian Chen, Xingyu Shi, Zhengliang Chen, Wanshun Liu, Huang Yao, Jiadi Yang, Qu Jiang, Qingxuan Ye, Fanghua Li, Juntao Zhang, Min Tu, Zhaopeng Li, Xiaolong Linus |
| author_facet | Wang, Yue Ma, Ruotian Chen, Xingyu Shi, Zhengliang Chen, Wanshun Liu, Huang Yao, Jiadi Yang, Qu Jiang, Qingxuan Ye, Fanghua Li, Juntao Zhang, Min Tu, Zhaopeng Li, Xiaolong Linus |
| contents | The rise of Large Language Models (LLMs) is reshaping multimodel models, with speech synthesis being a prominent application. However, existing approaches often underutilize the linguistic intelligence of these models, typically failing to leverage their powerful instruction-following capabilities. This limitation hinders the model's ability to follow text instructions for controllable Text-to-Speech~(TTS). To address this, we propose a new paradigm inspired by ``operationalism'' that decouples instruction understanding from speech generation. We introduce BatonVoice, a framework where an LLM acts as a ``conductor'', understanding user instructions and generating a textual ``plan'' -- explicit vocal features (e.g., pitch, energy). A separate TTS model, the ``orchestra'', then generates the speech from these features. To realize this component, we develop BatonTTS, a TTS model trained specifically for this task. Our experiments demonstrate that BatonVoice achieves strong performance in controllable and emotional speech synthesis, outperforming strong open- and closed-source baselines. Notably, our approach enables remarkable zero-shot cross-lingual generalization, accurately applying feature control abilities to languages unseen during post-training. This demonstrates that objectifying speech into textual vocal features can more effectively unlock the linguistic intelligence of LLMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_26514 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | BatonVoice: An Operationalist Framework for Enhancing Controllable Speech Synthesis with Linguistic Intelligence from LLMs Wang, Yue Ma, Ruotian Chen, Xingyu Shi, Zhengliang Chen, Wanshun Liu, Huang Yao, Jiadi Yang, Qu Jiang, Qingxuan Ye, Fanghua Li, Juntao Zhang, Min Tu, Zhaopeng Li, Xiaolong Linus Computation and Language The rise of Large Language Models (LLMs) is reshaping multimodel models, with speech synthesis being a prominent application. However, existing approaches often underutilize the linguistic intelligence of these models, typically failing to leverage their powerful instruction-following capabilities. This limitation hinders the model's ability to follow text instructions for controllable Text-to-Speech~(TTS). To address this, we propose a new paradigm inspired by ``operationalism'' that decouples instruction understanding from speech generation. We introduce BatonVoice, a framework where an LLM acts as a ``conductor'', understanding user instructions and generating a textual ``plan'' -- explicit vocal features (e.g., pitch, energy). A separate TTS model, the ``orchestra'', then generates the speech from these features. To realize this component, we develop BatonTTS, a TTS model trained specifically for this task. Our experiments demonstrate that BatonVoice achieves strong performance in controllable and emotional speech synthesis, outperforming strong open- and closed-source baselines. Notably, our approach enables remarkable zero-shot cross-lingual generalization, accurately applying feature control abilities to languages unseen during post-training. This demonstrates that objectifying speech into textual vocal features can more effectively unlock the linguistic intelligence of LLMs. |
| title | BatonVoice: An Operationalist Framework for Enhancing Controllable Speech Synthesis with Linguistic Intelligence from LLMs |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2509.26514 |