Spark-TTS: An Efficient LLM-Based Text-to-Speech Model with Single-Stream Decoupled Speech Tokens
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910855707230208 |
|---|---|
| author | Wang, Xinsheng Jiang, Mingqi Ma, Ziyang Zhang, Ziyu Liu, Songxiang Li, Linqin Liang, Zheng Zheng, Qixi Wang, Rui Feng, Xiaoqin Bian, Weizhen Ye, Zhen Cheng, Sitong Yuan, Ruibin Zhao, Zhixian Zhu, Xinfa Pan, Jiahao Xue, Liumeng Zhu, Pengcheng Chen, Yunlin Li, Zhifei Chen, Xie Xie, Lei Guo, Yike Xue, Wei |
| author_facet | Wang, Xinsheng Jiang, Mingqi Ma, Ziyang Zhang, Ziyu Liu, Songxiang Li, Linqin Liang, Zheng Zheng, Qixi Wang, Rui Feng, Xiaoqin Bian, Weizhen Ye, Zhen Cheng, Sitong Yuan, Ruibin Zhao, Zhixian Zhu, Xinfa Pan, Jiahao Xue, Liumeng Zhu, Pengcheng Chen, Yunlin Li, Zhifei Chen, Xie Xie, Lei Guo, Yike Xue, Wei |
| contents | Recent advancements in large language models (LLMs) have driven significant progress in zero-shot text-to-speech (TTS) synthesis. However, existing foundation models rely on multi-stage processing or complex architectures for predicting multiple codebooks, limiting efficiency and integration flexibility. To overcome these challenges, we introduce Spark-TTS, a novel system powered by BiCodec, a single-stream speech codec that decomposes speech into two complementary token types: low-bitrate semantic tokens for linguistic content and fixed-length global tokens for speaker attributes. This disentangled representation, combined with the Qwen2.5 LLM and a chain-of-thought (CoT) generation approach, enables both coarse-grained control (e.g., gender, speaking style) and fine-grained adjustments (e.g., precise pitch values, speaking rate). To facilitate research in controllable TTS, we introduce VoxBox, a meticulously curated 100,000-hour dataset with comprehensive attribute annotations. Extensive experiments demonstrate that Spark-TTS not only achieves state-of-the-art zero-shot voice cloning but also generates highly customizable voices that surpass the limitations of reference-based synthesis. Source code, pre-trained models, and audio samples are available at https://github.com/SparkAudio/Spark-TTS. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_01710 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Spark-TTS: An Efficient LLM-Based Text-to-Speech Model with Single-Stream Decoupled Speech Tokens Wang, Xinsheng Jiang, Mingqi Ma, Ziyang Zhang, Ziyu Liu, Songxiang Li, Linqin Liang, Zheng Zheng, Qixi Wang, Rui Feng, Xiaoqin Bian, Weizhen Ye, Zhen Cheng, Sitong Yuan, Ruibin Zhao, Zhixian Zhu, Xinfa Pan, Jiahao Xue, Liumeng Zhu, Pengcheng Chen, Yunlin Li, Zhifei Chen, Xie Xie, Lei Guo, Yike Xue, Wei Sound Artificial Intelligence Audio and Speech Processing Recent advancements in large language models (LLMs) have driven significant progress in zero-shot text-to-speech (TTS) synthesis. However, existing foundation models rely on multi-stage processing or complex architectures for predicting multiple codebooks, limiting efficiency and integration flexibility. To overcome these challenges, we introduce Spark-TTS, a novel system powered by BiCodec, a single-stream speech codec that decomposes speech into two complementary token types: low-bitrate semantic tokens for linguistic content and fixed-length global tokens for speaker attributes. This disentangled representation, combined with the Qwen2.5 LLM and a chain-of-thought (CoT) generation approach, enables both coarse-grained control (e.g., gender, speaking style) and fine-grained adjustments (e.g., precise pitch values, speaking rate). To facilitate research in controllable TTS, we introduce VoxBox, a meticulously curated 100,000-hour dataset with comprehensive attribute annotations. Extensive experiments demonstrate that Spark-TTS not only achieves state-of-the-art zero-shot voice cloning but also generates highly customizable voices that surpass the limitations of reference-based synthesis. Source code, pre-trained models, and audio samples are available at https://github.com/SparkAudio/Spark-TTS. |
| title | Spark-TTS: An Efficient LLM-Based Text-to-Speech Model with Single-Stream Decoupled Speech Tokens |
| topic | Sound Artificial Intelligence Audio and Speech Processing |
| url | https://arxiv.org/abs/2503.01710 |