Adapting Speech Language Model to Singing Voice Synthesis
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911323257831424 |
|---|---|
| author | Zhao, Yiwen Shi, Jiatong Tian, Jinchuan Tang, Yuxun Hai, Jiarui Han, Jionghao Watanabe, Shinji |
| author_facet | Zhao, Yiwen Shi, Jiatong Tian, Jinchuan Tang, Yuxun Hai, Jiarui Han, Jionghao Watanabe, Shinji |
| contents | Speech Language Models (SLMs) have recently emerged as a unified paradigm for addressing a wide range of speech-related tasks, including text-to-speech (TTS), speech enhancement (SE), and automatic speech recognition (ASR). However, the generalization capability of large-scale pre-trained SLMs remains underexplored. In this work, we adapt a 1.7B parameter TTS pretrained SLM for singing voice synthesis (SVS), using only a 135-hour synthetic singing corpus, ACE-Opencpop. Building upon the ESPNet-SpeechLM, our recipe involves the following procedure: (1) tokenization of music score conditions and singing waveforms, (2) multi-stream language model token prediction, (3) conditional flow matching-based mel-spectrogram generation. (4) a mel-to-wave vocoder. Experimental results demonstrate that our adapted SLM generalizes well to SVS and achieves performance comparable to leading discrete token-based SVS models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_14657 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Adapting Speech Language Model to Singing Voice Synthesis Zhao, Yiwen Shi, Jiatong Tian, Jinchuan Tang, Yuxun Hai, Jiarui Han, Jionghao Watanabe, Shinji Sound Speech Language Models (SLMs) have recently emerged as a unified paradigm for addressing a wide range of speech-related tasks, including text-to-speech (TTS), speech enhancement (SE), and automatic speech recognition (ASR). However, the generalization capability of large-scale pre-trained SLMs remains underexplored. In this work, we adapt a 1.7B parameter TTS pretrained SLM for singing voice synthesis (SVS), using only a 135-hour synthetic singing corpus, ACE-Opencpop. Building upon the ESPNet-SpeechLM, our recipe involves the following procedure: (1) tokenization of music score conditions and singing waveforms, (2) multi-stream language model token prediction, (3) conditional flow matching-based mel-spectrogram generation. (4) a mel-to-wave vocoder. Experimental results demonstrate that our adapted SLM generalizes well to SVS and achieves performance comparable to leading discrete token-based SVS models. |
| title | Adapting Speech Language Model to Singing Voice Synthesis |
| topic | Sound |
| url | https://arxiv.org/abs/2512.14657 |