TCSinger 2: Customizable Multilingual Zero-shot Singing Voice Synthesis
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866911112977448960 |
|---|---|
| author | Zhang, Yu Guo, Wenxiang Pan, Changhao Yao, Dongyu Zhu, Zhiyuan Jiang, Ziyue Wang, Yuhan Jin, Tao Zhao, Zhou |
| author_facet | Zhang, Yu Guo, Wenxiang Pan, Changhao Yao, Dongyu Zhu, Zhiyuan Jiang, Ziyue Wang, Yuhan Jin, Tao Zhao, Zhou |
| contents | Customizable multilingual zero-shot singing voice synthesis (SVS) has various potential applications in music composition and short video dubbing. However, existing SVS models overly depend on phoneme and note boundary annotations, limiting their robustness in zero-shot scenarios and producing poor transitions between phonemes and notes. Moreover, they also lack effective multi-level style control via diverse prompts. To overcome these challenges, we introduce TCSinger 2, a multi-task multilingual zero-shot SVS model with style transfer and style control based on various prompts. TCSinger 2 mainly includes three key modules: 1) Blurred Boundary Content (BBC) Encoder, predicts duration, extends content embedding, and applies masking to the boundaries to enable smooth transitions. 2) Custom Audio Encoder, uses contrastive learning to extract aligned representations from singing, speech, and textual prompts. 3) Flow-based Custom Transformer, leverages Cus-MOE, with F0 supervision, enhancing both the synthesis quality and style modeling of the generated singing voice. Experimental results show that TCSinger 2 outperforms baseline models in both subjective and objective metrics across multiple related tasks. Singing voice samples are available at https://aaronz345.github.io/TCSinger2Demo/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_14910 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | TCSinger 2: Customizable Multilingual Zero-shot Singing Voice Synthesis Zhang, Yu Guo, Wenxiang Pan, Changhao Yao, Dongyu Zhu, Zhiyuan Jiang, Ziyue Wang, Yuhan Jin, Tao Zhao, Zhou Audio and Speech Processing Computation and Language Sound Customizable multilingual zero-shot singing voice synthesis (SVS) has various potential applications in music composition and short video dubbing. However, existing SVS models overly depend on phoneme and note boundary annotations, limiting their robustness in zero-shot scenarios and producing poor transitions between phonemes and notes. Moreover, they also lack effective multi-level style control via diverse prompts. To overcome these challenges, we introduce TCSinger 2, a multi-task multilingual zero-shot SVS model with style transfer and style control based on various prompts. TCSinger 2 mainly includes three key modules: 1) Blurred Boundary Content (BBC) Encoder, predicts duration, extends content embedding, and applies masking to the boundaries to enable smooth transitions. 2) Custom Audio Encoder, uses contrastive learning to extract aligned representations from singing, speech, and textual prompts. 3) Flow-based Custom Transformer, leverages Cus-MOE, with F0 supervision, enhancing both the synthesis quality and style modeling of the generated singing voice. Experimental results show that TCSinger 2 outperforms baseline models in both subjective and objective metrics across multiple related tasks. Singing voice samples are available at https://aaronz345.github.io/TCSinger2Demo/. |
| title | TCSinger 2: Customizable Multilingual Zero-shot Singing Voice Synthesis |
| topic | Audio and Speech Processing Computation and Language Sound |
| url | https://arxiv.org/abs/2505.14910 |