The Codec Language Model-based Zero-Shot Spontaneous Style TTS System for CoVoC Challenge 2024
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916596712210432 |
|---|---|
| author | Zhou, Shuoyi Zhou, Yixuan Li, Weiqin Chen, Jun Ye, Runchuan Wu, Weihao Lin, Zijian Lei, Shun Wu, Zhiyong |
| author_facet | Zhou, Shuoyi Zhou, Yixuan Li, Weiqin Chen, Jun Ye, Runchuan Wu, Weihao Lin, Zijian Lei, Shun Wu, Zhiyong |
| contents | This paper describes the zero-shot spontaneous style TTS system for the ISCSLP 2024 Conversational Voice Clone Challenge (CoVoC). We propose a LLaMA-based codec language model with a delay pattern to achieve spontaneous style voice cloning. To improve speech intelligibility, we introduce the Classifier-Free Guidance (CFG) strategy in the language model to strengthen conditional guidance on token prediction. To generate high-quality utterances, we adopt effective data preprocessing operations and fine-tune our model with selected high-quality spontaneous speech data. The official evaluations in the CoVoC constrained track show that our system achieves the best speech naturalness MOS of 3.80 and obtains considerable speech quality and speaker similarity results. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_01100 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | The Codec Language Model-based Zero-Shot Spontaneous Style TTS System for CoVoC Challenge 2024 Zhou, Shuoyi Zhou, Yixuan Li, Weiqin Chen, Jun Ye, Runchuan Wu, Weihao Lin, Zijian Lei, Shun Wu, Zhiyong Sound Audio and Speech Processing This paper describes the zero-shot spontaneous style TTS system for the ISCSLP 2024 Conversational Voice Clone Challenge (CoVoC). We propose a LLaMA-based codec language model with a delay pattern to achieve spontaneous style voice cloning. To improve speech intelligibility, we introduce the Classifier-Free Guidance (CFG) strategy in the language model to strengthen conditional guidance on token prediction. To generate high-quality utterances, we adopt effective data preprocessing operations and fine-tune our model with selected high-quality spontaneous speech data. The official evaluations in the CoVoC constrained track show that our system achieves the best speech naturalness MOS of 3.80 and obtains considerable speech quality and speaker similarity results. |
| title | The Codec Language Model-based Zero-Shot Spontaneous Style TTS System for CoVoC Challenge 2024 |
| topic | Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2412.01100 |