Bailing-TTS: Chinese Dialectal Speech Synthesis Towards Human-like Spontaneous Representation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910550423764992 |
|---|---|
| author | Di, Xinhan Chen, Zihao Liang, Yunming Zheng, Junjie Wang, Yihua Ding, Chaofan |
| author_facet | Di, Xinhan Chen, Zihao Liang, Yunming Zheng, Junjie Wang, Yihua Ding, Chaofan |
| contents | Large-scale text-to-speech (TTS) models have made significant progress recently.However, they still fall short in the generation of Chinese dialectal speech. Toaddress this, we propose Bailing-TTS, a family of large-scale TTS models capable of generating high-quality Chinese dialectal speech. Bailing-TTS serves as a foundation model for Chinese dialectal speech generation. First, continual semi-supervised learning is proposed to facilitate the alignment of text tokens and speech tokens. Second, the Chinese dialectal representation learning is developed using a specific transformer architecture and multi-stage training processes. With the proposed design of novel network architecture and corresponding strategy, Bailing-TTS is able to generate Chinese dialectal speech from text effectively and efficiently. Experiments demonstrate that Bailing-TTS generates Chinese dialectal speech towards human-like spontaneous representation. Readers are encouraged to listen to demos at \url{https://c9412600.github.io/bltts_tech_report/index.html}. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_00284 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Bailing-TTS: Chinese Dialectal Speech Synthesis Towards Human-like Spontaneous Representation Di, Xinhan Chen, Zihao Liang, Yunming Zheng, Junjie Wang, Yihua Ding, Chaofan Computation and Language Sound Audio and Speech Processing Large-scale text-to-speech (TTS) models have made significant progress recently.However, they still fall short in the generation of Chinese dialectal speech. Toaddress this, we propose Bailing-TTS, a family of large-scale TTS models capable of generating high-quality Chinese dialectal speech. Bailing-TTS serves as a foundation model for Chinese dialectal speech generation. First, continual semi-supervised learning is proposed to facilitate the alignment of text tokens and speech tokens. Second, the Chinese dialectal representation learning is developed using a specific transformer architecture and multi-stage training processes. With the proposed design of novel network architecture and corresponding strategy, Bailing-TTS is able to generate Chinese dialectal speech from text effectively and efficiently. Experiments demonstrate that Bailing-TTS generates Chinese dialectal speech towards human-like spontaneous representation. Readers are encouraged to listen to demos at \url{https://c9412600.github.io/bltts_tech_report/index.html}. |
| title | Bailing-TTS: Chinese Dialectal Speech Synthesis Towards Human-like Spontaneous Representation |
| topic | Computation and Language Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2408.00284 |