Expressive paragraph text-to-speech synthesis with multi-step variational autoencoder
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2023
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912044264980480 |
|---|---|
| author | Li, Xuyuan Shang, Zengqiang Shi, Peiyang Hua, Hua Li, Ta Zhang, Pengyuan |
| author_facet | Li, Xuyuan Shang, Zengqiang Shi, Peiyang Hua, Hua Li, Ta Zhang, Pengyuan |
| contents | Neural networks have been able to generate high-quality single-sentence speech. However, it remains a challenge concerning audio-book speech synthesis due to the intra-paragraph correlation of semantic and acoustic features as well as variable styles. In this paper, we propose a highly expressive paragraph speech synthesis system with a multi-step variational autoencoder, called EP-MSTTS. EP-MSTTS is the first VITS-based paragraph speech synthesis model and models the variable style of paragraph speech at five levels: frame, phoneme, word, sentence, and paragraph. We also propose a series of improvements to enhance the performance of this hierarchical model. In addition, we directly train EP-MSTTS on speech sliced by paragraph rather than sentence. Experiment results on the single-speaker French audiobook corpus released at Blizzard Challenge 2023 show EP-MSTTS obtains better performance than baseline models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2308_13365 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Expressive paragraph text-to-speech synthesis with multi-step variational autoencoder Li, Xuyuan Shang, Zengqiang Shi, Peiyang Hua, Hua Li, Ta Zhang, Pengyuan Sound Audio and Speech Processing Neural networks have been able to generate high-quality single-sentence speech. However, it remains a challenge concerning audio-book speech synthesis due to the intra-paragraph correlation of semantic and acoustic features as well as variable styles. In this paper, we propose a highly expressive paragraph speech synthesis system with a multi-step variational autoencoder, called EP-MSTTS. EP-MSTTS is the first VITS-based paragraph speech synthesis model and models the variable style of paragraph speech at five levels: frame, phoneme, word, sentence, and paragraph. We also propose a series of improvements to enhance the performance of this hierarchical model. In addition, we directly train EP-MSTTS on speech sliced by paragraph rather than sentence. Experiment results on the single-speaker French audiobook corpus released at Blizzard Challenge 2023 show EP-MSTTS obtains better performance than baseline models. |
| title | Expressive paragraph text-to-speech synthesis with multi-step variational autoencoder |
| topic | Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2308.13365 |