Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2410.21951 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916605476208640 |
|---|---|
| author | Li, Bohan Wang, Hankun Zhang, Situo Guo, Yiwei Yu, Kai |
| author_facet | Li, Bohan Wang, Hankun Zhang, Situo Guo, Yiwei Yu, Kai |
| contents | The auto-regressive architecture, like GPTs, is widely used in modern Text-to-Speech (TTS) systems. However, it incurs substantial inference time, particularly due to the challenges in the next-token prediction posed by lengthy sequences of speech tokens. In this work, we introduce VADUSA, one of the first approaches to accelerate auto-regressive TTS through speculative decoding. Our results show that VADUSA not only significantly improves inference speed but also enhances performance by incorporating draft heads to predict future speech content auto-regressively. Furthermore, the inclusion of a tolerance mechanism during sampling accelerates inference without compromising quality. Our approach demonstrates strong generalization across large datasets and various types of speech tokens. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_21951 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Fast and High-Quality Auto-Regressive Speech Synthesis via Speculative Decoding Li, Bohan Wang, Hankun Zhang, Situo Guo, Yiwei Yu, Kai Audio and Speech Processing Artificial Intelligence Sound 68T07 The auto-regressive architecture, like GPTs, is widely used in modern Text-to-Speech (TTS) systems. However, it incurs substantial inference time, particularly due to the challenges in the next-token prediction posed by lengthy sequences of speech tokens. In this work, we introduce VADUSA, one of the first approaches to accelerate auto-regressive TTS through speculative decoding. Our results show that VADUSA not only significantly improves inference speed but also enhances performance by incorporating draft heads to predict future speech content auto-regressively. Furthermore, the inclusion of a tolerance mechanism during sampling accelerates inference without compromising quality. Our approach demonstrates strong generalization across large datasets and various types of speech tokens. |
| title | Fast and High-Quality Auto-Regressive Speech Synthesis via Speculative Decoding |
| topic | Audio and Speech Processing Artificial Intelligence Sound 68T07 |
| url | https://arxiv.org/abs/2410.21951 |