Should you use a probabilistic duration model in TTS? Probably! Especially for spontaneous speech
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916280121950208 |
|---|---|
| author | Mehta, Shivam Lameris, Harm Punmiya, Rajiv Beskow, Jonas Székely, Éva Henter, Gustav Eje |
| author_facet | Mehta, Shivam Lameris, Harm Punmiya, Rajiv Beskow, Jonas Székely, Éva Henter, Gustav Eje |
| contents | Converting input symbols to output audio in TTS requires modelling the durations of speech sounds. Leading non-autoregressive (NAR) TTS models treat duration modelling as a regression problem. The same utterance is then spoken with identical timings every time, unlike when a human speaks. Probabilistic models of duration have been proposed, but there is mixed evidence of their benefits. However, prior studies generally only consider speech read aloud, and ignore spontaneous speech, despite the latter being both a more common and a more variable mode of speaking. We compare the effect of conventional deterministic duration modelling to durations sampled from a powerful probabilistic model based on conditional flow matching (OT-CFM), in three different NAR TTS approaches: regression-based, deep generative, and end-to-end. Across four different corpora, stochastic duration modelling improves probabilistic NAR TTS approaches, especially for spontaneous speech. Please see https://shivammehta25.github.io/prob_dur/ for audio and resources. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_05401 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Should you use a probabilistic duration model in TTS? Probably! Especially for spontaneous speech Mehta, Shivam Lameris, Harm Punmiya, Rajiv Beskow, Jonas Székely, Éva Henter, Gustav Eje Audio and Speech Processing Human-Computer Interaction Sound 68T07 I.2.7; I.2.6; H.5.5 Converting input symbols to output audio in TTS requires modelling the durations of speech sounds. Leading non-autoregressive (NAR) TTS models treat duration modelling as a regression problem. The same utterance is then spoken with identical timings every time, unlike when a human speaks. Probabilistic models of duration have been proposed, but there is mixed evidence of their benefits. However, prior studies generally only consider speech read aloud, and ignore spontaneous speech, despite the latter being both a more common and a more variable mode of speaking. We compare the effect of conventional deterministic duration modelling to durations sampled from a powerful probabilistic model based on conditional flow matching (OT-CFM), in three different NAR TTS approaches: regression-based, deep generative, and end-to-end. Across four different corpora, stochastic duration modelling improves probabilistic NAR TTS approaches, especially for spontaneous speech. Please see https://shivammehta25.github.io/prob_dur/ for audio and resources. |
| title | Should you use a probabilistic duration model in TTS? Probably! Especially for spontaneous speech |
| topic | Audio and Speech Processing Human-Computer Interaction Sound 68T07 I.2.7; I.2.6; H.5.5 |
| url | https://arxiv.org/abs/2406.05401 |