Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2507.00227 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916818037243904 |
|---|---|
| author | Mayer, Paul Lux, Florian Pérez-González-de-Martos, Alejandro Elizarova, Angelina Vanderlyn, Lindsey Väth, Dirk Vu, Ngoc Thang |
| author_facet | Mayer, Paul Lux, Florian Pérez-González-de-Martos, Alejandro Elizarova, Angelina Vanderlyn, Lindsey Väth, Dirk Vu, Ngoc Thang |
| contents | While generative methods have progressed rapidly in recent years, generating expressive prosody for an utterance remains a challenging task in text-to-speech synthesis. This is particularly true for systems that model prosody explicitly through parameters such as pitch, energy, and duration, which is commonly done for the sake of interpretability and controllability. In this work, we investigate the effectiveness of stochastic methods for this task, including Normalizing Flows, Conditional Flow Matching, and Rectified Flows. We compare these methods to a traditional deterministic baseline, as well as to real human realizations. Our extensive subjective and objective evaluations demonstrate that stochastic methods produce natural prosody on par with human speakers by capturing the variability inherent in human speech. Further, they open up additional controllability options by allowing the sampling temperature to be tuned. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_00227 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Investigating Stochastic Methods for Prosody Modeling in Speech Synthesis Mayer, Paul Lux, Florian Pérez-González-de-Martos, Alejandro Elizarova, Angelina Vanderlyn, Lindsey Väth, Dirk Vu, Ngoc Thang Audio and Speech Processing Artificial Intelligence While generative methods have progressed rapidly in recent years, generating expressive prosody for an utterance remains a challenging task in text-to-speech synthesis. This is particularly true for systems that model prosody explicitly through parameters such as pitch, energy, and duration, which is commonly done for the sake of interpretability and controllability. In this work, we investigate the effectiveness of stochastic methods for this task, including Normalizing Flows, Conditional Flow Matching, and Rectified Flows. We compare these methods to a traditional deterministic baseline, as well as to real human realizations. Our extensive subjective and objective evaluations demonstrate that stochastic methods produce natural prosody on par with human speakers by capturing the variability inherent in human speech. Further, they open up additional controllability options by allowing the sampling temperature to be tuned. |
| title | Investigating Stochastic Methods for Prosody Modeling in Speech Synthesis |
| topic | Audio and Speech Processing Artificial Intelligence |
| url | https://arxiv.org/abs/2507.00227 |