Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2505.19669 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908389243617280 |
|---|---|
| author | Sun, Haiyang Hu, Shujie Liu, Shujie Meng, Lingwei Wang, Hui Han, Bing Yang, Yifan Liu, Yanqing Zhao, Sheng Lu, Yan Qian, Yanmin |
| author_facet | Sun, Haiyang Hu, Shujie Liu, Shujie Meng, Lingwei Wang, Hui Han, Bing Yang, Yifan Liu, Yanqing Zhao, Sheng Lu, Yan Qian, Yanmin |
| contents | Zero-shot streaming text-to-speech is an important research topic in human-computer interaction. Existing methods primarily use a lookahead mechanism, relying on future text to achieve natural streaming speech synthesis, which introduces high processing latency. To address this issue, we propose SMLLE, a streaming framework for generating high-quality speech frame-by-frame. SMLLE employs a Transducer to convert text into semantic tokens in real time while simultaneously obtaining duration alignment information. The combined outputs are then fed into a fully autoregressive (AR) streaming model to reconstruct mel-spectrograms. To further stabilize the generation process, we design a Delete < Bos > Mechanism that allows the AR model to access future text introducing as minimal delay as possible. Experimental results suggest that the SMLLE outperforms current streaming TTS methods and achieves comparable performance over sentence-level TTS systems. Samples are available on shy-98.github.io/SMLLE_demo_page/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_19669 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Zero-Shot Streaming Text to Speech Synthesis with Transducer and Auto-Regressive Modeling Sun, Haiyang Hu, Shujie Liu, Shujie Meng, Lingwei Wang, Hui Han, Bing Yang, Yifan Liu, Yanqing Zhao, Sheng Lu, Yan Qian, Yanmin Machine Learning Zero-shot streaming text-to-speech is an important research topic in human-computer interaction. Existing methods primarily use a lookahead mechanism, relying on future text to achieve natural streaming speech synthesis, which introduces high processing latency. To address this issue, we propose SMLLE, a streaming framework for generating high-quality speech frame-by-frame. SMLLE employs a Transducer to convert text into semantic tokens in real time while simultaneously obtaining duration alignment information. The combined outputs are then fed into a fully autoregressive (AR) streaming model to reconstruct mel-spectrograms. To further stabilize the generation process, we design a Delete < Bos > Mechanism that allows the AR model to access future text introducing as minimal delay as possible. Experimental results suggest that the SMLLE outperforms current streaming TTS methods and achieves comparable performance over sentence-level TTS systems. Samples are available on shy-98.github.io/SMLLE_demo_page/. |
| title | Zero-Shot Streaming Text to Speech Synthesis with Transducer and Auto-Regressive Modeling |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2505.19669 |