Single-stage TTS with Masked Audio Token Modeling and Semantic Knowledge Distillation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916397280395264 |
|---|---|
| author | Gállego, Gerard I. Fejgin, Roy Yeh, Chunghsin Liu, Xiaoyu Bhattacharya, Gautam |
| author_facet | Gállego, Gerard I. Fejgin, Roy Yeh, Chunghsin Liu, Xiaoyu Bhattacharya, Gautam |
| contents | Audio token modeling has become a powerful framework for speech synthesis, with two-stage approaches employing semantic tokens remaining prevalent. In this paper, we aim to simplify this process by introducing a semantic knowledge distillation method that enables high-quality speech generation in a single stage. Our proposed model improves speech quality, intelligibility, and speaker similarity compared to a single-stage baseline. Although two-stage systems still lead in intelligibility, our model significantly narrows the gap while delivering comparable speech quality. These findings showcase the potential of single-stage models to achieve efficient, high-quality TTS with a more compact and streamlined architecture. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_11003 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Single-stage TTS with Masked Audio Token Modeling and Semantic Knowledge Distillation Gállego, Gerard I. Fejgin, Roy Yeh, Chunghsin Liu, Xiaoyu Bhattacharya, Gautam Sound Artificial Intelligence Audio and Speech Processing Signal Processing Audio token modeling has become a powerful framework for speech synthesis, with two-stage approaches employing semantic tokens remaining prevalent. In this paper, we aim to simplify this process by introducing a semantic knowledge distillation method that enables high-quality speech generation in a single stage. Our proposed model improves speech quality, intelligibility, and speaker similarity compared to a single-stage baseline. Although two-stage systems still lead in intelligibility, our model significantly narrows the gap while delivering comparable speech quality. These findings showcase the potential of single-stage models to achieve efficient, high-quality TTS with a more compact and streamlined architecture. |
| title | Single-stage TTS with Masked Audio Token Modeling and Semantic Knowledge Distillation |
| topic | Sound Artificial Intelligence Audio and Speech Processing Signal Processing |
| url | https://arxiv.org/abs/2409.11003 |