SNIPER Training: Single-Shot Sparse Training for Text-to-Speech
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2022
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911897274548224 |
|---|---|
| author | Lam, Perry Zhang, Huayun Chen, Nancy F. Sisman, Berrak Herremans, Dorien |
| author_facet | Lam, Perry Zhang, Huayun Chen, Nancy F. Sisman, Berrak Herremans, Dorien |
| contents | Text-to-speech (TTS) models have achieved remarkable naturalness in recent years, yet like most deep neural models, they have more parameters than necessary. Sparse TTS models can improve on dense models via pruning and extra retraining, or converge faster than dense models with some performance loss. Thus, we propose training TTS models using decaying sparsity, i.e. a high initial sparsity to accelerate training first, followed by a progressive rate reduction to obtain better eventual performance. This decremental approach differs from current methods of incrementing sparsity to a desired target, which costs significantly more time than dense training. We call our method SNIPER training: Single-shot Initialization Pruning Evolving-Rate training. Our experiments on FastSpeech2 show that we were able to obtain better losses in the first few training epochs with SNIPER, and that the final SNIPER-trained models outperformed constant-sparsity models and edged out dense models, with negligible difference in training time. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2211_07283 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | SNIPER Training: Single-Shot Sparse Training for Text-to-Speech Lam, Perry Zhang, Huayun Chen, Nancy F. Sisman, Berrak Herremans, Dorien Audio and Speech Processing Sound Text-to-speech (TTS) models have achieved remarkable naturalness in recent years, yet like most deep neural models, they have more parameters than necessary. Sparse TTS models can improve on dense models via pruning and extra retraining, or converge faster than dense models with some performance loss. Thus, we propose training TTS models using decaying sparsity, i.e. a high initial sparsity to accelerate training first, followed by a progressive rate reduction to obtain better eventual performance. This decremental approach differs from current methods of incrementing sparsity to a desired target, which costs significantly more time than dense training. We call our method SNIPER training: Single-shot Initialization Pruning Evolving-Rate training. Our experiments on FastSpeech2 show that we were able to obtain better losses in the first few training epochs with SNIPER, and that the final SNIPER-trained models outperformed constant-sparsity models and edged out dense models, with negligible difference in training time. |
| title | SNIPER Training: Single-Shot Sparse Training for Text-to-Speech |
| topic | Audio and Speech Processing Sound |
| url | https://arxiv.org/abs/2211.07283 |