CoMoSVC: Consistency Model-based Singing Voice Conversion
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910286578974720 |
|---|---|
| author | Lu, Yiwen Ye, Zhen Xue, Wei Tan, Xu Liu, Qifeng Guo, Yike |
| author_facet | Lu, Yiwen Ye, Zhen Xue, Wei Tan, Xu Liu, Qifeng Guo, Yike |
| contents | The diffusion-based Singing Voice Conversion (SVC) methods have achieved remarkable performances, producing natural audios with high similarity to the target timbre. However, the iterative sampling process results in slow inference speed, and acceleration thus becomes crucial. In this paper, we propose CoMoSVC, a consistency model-based SVC method, which aims to achieve both high-quality generation and high-speed sampling. A diffusion-based teacher model is first specially designed for SVC, and a student model is further distilled under self-consistency properties to achieve one-step sampling. Experiments on a single NVIDIA GTX4090 GPU reveal that although CoMoSVC has a significantly faster inference speed than the state-of-the-art (SOTA) diffusion-based SVC system, it still achieves comparable or superior conversion performance based on both subjective and objective metrics. Audio samples and codes are available at https://comosvc.github.io/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_01792 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | CoMoSVC: Consistency Model-based Singing Voice Conversion Lu, Yiwen Ye, Zhen Xue, Wei Tan, Xu Liu, Qifeng Guo, Yike Audio and Speech Processing Artificial Intelligence Machine Learning Sound The diffusion-based Singing Voice Conversion (SVC) methods have achieved remarkable performances, producing natural audios with high similarity to the target timbre. However, the iterative sampling process results in slow inference speed, and acceleration thus becomes crucial. In this paper, we propose CoMoSVC, a consistency model-based SVC method, which aims to achieve both high-quality generation and high-speed sampling. A diffusion-based teacher model is first specially designed for SVC, and a student model is further distilled under self-consistency properties to achieve one-step sampling. Experiments on a single NVIDIA GTX4090 GPU reveal that although CoMoSVC has a significantly faster inference speed than the state-of-the-art (SOTA) diffusion-based SVC system, it still achieves comparable or superior conversion performance based on both subjective and objective metrics. Audio samples and codes are available at https://comosvc.github.io/. |
| title | CoMoSVC: Consistency Model-based Singing Voice Conversion |
| topic | Audio and Speech Processing Artificial Intelligence Machine Learning Sound |
| url | https://arxiv.org/abs/2401.01792 |