MAIN-VC: Lightweight Speech Representation Disentanglement for One-shot Voice Conversion
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917846764748800 |
|---|---|
| author | Li, Pengcheng Wang, Jianzong Zhang, Xulong Zhang, Yong Xiao, Jing Cheng, Ning |
| author_facet | Li, Pengcheng Wang, Jianzong Zhang, Xulong Zhang, Yong Xiao, Jing Cheng, Ning |
| contents | One-shot voice conversion aims to change the timbre of any source speech to match that of the unseen target speaker with only one speech sample. Existing methods face difficulties in satisfactory speech representation disentanglement and suffer from sizable networks as some of them leverage numerous complex modules for disentanglement. In this paper, we propose a model named MAIN-VC to effectively disentangle via a concise neural network. The proposed model utilizes Siamese encoders to learn clean representations, further enhanced by the designed mutual information estimator. The Siamese structure and the newly designed convolution module contribute to the lightweight of our model while ensuring performance in diverse voice conversion tasks. The experimental results show that the proposed model achieves comparable subjective scores and exhibits improvements in objective metrics compared to existing methods in a one-shot voice conversion scenario. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_00930 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | MAIN-VC: Lightweight Speech Representation Disentanglement for One-shot Voice Conversion Li, Pengcheng Wang, Jianzong Zhang, Xulong Zhang, Yong Xiao, Jing Cheng, Ning Sound Audio and Speech Processing I.2.7; H.5.5 One-shot voice conversion aims to change the timbre of any source speech to match that of the unseen target speaker with only one speech sample. Existing methods face difficulties in satisfactory speech representation disentanglement and suffer from sizable networks as some of them leverage numerous complex modules for disentanglement. In this paper, we propose a model named MAIN-VC to effectively disentangle via a concise neural network. The proposed model utilizes Siamese encoders to learn clean representations, further enhanced by the designed mutual information estimator. The Siamese structure and the newly designed convolution module contribute to the lightweight of our model while ensuring performance in diverse voice conversion tasks. The experimental results show that the proposed model achieves comparable subjective scores and exhibits improvements in objective metrics compared to existing methods in a one-shot voice conversion scenario. |
| title | MAIN-VC: Lightweight Speech Representation Disentanglement for One-shot Voice Conversion |
| topic | Sound Audio and Speech Processing I.2.7; H.5.5 |
| url | https://arxiv.org/abs/2405.00930 |