Noro: Noise-Robust One-shot Voice Conversion with Hidden Speaker Representation Learning
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916922334904320 |
|---|---|
| author | He, Haorui Song, Yuchen Wang, Yuancheng Li, Haoyang Zhang, Xueyao Wang, Li Huang, Gongping Chng, Eng Siong Wu, Zhizheng |
| author_facet | He, Haorui Song, Yuchen Wang, Yuancheng Li, Haoyang Zhang, Xueyao Wang, Li Huang, Gongping Chng, Eng Siong Wu, Zhizheng |
| contents | The effectiveness of one-shot voice conversion (VC) decreases in real-world scenarios where reference speeches, which are often sourced from the internet, contain various disturbances like background noise. To address this issue, we introduce Noro, a noise-robust one-shot VC system. Noro features innovative components tailored for VC using noisy reference speeches, including a dual-branch reference encoding module and a noise-agnostic contrastive speaker loss. Experimental results demonstrate that Noro outperforms our baseline system in both clean and noisy scenarios, highlighting its efficacy for real-world applications. Additionally, we investigate the hidden speaker representation capabilities of our baseline system by repurposing its reference encoder as a speaker encoder. The results show that it is competitive with several advanced self-supervised learning models for speaker representation under the SUPERB settings, highlighting the potential for advancing speaker representation learning through one-shot VC tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_19770 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Noro: Noise-Robust One-shot Voice Conversion with Hidden Speaker Representation Learning He, Haorui Song, Yuchen Wang, Yuancheng Li, Haoyang Zhang, Xueyao Wang, Li Huang, Gongping Chng, Eng Siong Wu, Zhizheng Sound Computation and Language Audio and Speech Processing The effectiveness of one-shot voice conversion (VC) decreases in real-world scenarios where reference speeches, which are often sourced from the internet, contain various disturbances like background noise. To address this issue, we introduce Noro, a noise-robust one-shot VC system. Noro features innovative components tailored for VC using noisy reference speeches, including a dual-branch reference encoding module and a noise-agnostic contrastive speaker loss. Experimental results demonstrate that Noro outperforms our baseline system in both clean and noisy scenarios, highlighting its efficacy for real-world applications. Additionally, we investigate the hidden speaker representation capabilities of our baseline system by repurposing its reference encoder as a speaker encoder. The results show that it is competitive with several advanced self-supervised learning models for speaker representation under the SUPERB settings, highlighting the potential for advancing speaker representation learning through one-shot VC tasks. |
| title | Noro: Noise-Robust One-shot Voice Conversion with Hidden Speaker Representation Learning |
| topic | Sound Computation and Language Audio and Speech Processing |
| url | https://arxiv.org/abs/2411.19770 |