RT-VC: Real-Time Zero-Shot Voice Conversion with Speech Articulatory Coding
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866908405626568704 |
|---|---|
| author | Liu, Yisi Wang, Chenyang Kim, Hanjo Khan, Raniya Anumanchipalli, Gopala |
| author_facet | Liu, Yisi Wang, Chenyang Kim, Hanjo Khan, Raniya Anumanchipalli, Gopala |
| contents | Voice conversion has emerged as a pivotal technology in numerous applications ranging from assistive communication to entertainment. In this paper, we present RT-VC, a zero-shot real-time voice conversion system that delivers ultra-low latency and high-quality performance. Our approach leverages an articulatory feature space to naturally disentangle content and speaker characteristics, facilitating more robust and interpretable voice transformations. Additionally, the integration of differentiable digital signal processing (DDSP) enables efficient vocoding directly from articulatory features, significantly reducing conversion latency. Experimental evaluations demonstrate that, while maintaining synthesis quality comparable to the current state-of-the-art (SOTA) method, RT-VC achieves a CPU latency of 61.4 ms, representing a 13.3\% reduction in latency. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_10289 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | RT-VC: Real-Time Zero-Shot Voice Conversion with Speech Articulatory Coding Liu, Yisi Wang, Chenyang Kim, Hanjo Khan, Raniya Anumanchipalli, Gopala Audio and Speech Processing Artificial Intelligence Voice conversion has emerged as a pivotal technology in numerous applications ranging from assistive communication to entertainment. In this paper, we present RT-VC, a zero-shot real-time voice conversion system that delivers ultra-low latency and high-quality performance. Our approach leverages an articulatory feature space to naturally disentangle content and speaker characteristics, facilitating more robust and interpretable voice transformations. Additionally, the integration of differentiable digital signal processing (DDSP) enables efficient vocoding directly from articulatory features, significantly reducing conversion latency. Experimental evaluations demonstrate that, while maintaining synthesis quality comparable to the current state-of-the-art (SOTA) method, RT-VC achieves a CPU latency of 61.4 ms, representing a 13.3\% reduction in latency. |
| title | RT-VC: Real-Time Zero-Shot Voice Conversion with Speech Articulatory Coding |
| topic | Audio and Speech Processing Artificial Intelligence |
| url | https://arxiv.org/abs/2506.10289 |