LatentVoiceGrad: Nonparallel Voice Conversion with Latent Diffusion/Flow-Matching Models
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912580500455424 |
|---|---|
| author | Kameoka, Hirokazu Kaneko, Takuhiro Tanaka, Kou Kondo, Yuto |
| author_facet | Kameoka, Hirokazu Kaneko, Takuhiro Tanaka, Kou Kondo, Yuto |
| contents | Previously, we introduced VoiceGrad, a nonparallel voice conversion (VC) technique enabling mel-spectrogram conversion from source to target speakers using a score-based diffusion model. The concept involves training a score network to predict the gradient of the log density of mel-spectrograms from various speakers. VC is executed by iteratively adjusting an input mel-spectrogram until resembling the target speaker's. However, challenges persist: audio quality needs improvement, and conversion is slower compared to modern VC methods designed to operate at very high speeds. To address these, we introduce latent diffusion models into VoiceGrad, proposing an improved version with reverse diffusion in the autoencoder bottleneck. Additionally, we propose using a flow matching model as an alternative to the diffusion model to further speed up the conversion process without compromising the conversion quality. Experimental results show enhanced speech quality and accelerated conversion compared to the original. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_08379 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | LatentVoiceGrad: Nonparallel Voice Conversion with Latent Diffusion/Flow-Matching Models Kameoka, Hirokazu Kaneko, Takuhiro Tanaka, Kou Kondo, Yuto Sound Audio and Speech Processing Previously, we introduced VoiceGrad, a nonparallel voice conversion (VC) technique enabling mel-spectrogram conversion from source to target speakers using a score-based diffusion model. The concept involves training a score network to predict the gradient of the log density of mel-spectrograms from various speakers. VC is executed by iteratively adjusting an input mel-spectrogram until resembling the target speaker's. However, challenges persist: audio quality needs improvement, and conversion is slower compared to modern VC methods designed to operate at very high speeds. To address these, we introduce latent diffusion models into VoiceGrad, proposing an improved version with reverse diffusion in the autoencoder bottleneck. Additionally, we propose using a flow matching model as an alternative to the diffusion model to further speed up the conversion process without compromising the conversion quality. Experimental results show enhanced speech quality and accelerated conversion compared to the original. |
| title | LatentVoiceGrad: Nonparallel Voice Conversion with Latent Diffusion/Flow-Matching Models |
| topic | Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2509.08379 |