StarVC: A Unified Auto-Regressive Framework for Joint Text and Speech Generation in Voice Conversion
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866915320503992320 |
|---|---|
| author | Li, Fengjin Wang, Jie Niu, Yadong Wang, Yongqing Meng, Meng Luan, Jian Wu, Zhiyong |
| author_facet | Li, Fengjin Wang, Jie Niu, Yadong Wang, Yongqing Meng, Meng Luan, Jian Wu, Zhiyong |
| contents | Voice Conversion (VC) modifies speech to match a target speaker while preserving linguistic content. Traditional methods usually extract speaker information directly from speech while neglecting the explicit utilization of linguistic content. Since VC fundamentally involves disentangling speaker identity from linguistic content, leveraging structured semantic features could enhance conversion performance. However, previous attempts to incorporate semantic features into VC have shown limited effectiveness, motivating the integration of explicit text modeling. We propose StarVC, a unified autoregressive VC framework that first predicts text tokens before synthesizing acoustic features. The experiments demonstrate that StarVC outperforms conventional VC methods in preserving both linguistic content (i.e., WER and CER) and speaker characteristics (i.e., SECS and MOS). Audio demo can be found at: https://thuhcsi.github.io/StarVC/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_02414 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | StarVC: A Unified Auto-Regressive Framework for Joint Text and Speech Generation in Voice Conversion Li, Fengjin Wang, Jie Niu, Yadong Wang, Yongqing Meng, Meng Luan, Jian Wu, Zhiyong Multimedia Computation and Language Sound Audio and Speech Processing Voice Conversion (VC) modifies speech to match a target speaker while preserving linguistic content. Traditional methods usually extract speaker information directly from speech while neglecting the explicit utilization of linguistic content. Since VC fundamentally involves disentangling speaker identity from linguistic content, leveraging structured semantic features could enhance conversion performance. However, previous attempts to incorporate semantic features into VC have shown limited effectiveness, motivating the integration of explicit text modeling. We propose StarVC, a unified autoregressive VC framework that first predicts text tokens before synthesizing acoustic features. The experiments demonstrate that StarVC outperforms conventional VC methods in preserving both linguistic content (i.e., WER and CER) and speaker characteristics (i.e., SECS and MOS). Audio demo can be found at: https://thuhcsi.github.io/StarVC/. |
| title | StarVC: A Unified Auto-Regressive Framework for Joint Text and Speech Generation in Voice Conversion |
| topic | Multimedia Computation and Language Sound Audio and Speech Processing |
| url | https://arxiv.org/abs/2506.02414 |