Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2511.10112 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914156260622336 |
|---|---|
| author | Wang, Wenyu Hu, Zhetao Zhou, Yiquan Xu, Jiacheng Wu, Zhiyu Li, Chen Li, Shihao |
| author_facet | Wang, Wenyu Hu, Zhetao Zhou, Yiquan Xu, Jiacheng Wu, Zhiyu Li, Chen Li, Shihao |
| contents | In voice conversion (VC), it is crucial to preserve complete semantic information while accurately modeling the target speaker's timbre and prosody. This paper proposes FabasedVC to achieve VC with enhanced similarity in timbre, prosody, and duration to the target speaker, as well as improved content integrity. It is an end-to-end VITS-based VC system that integrates relevant textual modality information, phoneme-level self-supervised learning (SSL) features, and a duration predictor. Specifically, we employ a text feature encoder to encode attributes such as text, phonemes, tones and BERT features. We then process the frame-level SSL features into phoneme-level features using two methods: average pooling and attention mechanism based on each phoneme's duration. Moreover, a duration predictor is incorporated to better align the speech rate and prosody of the target speaker. Experimental results demonstrate that our method outperforms competing systems in terms of naturalness, similarity, and content integrity. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_10112 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | FabasedVC: Enhancing Voice Conversion with Text Modality Fusion and Phoneme-Level SSL Features Wang, Wenyu Hu, Zhetao Zhou, Yiquan Xu, Jiacheng Wu, Zhiyu Li, Chen Li, Shihao Sound In voice conversion (VC), it is crucial to preserve complete semantic information while accurately modeling the target speaker's timbre and prosody. This paper proposes FabasedVC to achieve VC with enhanced similarity in timbre, prosody, and duration to the target speaker, as well as improved content integrity. It is an end-to-end VITS-based VC system that integrates relevant textual modality information, phoneme-level self-supervised learning (SSL) features, and a duration predictor. Specifically, we employ a text feature encoder to encode attributes such as text, phonemes, tones and BERT features. We then process the frame-level SSL features into phoneme-level features using two methods: average pooling and attention mechanism based on each phoneme's duration. Moreover, a duration predictor is incorporated to better align the speech rate and prosody of the target speaker. Experimental results demonstrate that our method outperforms competing systems in terms of naturalness, similarity, and content integrity. |
| title | FabasedVC: Enhancing Voice Conversion with Text Modality Fusion and Phoneme-Level SSL Features |
| topic | Sound |
| url | https://arxiv.org/abs/2511.10112 |