PromptEVC: Controllable Emotional Voice Conversion with Natural Language Prompts
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915306093412352 |
|---|---|
| author | Qi, Tianhua Wang, Shiyan Lu, Cheng Song, Tengfei Yang, Hao Wu, Zhanglin Zheng, Wenming |
| author_facet | Qi, Tianhua Wang, Shiyan Lu, Cheng Song, Tengfei Yang, Hao Wu, Zhanglin Zheng, Wenming |
| contents | Controllable emotional voice conversion (EVC) aims to manipulate emotional expressions to increase the diversity of synthesized speech. Existing methods typically rely on predefined labels, reference audios, or prespecified factor values, often overlooking individual differences in emotion perception and expression. In this paper, we introduce PromptEVC that utilizes natural language prompts for precise and flexible emotion control. To bridge text descriptions with emotional speech, we propose emotion descriptor and prompt mapper to generate fine-grained emotion embeddings, trained jointly with reference embeddings. To enhance naturalness, we present a prosody modeling and control pipeline that adjusts the rhythm based on linguistic content and emotional cues. Additionally, a speaker encoder is incorporated to preserve identity. Experimental results demonstrate that PromptEVC outperforms state-of-the-art controllable EVC methods in emotion conversion, intensity control, mixed emotion synthesis, and prosody manipulation. Speech samples are available at https://jeremychee4.github.io/PromptEVC/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_20678 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | PromptEVC: Controllable Emotional Voice Conversion with Natural Language Prompts Qi, Tianhua Wang, Shiyan Lu, Cheng Song, Tengfei Yang, Hao Wu, Zhanglin Zheng, Wenming Audio and Speech Processing Sound Signal Processing Controllable emotional voice conversion (EVC) aims to manipulate emotional expressions to increase the diversity of synthesized speech. Existing methods typically rely on predefined labels, reference audios, or prespecified factor values, often overlooking individual differences in emotion perception and expression. In this paper, we introduce PromptEVC that utilizes natural language prompts for precise and flexible emotion control. To bridge text descriptions with emotional speech, we propose emotion descriptor and prompt mapper to generate fine-grained emotion embeddings, trained jointly with reference embeddings. To enhance naturalness, we present a prosody modeling and control pipeline that adjusts the rhythm based on linguistic content and emotional cues. Additionally, a speaker encoder is incorporated to preserve identity. Experimental results demonstrate that PromptEVC outperforms state-of-the-art controllable EVC methods in emotion conversion, intensity control, mixed emotion synthesis, and prosody manipulation. Speech samples are available at https://jeremychee4.github.io/PromptEVC/. |
| title | PromptEVC: Controllable Emotional Voice Conversion with Natural Language Prompts |
| topic | Audio and Speech Processing Sound Signal Processing |
| url | https://arxiv.org/abs/2505.20678 |