Fine-grained Preference Optimization Improves Zero-shot Text-to-Speech
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909975592304640 |
|---|---|
| author | Yao, Jixun Yang, Yuguang Pan, Yu Feng, Yuan Ning, Ziqian Ye, Jianhao Zhou, Hongbin Xie, Lei |
| author_facet | Yao, Jixun Yang, Yuguang Pan, Yu Feng, Yuan Ning, Ziqian Ye, Jianhao Zhou, Hongbin Xie, Lei |
| contents | Integrating human feedback to align text-to-speech (TTS) system outputs with human preferences has proven to be an effective approach for enhancing the robustness of language model-based TTS systems. Current approaches primarily focus on using preference data annotated at the utterance level. However, frequent issues that affect the listening experience often only arise in specific segments of audio samples, while other segments are well-generated. In this study, we propose a fine-grained preference optimization approach (FPO) to enhance the robustness of TTS systems. FPO focuses on addressing localized issues in generated samples rather than uniformly optimizing the entire utterance. Specifically, we first analyze the types of issues in generated samples, categorize them into two groups, and propose a selective training loss strategy to optimize preferences based on fine-grained labels for each issue type. Experimental results show that FPO enhances the robustness of zero-shot TTS systems by effectively addressing local issues, significantly reducing the bad case ratio, and improving intelligibility. Furthermore, FPO exhibits superior data efficiency compared with baseline systems, achieving similar performance with fewer training samples. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_02950 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Fine-grained Preference Optimization Improves Zero-shot Text-to-Speech Yao, Jixun Yang, Yuguang Pan, Yu Feng, Yuan Ning, Ziqian Ye, Jianhao Zhou, Hongbin Xie, Lei Audio and Speech Processing Sound Integrating human feedback to align text-to-speech (TTS) system outputs with human preferences has proven to be an effective approach for enhancing the robustness of language model-based TTS systems. Current approaches primarily focus on using preference data annotated at the utterance level. However, frequent issues that affect the listening experience often only arise in specific segments of audio samples, while other segments are well-generated. In this study, we propose a fine-grained preference optimization approach (FPO) to enhance the robustness of TTS systems. FPO focuses on addressing localized issues in generated samples rather than uniformly optimizing the entire utterance. Specifically, we first analyze the types of issues in generated samples, categorize them into two groups, and propose a selective training loss strategy to optimize preferences based on fine-grained labels for each issue type. Experimental results show that FPO enhances the robustness of zero-shot TTS systems by effectively addressing local issues, significantly reducing the bad case ratio, and improving intelligibility. Furthermore, FPO exhibits superior data efficiency compared with baseline systems, achieving similar performance with fewer training samples. |
| title | Fine-grained Preference Optimization Improves Zero-shot Text-to-Speech |
| topic | Audio and Speech Processing Sound |
| url | https://arxiv.org/abs/2502.02950 |