Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2409.12403 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912034632761344 |
|---|---|
| author | Tian, Jinchuan Zhang, Chunlei Shi, Jiatong Zhang, Hao Yu, Jianwei Watanabe, Shinji Yu, Dong |
| author_facet | Tian, Jinchuan Zhang, Chunlei Shi, Jiatong Zhang, Hao Yu, Jianwei Watanabe, Shinji Yu, Dong |
| contents | Recent advancements in text-to-speech (TTS) have shown that language model (LM)-based systems offer competitive performance to their counterparts. Further optimization can be achieved through preference alignment algorithms, which adjust LMs to align with the preferences of reward models, enhancing the desirability of the generated content. This study presents a thorough empirical evaluation of how preference alignment algorithms, particularly Direct Preference Optimization (DPO), enhance LM-based TTS. With a 1.15B parameter LM-based TTS model, we demonstrate that preference alignment consistently improves intelligibility, speaker similarity, and proxy subjective evaluation scores, with the latter two metrics surpassing even human speech in certain evaluations. We also show preference alignment is applicable to low-resource scenarios and effectively generalized to out-of-domain applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_12403 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Preference Alignment Improves Language Model-Based TTS Tian, Jinchuan Zhang, Chunlei Shi, Jiatong Zhang, Hao Yu, Jianwei Watanabe, Shinji Yu, Dong Computation and Language Artificial Intelligence Recent advancements in text-to-speech (TTS) have shown that language model (LM)-based systems offer competitive performance to their counterparts. Further optimization can be achieved through preference alignment algorithms, which adjust LMs to align with the preferences of reward models, enhancing the desirability of the generated content. This study presents a thorough empirical evaluation of how preference alignment algorithms, particularly Direct Preference Optimization (DPO), enhance LM-based TTS. With a 1.15B parameter LM-based TTS model, we demonstrate that preference alignment consistently improves intelligibility, speaker similarity, and proxy subjective evaluation scores, with the latter two metrics surpassing even human speech in certain evaluations. We also show preference alignment is applicable to low-resource scenarios and effectively generalized to out-of-domain applications. |
| title | Preference Alignment Improves Language Model-Based TTS |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2409.12403 |