BATON: Aligning Text-to-Audio Model with Human Preference Feedback
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866914663486193664 |
|---|---|
| author | Liao, Huan Han, Haonan Yang, Kai Du, Tianjiao Yang, Rui Xu, Zunnan Xu, Qinmei Liu, Jingquan Lu, Jiasheng Li, Xiu |
| author_facet | Liao, Huan Han, Haonan Yang, Kai Du, Tianjiao Yang, Rui Xu, Zunnan Xu, Qinmei Liu, Jingquan Lu, Jiasheng Li, Xiu |
| contents | With the development of AI-Generated Content (AIGC), text-to-audio models are gaining widespread attention. However, it is challenging for these models to generate audio aligned with human preference due to the inherent information density of natural language and limited model understanding ability. To alleviate this issue, we formulate the BATON, a framework designed to enhance the alignment between generated audio and text prompt using human preference feedback. Our BATON comprises three key stages: Firstly, we curated a dataset containing both prompts and the corresponding generated audio, which was then annotated based on human feedback. Secondly, we introduced a reward model using the constructed dataset, which can mimic human preference by assigning rewards to input text-audio pairs. Finally, we employed the reward model to fine-tune an off-the-shelf text-to-audio model. The experiment results demonstrate that our BATON can significantly improve the generation quality of the original text-to-audio models, concerning audio integrity, temporal relationship, and alignment with human preference. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_00744 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | BATON: Aligning Text-to-Audio Model with Human Preference Feedback Liao, Huan Han, Haonan Yang, Kai Du, Tianjiao Yang, Rui Xu, Zunnan Xu, Qinmei Liu, Jingquan Lu, Jiasheng Li, Xiu Sound Computation and Language Audio and Speech Processing With the development of AI-Generated Content (AIGC), text-to-audio models are gaining widespread attention. However, it is challenging for these models to generate audio aligned with human preference due to the inherent information density of natural language and limited model understanding ability. To alleviate this issue, we formulate the BATON, a framework designed to enhance the alignment between generated audio and text prompt using human preference feedback. Our BATON comprises three key stages: Firstly, we curated a dataset containing both prompts and the corresponding generated audio, which was then annotated based on human feedback. Secondly, we introduced a reward model using the constructed dataset, which can mimic human preference by assigning rewards to input text-audio pairs. Finally, we employed the reward model to fine-tune an off-the-shelf text-to-audio model. The experiment results demonstrate that our BATON can significantly improve the generation quality of the original text-to-audio models, concerning audio integrity, temporal relationship, and alignment with human preference. |
| title | BATON: Aligning Text-to-Audio Model with Human Preference Feedback |
| topic | Sound Computation and Language Audio and Speech Processing |
| url | https://arxiv.org/abs/2402.00744 |