Multi-Response Preference Optimization with Augmented Ranking Dataset
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917865712517120 |
|---|---|
| author | Gwon, Hansle Ahn, Imjin Kim, Young-Hak Park, Sanghyun Jun, Tae Joon |
| author_facet | Gwon, Hansle Ahn, Imjin Kim, Young-Hak Park, Sanghyun Jun, Tae Joon |
| contents | Recent advancements in Large Language Models (LLMs) have been remarkable, with new models consistently surpassing their predecessors. These advancements are underpinned by extensive research on various training mechanisms. Among these, Preference Optimization has played a significant role in improving the performance of LLMs by incorporating human preferences into the training process. However, constructing preference optimization datasets is challenging and the optimization process is highly sensitive to the dataset quality. In this study, we propose a novel approach to augment Preference Optimization datasets. Additionally, we introduce a Multi-response-based Preference Optimization training method that enables the simultaneous learning of multiple responses. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_07812 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Multi-Response Preference Optimization with Augmented Ranking Dataset Gwon, Hansle Ahn, Imjin Kim, Young-Hak Park, Sanghyun Jun, Tae Joon Computation and Language Machine Learning Recent advancements in Large Language Models (LLMs) have been remarkable, with new models consistently surpassing their predecessors. These advancements are underpinned by extensive research on various training mechanisms. Among these, Preference Optimization has played a significant role in improving the performance of LLMs by incorporating human preferences into the training process. However, constructing preference optimization datasets is challenging and the optimization process is highly sensitive to the dataset quality. In this study, we propose a novel approach to augment Preference Optimization datasets. Additionally, we introduce a Multi-response-based Preference Optimization training method that enables the simultaneous learning of multiple responses. |
| title | Multi-Response Preference Optimization with Augmented Ranking Dataset |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2412.07812 |