Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | https://arxiv.org/abs/2506.23979 |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917284742692864 |
|---|---|
| author | Jin, Renren Shen, Tianhao Wu, Xinwei Shi, Dan Sun, Haoran Ren, Yuqi Huang, Wuwei Wang, Quandong Liu, Wei Luan, Jian Wang, Bin Xiong, Deyi |
| author_facet | Jin, Renren Shen, Tianhao Wu, Xinwei Shi, Dan Sun, Haoran Ren, Yuqi Huang, Wuwei Wang, Quandong Liu, Wei Luan, Jian Wang, Bin Xiong, Deyi |
| contents | Conducting supervised and preference fine-tuning of large language models (LLMs) requires high-quality datasets to improve their ability to follow instructions and align with human preferences and values. However, constructing such datasets is resource-intensive, and most publicly available datasets are in English. To address these challenges, we propose the \underline{\textbf{Ta}}xonomy-Guided \underline{\textbf{P}}reference Data Generation (TaP) framework for automated, scalable preference dataset construction across languages. TaP uses a structured taxonomy to provide fine-grained control over dataset composition, ensuring diversity and broad coverage. We use TaP-generated datasets to perform supervised and preference fine-tuning on multiple LLMs. Experimental results demonstrate that LLMs trained on TaP-generated datasets outperform those trained on existing open-source datasets. Remarkably, LLMs trained on TaP-generated datasets outperform models trained on an open-source dataset that is 180$\times$ larger. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_23979 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | TaP: A Taxonomy-Guided Framework for Automated and Scalable Preference Data Generation Jin, Renren Shen, Tianhao Wu, Xinwei Shi, Dan Sun, Haoran Ren, Yuqi Huang, Wuwei Wang, Quandong Liu, Wei Luan, Jian Wang, Bin Xiong, Deyi Computation and Language Conducting supervised and preference fine-tuning of large language models (LLMs) requires high-quality datasets to improve their ability to follow instructions and align with human preferences and values. However, constructing such datasets is resource-intensive, and most publicly available datasets are in English. To address these challenges, we propose the \underline{\textbf{Ta}}xonomy-Guided \underline{\textbf{P}}reference Data Generation (TaP) framework for automated, scalable preference dataset construction across languages. TaP uses a structured taxonomy to provide fine-grained control over dataset composition, ensuring diversity and broad coverage. We use TaP-generated datasets to perform supervised and preference fine-tuning on multiple LLMs. Experimental results demonstrate that LLMs trained on TaP-generated datasets outperform those trained on existing open-source datasets. Remarkably, LLMs trained on TaP-generated datasets outperform models trained on an open-source dataset that is 180$\times$ larger. |
| title | TaP: A Taxonomy-Guided Framework for Automated and Scalable Preference Data Generation |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2506.23979 |