Aligning Instruction Tuning with Pre-training
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866911101027876864 |
|---|---|
| author | Liang, Yiming Zheng, Tianyu Du, Xinrun Zhang, Ge Liu, Jiaheng Qu, Xingwei Zu, Wenqiang Xing, Xingrun Zheng, Chujie Ma, Lei Wang, Guoyin Zhang, Zhaoxiang Huang, Wenhao Yue, Xiang Zhang, Jiajun |
| author_facet | Liang, Yiming Zheng, Tianyu Du, Xinrun Zhang, Ge Liu, Jiaheng Qu, Xingwei Zu, Wenqiang Xing, Xingrun Zheng, Chujie Ma, Lei Wang, Guoyin Zhang, Zhaoxiang Huang, Wenhao Yue, Xiang Zhang, Jiajun |
| contents | Instruction tuning enhances large language models (LLMs) to follow human instructions across diverse tasks, relying on high-quality datasets to guide behavior. However, these datasets, whether manually curated or synthetically generated, are often narrowly focused and misaligned with the broad distributions captured during pre-training, limiting LLM generalization and effective use of pre-trained knowledge. We propose Aligning Instruction Tuning with Pre-training (AITP), a method that bridges this gap by identifying coverage shortfalls in instruction-tuning datasets and rewriting underrepresented pre-training data into high-quality instruction-response pairs. This approach enriches dataset diversity while preserving task-specific objectives. Evaluations on three fully open LLMs across eight benchmarks demonstrate consistent performance improvements with AITP. Ablations highlight the benefits of adaptive data selection, controlled rewriting, and balanced integration, emphasizing the importance of aligning instruction tuning with pre-training distributions to unlock the full potential of LLMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_09368 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Aligning Instruction Tuning with Pre-training Liang, Yiming Zheng, Tianyu Du, Xinrun Zhang, Ge Liu, Jiaheng Qu, Xingwei Zu, Wenqiang Xing, Xingrun Zheng, Chujie Ma, Lei Wang, Guoyin Zhang, Zhaoxiang Huang, Wenhao Yue, Xiang Zhang, Jiajun Artificial Intelligence Instruction tuning enhances large language models (LLMs) to follow human instructions across diverse tasks, relying on high-quality datasets to guide behavior. However, these datasets, whether manually curated or synthetically generated, are often narrowly focused and misaligned with the broad distributions captured during pre-training, limiting LLM generalization and effective use of pre-trained knowledge. We propose Aligning Instruction Tuning with Pre-training (AITP), a method that bridges this gap by identifying coverage shortfalls in instruction-tuning datasets and rewriting underrepresented pre-training data into high-quality instruction-response pairs. This approach enriches dataset diversity while preserving task-specific objectives. Evaluations on three fully open LLMs across eight benchmarks demonstrate consistent performance improvements with AITP. Ablations highlight the benefits of adaptive data selection, controlled rewriting, and balanced integration, emphasizing the importance of aligning instruction tuning with pre-training distributions to unlock the full potential of LLMs. |
| title | Aligning Instruction Tuning with Pre-training |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2501.09368 |