InsBank: Evolving Instruction Subset for Ongoing Alignment
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912561230774272 |
|---|---|
| author | Shi, Jiayi Li, Yiwei Feng, Shaoxiong Yuan, Peiwen Wang, Xinglin Zhang, Yueqi Tan, Chuyi Pan, Boyuan Ren, Huan Hu, Yao Li, Kan |
| author_facet | Shi, Jiayi Li, Yiwei Feng, Shaoxiong Yuan, Peiwen Wang, Xinglin Zhang, Yueqi Tan, Chuyi Pan, Boyuan Ren, Huan Hu, Yao Li, Kan |
| contents | Large language models (LLMs) typically undergo instruction tuning to enhance alignment. Recent studies emphasize that quality and diversity of instruction data are more crucial than quantity, highlighting the need to select diverse, high-quality subsets to reduce training costs. However, how to evolve these selected subsets alongside the development of new instruction data remains insufficiently explored. To achieve LLMs' ongoing alignment, we introduce Instruction Bank (\textbf{InsBank}), a continuously updated repository that integrates the latest valuable instruction data. We further propose Progressive Instruction Bank Evolution (\textbf{PIBE}), a novel framework designed to evolve InsBank effectively and efficiently over time. PIBE employs a gradual data selection strategy to maintain long-term efficiency, leveraging a representation-based diversity score to capture relationships between data points and retain historical information for comprehensive diversity evaluation. This also allows for flexible combination of diversity and quality scores during data selection and ranking. Extensive experiments demonstrate that PIBE significantly outperforms baselines in InsBank evolution and is able to extract budget-specific subsets, demonstrating its effectiveness and adaptability. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_11419 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | InsBank: Evolving Instruction Subset for Ongoing Alignment Shi, Jiayi Li, Yiwei Feng, Shaoxiong Yuan, Peiwen Wang, Xinglin Zhang, Yueqi Tan, Chuyi Pan, Boyuan Ren, Huan Hu, Yao Li, Kan Computation and Language Large language models (LLMs) typically undergo instruction tuning to enhance alignment. Recent studies emphasize that quality and diversity of instruction data are more crucial than quantity, highlighting the need to select diverse, high-quality subsets to reduce training costs. However, how to evolve these selected subsets alongside the development of new instruction data remains insufficiently explored. To achieve LLMs' ongoing alignment, we introduce Instruction Bank (\textbf{InsBank}), a continuously updated repository that integrates the latest valuable instruction data. We further propose Progressive Instruction Bank Evolution (\textbf{PIBE}), a novel framework designed to evolve InsBank effectively and efficiently over time. PIBE employs a gradual data selection strategy to maintain long-term efficiency, leveraging a representation-based diversity score to capture relationships between data points and retain historical information for comprehensive diversity evaluation. This also allows for flexible combination of diversity and quality scores during data selection and ranking. Extensive experiments demonstrate that PIBE significantly outperforms baselines in InsBank evolution and is able to extract budget-specific subsets, demonstrating its effectiveness and adaptability. |
| title | InsBank: Evolving Instruction Subset for Ongoing Alignment |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2502.11419 |