Beyond IID: Optimizing Instruction Learning from the Perspective of Instruction Interaction and Dependency
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866913496972656640 |
|---|---|
| author | Zhao, Hanyu Du, Li Ju, Yiming Wu, Chengwei Pan, Tengfei |
| author_facet | Zhao, Hanyu Du, Li Ju, Yiming Wu, Chengwei Pan, Tengfei |
| contents | With the availability of various instruction datasets, a pivotal challenge is how to effectively select and integrate these instructions to fine-tune large language models (LLMs). Previous research mainly focuses on selecting individual high-quality instructions. However, these works overlooked the joint interactions and dependencies between different categories of instructions, leading to suboptimal selection strategies. Moreover, the nature of these interaction patterns remains largely unexplored, let alone optimize the instruction set with regard to them. To fill these gaps, in this paper, we: (1) systemically investigate interaction and dependency patterns between different categories of instructions, (2) manage to optimize the instruction set concerning the interaction patterns using a linear programming-based method, and optimize the learning schema of SFT using an instruction dependency taxonomy guided curriculum learning. Experimental results across different LLMs demonstrate improved performance over strong baselines on widely adopted benchmarks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_07045 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Beyond IID: Optimizing Instruction Learning from the Perspective of Instruction Interaction and Dependency Zhao, Hanyu Du, Li Ju, Yiming Wu, Chengwei Pan, Tengfei Computation and Language Artificial Intelligence With the availability of various instruction datasets, a pivotal challenge is how to effectively select and integrate these instructions to fine-tune large language models (LLMs). Previous research mainly focuses on selecting individual high-quality instructions. However, these works overlooked the joint interactions and dependencies between different categories of instructions, leading to suboptimal selection strategies. Moreover, the nature of these interaction patterns remains largely unexplored, let alone optimize the instruction set with regard to them. To fill these gaps, in this paper, we: (1) systemically investigate interaction and dependency patterns between different categories of instructions, (2) manage to optimize the instruction set concerning the interaction patterns using a linear programming-based method, and optimize the learning schema of SFT using an instruction dependency taxonomy guided curriculum learning. Experimental results across different LLMs demonstrate improved performance over strong baselines on widely adopted benchmarks. |
| title | Beyond IID: Optimizing Instruction Learning from the Perspective of Instruction Interaction and Dependency |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2409.07045 |