Teaching According to Talents! Instruction Tuning LLMs with Competence-Aware Curriculum Learning
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866908624147709952 |
|---|---|
| author | Li, Yangning Lu, Tingwei Li, Yinghui Chen, Yankai Huang, Wei-Chieh Jiang, Wenhao Wang, Hui Zheng, Hai-Tao Yu, Philip S. |
| author_facet | Li, Yangning Lu, Tingwei Li, Yinghui Chen, Yankai Huang, Wei-Chieh Jiang, Wenhao Wang, Hui Zheng, Hai-Tao Yu, Philip S. |
| contents | Efficient instruction tuning aims to enhance the ultimate performance of large language models (LLMs) trained on a given instruction dataset. Curriculum learning as a typical data organization strategy has shown preliminary effectiveness in instruction tuning. However, current curriculum tuning methods suffer from the curriculum rigidity, since they rely solely on static heuristic difficulty metrics. These methods fail to adapt to the evolving capabilities of models during training, resulting in a fixed and potentially sub-optimal learning trajectory. To address the issue, Competence-Aware Multi-Perspective cUrriculum inStruction tuning framework termed CAMPUS is proposed. CAMPUS offers several advantages: (1) Dynamic selection for sub-curriculum. (2) Competency-aware adjustment to the curriculum schedule. (3) Multiple difficulty-based scheduling. Extensive experiments prove the superior performance of CAMPUS, compared to other state-of-the-art baselines for efficient instruction tuning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_13790 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Teaching According to Talents! Instruction Tuning LLMs with Competence-Aware Curriculum Learning Li, Yangning Lu, Tingwei Li, Yinghui Chen, Yankai Huang, Wei-Chieh Jiang, Wenhao Wang, Hui Zheng, Hai-Tao Yu, Philip S. Computation and Language Artificial Intelligence Efficient instruction tuning aims to enhance the ultimate performance of large language models (LLMs) trained on a given instruction dataset. Curriculum learning as a typical data organization strategy has shown preliminary effectiveness in instruction tuning. However, current curriculum tuning methods suffer from the curriculum rigidity, since they rely solely on static heuristic difficulty metrics. These methods fail to adapt to the evolving capabilities of models during training, resulting in a fixed and potentially sub-optimal learning trajectory. To address the issue, Competence-Aware Multi-Perspective cUrriculum inStruction tuning framework termed CAMPUS is proposed. CAMPUS offers several advantages: (1) Dynamic selection for sub-curriculum. (2) Competency-aware adjustment to the curriculum schedule. (3) Multiple difficulty-based scheduling. Extensive experiments prove the superior performance of CAMPUS, compared to other state-of-the-art baselines for efficient instruction tuning. |
| title | Teaching According to Talents! Instruction Tuning LLMs with Competence-Aware Curriculum Learning |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2509.13790 |