DAST: Difficulty-Aware Self-Training on Large Language Models
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912270653587456 |
|---|---|
| author | Xue, Boyang Zhu, Qi Wang, Hongru Wang, Rui Wang, Sheng Xu, Hongling Mi, Fei Wang, Yasheng Shang, Lifeng Liu, Qun Wong, Kam-Fai |
| author_facet | Xue, Boyang Zhu, Qi Wang, Hongru Wang, Rui Wang, Sheng Xu, Hongling Mi, Fei Wang, Yasheng Shang, Lifeng Liu, Qun Wong, Kam-Fai |
| contents | Present Large Language Models (LLM) self-training methods always under-sample on challenging queries, leading to inadequate learning on difficult problems which limits LLMs' ability. Therefore, this work proposes a difficulty-aware self-training (DAST) framework that focuses on improving both the quantity and quality of self-generated responses on challenging queries during self-training. DAST is specified in three components: 1) sampling-based difficulty level estimation, 2) difficulty-aware data augmentation, and 3) the self-training algorithm using SFT and DPO respectively. Experiments on mathematical tasks demonstrate the effectiveness and generalization of DAST, highlighting the critical role of difficulty-aware strategies in advancing LLM self-training. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_09029 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | DAST: Difficulty-Aware Self-Training on Large Language Models Xue, Boyang Zhu, Qi Wang, Hongru Wang, Rui Wang, Sheng Xu, Hongling Mi, Fei Wang, Yasheng Shang, Lifeng Liu, Qun Wong, Kam-Fai Computation and Language Present Large Language Models (LLM) self-training methods always under-sample on challenging queries, leading to inadequate learning on difficult problems which limits LLMs' ability. Therefore, this work proposes a difficulty-aware self-training (DAST) framework that focuses on improving both the quantity and quality of self-generated responses on challenging queries during self-training. DAST is specified in three components: 1) sampling-based difficulty level estimation, 2) difficulty-aware data augmentation, and 3) the self-training algorithm using SFT and DPO respectively. Experiments on mathematical tasks demonstrate the effectiveness and generalization of DAST, highlighting the critical role of difficulty-aware strategies in advancing LLM self-training. |
| title | DAST: Difficulty-Aware Self-Training on Large Language Models |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2503.09029 |