Curriculum Learning for Efficient Chain-of-Thought Distillation via Structure-Aware Masking and GRPO
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866915836652945408 |
|---|---|
| author | Yu, Bowen Wang, Maolin Zhang, Sheng Wang, Binhao Wen, Yi Gao, Jingtong Liu, Bowen Zhao, Zimo Wang, Wanyu Zhao, Xiangyu |
| author_facet | Yu, Bowen Wang, Maolin Zhang, Sheng Wang, Binhao Wen, Yi Gao, Jingtong Liu, Bowen Zhao, Zimo Wang, Wanyu Zhao, Xiangyu |
| contents | Distilling Chain-of-Thought (CoT) reasoning from large language models into compact student models presents a fundamental challenge: teacher rationales are often too verbose for smaller models to faithfully reproduce. Existing approaches either compress reasoning into single-step, losing the interpretability that makes CoT valuable. We present a three-stage curriculum learning framework that addresses this capacity mismatch through progressive skill acquisition. First, we establish structural understanding via masked shuffled reconstruction. Second, we apply Group Relative Policy Optimization (GRPO) on masked completion tasks, enabling the model to discover its own balance between accuracy and brevity. Third, we identify persistent failure cases and guide the student to internalize teacher knowledge through targeted rewriting, again optimized with GRPO. Experiments on GSM8K demonstrate that our approach enables Qwen2.5-3B-Base to achieve an 11.29 percent accuracy improvement while reducing output length by 27.4 percent, surpassing both instruction-tuned variants and prior distillation methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_17686 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Curriculum Learning for Efficient Chain-of-Thought Distillation via Structure-Aware Masking and GRPO Yu, Bowen Wang, Maolin Zhang, Sheng Wang, Binhao Wen, Yi Gao, Jingtong Liu, Bowen Zhao, Zimo Wang, Wanyu Zhao, Xiangyu Machine Learning Artificial Intelligence Distilling Chain-of-Thought (CoT) reasoning from large language models into compact student models presents a fundamental challenge: teacher rationales are often too verbose for smaller models to faithfully reproduce. Existing approaches either compress reasoning into single-step, losing the interpretability that makes CoT valuable. We present a three-stage curriculum learning framework that addresses this capacity mismatch through progressive skill acquisition. First, we establish structural understanding via masked shuffled reconstruction. Second, we apply Group Relative Policy Optimization (GRPO) on masked completion tasks, enabling the model to discover its own balance between accuracy and brevity. Third, we identify persistent failure cases and guide the student to internalize teacher knowledge through targeted rewriting, again optimized with GRPO. Experiments on GSM8K demonstrate that our approach enables Qwen2.5-3B-Base to achieve an 11.29 percent accuracy improvement while reducing output length by 27.4 percent, surpassing both instruction-tuned variants and prior distillation methods. |
| title | Curriculum Learning for Efficient Chain-of-Thought Distillation via Structure-Aware Masking and GRPO |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2602.17686 |