Beyond Scaling Law: A Data-Efficient Distillation Framework for Reasoning
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911103959695360 |
|---|---|
| author | Wu, Xiaojun Jiang, Xiaoguang Li, Huiyang Zhai, Jucai Liu, Dengfeng Hao, Qiaobo Liu, Huang Yang, Zhiguo Xie, Ji Gu, Ninglun Yang, Jin Zhang, Kailai Bao, Yelun Wang, Jun |
| author_facet | Wu, Xiaojun Jiang, Xiaoguang Li, Huiyang Zhai, Jucai Liu, Dengfeng Hao, Qiaobo Liu, Huang Yang, Zhiguo Xie, Ji Gu, Ninglun Yang, Jin Zhang, Kailai Bao, Yelun Wang, Jun |
| contents | Large language models (LLMs) demonstrate remarkable reasoning capabilities in tasks such as algorithmic coding and mathematical problem-solving. Recent methods have improved reasoning through expanded corpus and multistage training combining reinforcement learning and supervised fine-tuning. Although some methods suggest that small but targeted dataset can incentivize reasoning via only distillation, a reasoning scaling laws is still taking shape, increasing computational costs. To address this, we propose a data-efficient distillation framework (DED) that optimizes the Pareto frontier of reasoning distillation. Inspired by the on-policy learning and diverse roll-out strategies of reinforcement learning, the key idea of our approach is threefold: (1) We identify that benchmark scores alone do not determine an effective teacher model. Through comprehensive comparisons of leading reasoning LLMs, we develop a method to select an optimal teacher model. (2) While scaling distillation can enhance reasoning, it often degrades out-of-domain performance. A carefully curated, smaller corpus achieves a balanced trade-off between in-domain and out-of-domain capabilities. (3) Diverse reasoning trajectories encourage the student model to develop robust reasoning skills. We validate our method through evaluations on mathematical reasoning (AIME 2024/2025, MATH-500) and code generation (LiveCodeBench), achieving state-of-the-art results with only 0.8k carefully curated examples, bypassing the need for extensive scaling. Our systematic analysis demonstrates that DED outperforms existing methods by considering factors beyond superficial hardness, token length, or teacher model capability. This work offers a practical and efficient pathway to advanced reasoning while preserving general capabilities. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_09883 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Beyond Scaling Law: A Data-Efficient Distillation Framework for Reasoning Wu, Xiaojun Jiang, Xiaoguang Li, Huiyang Zhai, Jucai Liu, Dengfeng Hao, Qiaobo Liu, Huang Yang, Zhiguo Xie, Ji Gu, Ninglun Yang, Jin Zhang, Kailai Bao, Yelun Wang, Jun Machine Learning Artificial Intelligence Large language models (LLMs) demonstrate remarkable reasoning capabilities in tasks such as algorithmic coding and mathematical problem-solving. Recent methods have improved reasoning through expanded corpus and multistage training combining reinforcement learning and supervised fine-tuning. Although some methods suggest that small but targeted dataset can incentivize reasoning via only distillation, a reasoning scaling laws is still taking shape, increasing computational costs. To address this, we propose a data-efficient distillation framework (DED) that optimizes the Pareto frontier of reasoning distillation. Inspired by the on-policy learning and diverse roll-out strategies of reinforcement learning, the key idea of our approach is threefold: (1) We identify that benchmark scores alone do not determine an effective teacher model. Through comprehensive comparisons of leading reasoning LLMs, we develop a method to select an optimal teacher model. (2) While scaling distillation can enhance reasoning, it often degrades out-of-domain performance. A carefully curated, smaller corpus achieves a balanced trade-off between in-domain and out-of-domain capabilities. (3) Diverse reasoning trajectories encourage the student model to develop robust reasoning skills. We validate our method through evaluations on mathematical reasoning (AIME 2024/2025, MATH-500) and code generation (LiveCodeBench), achieving state-of-the-art results with only 0.8k carefully curated examples, bypassing the need for extensive scaling. Our systematic analysis demonstrates that DED outperforms existing methods by considering factors beyond superficial hardness, token length, or teacher model capability. This work offers a practical and efficient pathway to advanced reasoning while preserving general capabilities. |
| title | Beyond Scaling Law: A Data-Efficient Distillation Framework for Reasoning |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2508.09883 |