Small Models Struggle to Learn from Strong Reasoners
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909899992072192 |
|---|---|
| author | Li, Yuetai Yue, Xiang Xu, Zhangchen Jiang, Fengqing Niu, Luyao Lin, Bill Yuchen Ramasubramanian, Bhaskar Poovendran, Radha |
| author_facet | Li, Yuetai Yue, Xiang Xu, Zhangchen Jiang, Fengqing Niu, Luyao Lin, Bill Yuchen Ramasubramanian, Bhaskar Poovendran, Radha |
| contents | Large language models (LLMs) excel in complex reasoning tasks, and distilling their reasoning capabilities into smaller models has shown promise. However, we uncover an interesting phenomenon, which we term the Small Model Learnability Gap: small models ($\leq$3B parameters) do not consistently benefit from long chain-of-thought (CoT) reasoning or distillation from larger models. Instead, they perform better when fine-tuned on shorter, simpler reasoning chains that better align with their intrinsic learning capacity. To address this, we propose Mix Distillation, a simple yet effective strategy that balances reasoning complexity by combining long and short CoT examples or reasoning from both larger and smaller models. Our experiments demonstrate that Mix Distillation significantly improves small model reasoning performance compared to training on either data alone. These findings highlight the limitations of direct strong model distillation and underscore the importance of adapting reasoning complexity for effective reasoning capability transfer. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_12143 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Small Models Struggle to Learn from Strong Reasoners Li, Yuetai Yue, Xiang Xu, Zhangchen Jiang, Fengqing Niu, Luyao Lin, Bill Yuchen Ramasubramanian, Bhaskar Poovendran, Radha Artificial Intelligence Large language models (LLMs) excel in complex reasoning tasks, and distilling their reasoning capabilities into smaller models has shown promise. However, we uncover an interesting phenomenon, which we term the Small Model Learnability Gap: small models ($\leq$3B parameters) do not consistently benefit from long chain-of-thought (CoT) reasoning or distillation from larger models. Instead, they perform better when fine-tuned on shorter, simpler reasoning chains that better align with their intrinsic learning capacity. To address this, we propose Mix Distillation, a simple yet effective strategy that balances reasoning complexity by combining long and short CoT examples or reasoning from both larger and smaller models. Our experiments demonstrate that Mix Distillation significantly improves small model reasoning performance compared to training on either data alone. These findings highlight the limitations of direct strong model distillation and underscore the importance of adapting reasoning complexity for effective reasoning capability transfer. |
| title | Small Models Struggle to Learn from Strong Reasoners |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2502.12143 |