Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2507.09850 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916845931462656 |
|---|---|
| author | Du, Wei Kisacanin, Branislav Armstrong, George Toshniwal, Shubham Moshkov, Ivan Ayrapetyan, Alexan Mahdavi, Sadegh Zhao, Dan Diao, Shizhe Masulovic, Dragan Stanean, Marius Avadhanam, Advaith Wang, Max Dutta, Ashmit Govil, Shitij Yanamandara, Sri Tandon, Mihir Ananthakrishnan, Sriram Rathi, Vedant Zhang, David Kang, Joonseok Luo, Leon Andreescu, Titu Ginsburg, Boris Gitman, Igor |
| author_facet | Du, Wei Kisacanin, Branislav Armstrong, George Toshniwal, Shubham Moshkov, Ivan Ayrapetyan, Alexan Mahdavi, Sadegh Zhao, Dan Diao, Shizhe Masulovic, Dragan Stanean, Marius Avadhanam, Advaith Wang, Max Dutta, Ashmit Govil, Shitij Yanamandara, Sri Tandon, Mihir Ananthakrishnan, Sriram Rathi, Vedant Zhang, David Kang, Joonseok Luo, Leon Andreescu, Titu Ginsburg, Boris Gitman, Igor |
| contents | Reasoning-capable language models achieve state-of-the-art performance in diverse complex tasks by generating long, explicit Chain-of-Thought (CoT) traces. While recent works show that base models can acquire such reasoning traces via reinforcement learning or distillation from stronger models like DeepSeek-R1, previous works demonstrate that even short CoT prompting without fine-tuning is able to improve reasoning. We ask whether long CoT can be induced in a base model using only prompting or minimal tuning. Using just 20 long CoT examples from the reasoning model \texttt{QwQ-32B-Preview}, we lightly fine-tune the base model \texttt{Qwen2.5-32B}. The resulting model outperforms the much larger \texttt{Qwen2.5-Math-72B-Instruct}, showing that a handful of high-quality examples can unlock strong reasoning capabilities. We further explore using CoT data from non-reasoning models and human annotators, enhanced with prompt engineering, multi-pass editing, and structural guidance. However, neither matches the performance of reasoning model traces, suggesting that certain latent qualities of expert CoT are difficult to replicate. We analyze key properties of reasoning data, such as problem difficulty, diversity, and answer length, that influence reasoning distillation. While challenges remain, we are optimistic that carefully curated human-written CoT, even in small quantities, can activate reasoning behaviors in base models. We release our human-authored dataset across refinement stages and invite further investigation into what makes small-scale reasoning supervision so effective. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_09850 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | The Challenge of Teaching Reasoning to LLMs Without RL or Distillation Du, Wei Kisacanin, Branislav Armstrong, George Toshniwal, Shubham Moshkov, Ivan Ayrapetyan, Alexan Mahdavi, Sadegh Zhao, Dan Diao, Shizhe Masulovic, Dragan Stanean, Marius Avadhanam, Advaith Wang, Max Dutta, Ashmit Govil, Shitij Yanamandara, Sri Tandon, Mihir Ananthakrishnan, Sriram Rathi, Vedant Zhang, David Kang, Joonseok Luo, Leon Andreescu, Titu Ginsburg, Boris Gitman, Igor Artificial Intelligence Reasoning-capable language models achieve state-of-the-art performance in diverse complex tasks by generating long, explicit Chain-of-Thought (CoT) traces. While recent works show that base models can acquire such reasoning traces via reinforcement learning or distillation from stronger models like DeepSeek-R1, previous works demonstrate that even short CoT prompting without fine-tuning is able to improve reasoning. We ask whether long CoT can be induced in a base model using only prompting or minimal tuning. Using just 20 long CoT examples from the reasoning model \texttt{QwQ-32B-Preview}, we lightly fine-tune the base model \texttt{Qwen2.5-32B}. The resulting model outperforms the much larger \texttt{Qwen2.5-Math-72B-Instruct}, showing that a handful of high-quality examples can unlock strong reasoning capabilities. We further explore using CoT data from non-reasoning models and human annotators, enhanced with prompt engineering, multi-pass editing, and structural guidance. However, neither matches the performance of reasoning model traces, suggesting that certain latent qualities of expert CoT are difficult to replicate. We analyze key properties of reasoning data, such as problem difficulty, diversity, and answer length, that influence reasoning distillation. While challenges remain, we are optimistic that carefully curated human-written CoT, even in small quantities, can activate reasoning behaviors in base models. We release our human-authored dataset across refinement stages and invite further investigation into what makes small-scale reasoning supervision so effective. |
| title | The Challenge of Teaching Reasoning to LLMs Without RL or Distillation |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2507.09850 |