Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2507.07498 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915387836203008 |
|---|---|
| author | Bao, Keqin Chen, Nuo Li, Xiaoyuan Hui, Binyuan Yu, Bowen Feng, Fuli He, Xiangnan Liu, Dayiheng |
| author_facet | Bao, Keqin Chen, Nuo Li, Xiaoyuan Hui, Binyuan Yu, Bowen Feng, Fuli He, Xiangnan Liu, Dayiheng |
| contents | Enhancing reasoning capabilities remains a central focus in the LLM reasearch community. A promising direction involves requiring models to simulate code execution step-by-step to derive outputs for given inputs. However, as code is often designed for large-scale systems, direct application leads to over-reliance on complex data structures and algorithms, even for simple cases, resulting in overfitting to algorithmic patterns rather than core reasoning structures. To address this, we propose TeaR, which aims at teaching LLMs to reason better. TeaR leverages careful data curation and reinforcement learning to guide models in discovering optimal reasoning paths through code-related tasks, thereby improving general reasoning abilities. We conduct extensive experiments using two base models and three long-CoT distillation models, with model sizes ranging from 1.5 billion to 32 billion parameters, and across 17 benchmarks spanning Math, Knowledge, Code, and Logical Reasoning. The results consistently show significant performance improvements. Notably, TeaR achieves a 35.9% improvement on Qwen2.5-7B and 5.9% on R1-Distilled-7B. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_07498 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Teaching LLM to Reason: Reinforcement Learning from Algorithmic Problems without Code Bao, Keqin Chen, Nuo Li, Xiaoyuan Hui, Binyuan Yu, Bowen Feng, Fuli He, Xiangnan Liu, Dayiheng Computation and Language Machine Learning Enhancing reasoning capabilities remains a central focus in the LLM reasearch community. A promising direction involves requiring models to simulate code execution step-by-step to derive outputs for given inputs. However, as code is often designed for large-scale systems, direct application leads to over-reliance on complex data structures and algorithms, even for simple cases, resulting in overfitting to algorithmic patterns rather than core reasoning structures. To address this, we propose TeaR, which aims at teaching LLMs to reason better. TeaR leverages careful data curation and reinforcement learning to guide models in discovering optimal reasoning paths through code-related tasks, thereby improving general reasoning abilities. We conduct extensive experiments using two base models and three long-CoT distillation models, with model sizes ranging from 1.5 billion to 32 billion parameters, and across 17 benchmarks spanning Math, Knowledge, Code, and Logical Reasoning. The results consistently show significant performance improvements. Notably, TeaR achieves a 35.9% improvement on Qwen2.5-7B and 5.9% on R1-Distilled-7B. |
| title | Teaching LLM to Reason: Reinforcement Learning from Algorithmic Problems without Code |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2507.07498 |