Enhancing Long-Chain Reasoning Distillation through Error-Aware Self-Reflection
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912645382144000 |
|---|---|
| author | Wu, Zhuoyang Li, Xinze Liu, Zhenghao Yan, Yukun Liu, Zhiyuan Yu, Minghe Yang, Cheng Gu, Yu Yu, Ge Sun, Maosong |
| author_facet | Wu, Zhuoyang Li, Xinze Liu, Zhenghao Yan, Yukun Liu, Zhiyuan Yu, Minghe Yang, Cheng Gu, Yu Yu, Ge Sun, Maosong |
| contents | Large Language Models (LLMs) have exhibited strong reasoning capabilities and achieved remarkable performance in mathematical problem-solving tasks. Recently, distilling reasoning ability from long-form Chains-of-Thought (CoTs) has emerged as a promising approach for enhancing Small Language Models (SLMs). Existing studies typically treat SLMs as student models and use long-form CoTs as supervision signals for Supervised Fine-Tuning (SFT) to transfer reasoning ability. However, such long-form CoT teachers are usually unaware of the student model's capacity, which limits the effective utilization of the provided reasoning traces. To overcome this limitation, we propose errOr-aware self-ReflectION (ORION), a framework that refines teacher CoTs through an Error-Aware Reflection process. ORION enables the student model to construct more tailored teacher CoTs by refining teacher CoTs and incorporating its own reasoning errors. Experiments on multiple mathematical reasoning benchmarks demonstrate that ORION consistently improves performance by more than 2% over all baselines. Further analysis reveals that the CoTs constructed by ORION exhibit higher coherence and logical consistency, thereby serving as more effective supervision signals for SFT. All codes are available at https://github.com/NEUIR/ORION.git. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_22131 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Enhancing Long-Chain Reasoning Distillation through Error-Aware Self-Reflection Wu, Zhuoyang Li, Xinze Liu, Zhenghao Yan, Yukun Liu, Zhiyuan Yu, Minghe Yang, Cheng Gu, Yu Yu, Ge Sun, Maosong Computation and Language Large Language Models (LLMs) have exhibited strong reasoning capabilities and achieved remarkable performance in mathematical problem-solving tasks. Recently, distilling reasoning ability from long-form Chains-of-Thought (CoTs) has emerged as a promising approach for enhancing Small Language Models (SLMs). Existing studies typically treat SLMs as student models and use long-form CoTs as supervision signals for Supervised Fine-Tuning (SFT) to transfer reasoning ability. However, such long-form CoT teachers are usually unaware of the student model's capacity, which limits the effective utilization of the provided reasoning traces. To overcome this limitation, we propose errOr-aware self-ReflectION (ORION), a framework that refines teacher CoTs through an Error-Aware Reflection process. ORION enables the student model to construct more tailored teacher CoTs by refining teacher CoTs and incorporating its own reasoning errors. Experiments on multiple mathematical reasoning benchmarks demonstrate that ORION consistently improves performance by more than 2% over all baselines. Further analysis reveals that the CoTs constructed by ORION exhibit higher coherence and logical consistency, thereby serving as more effective supervision signals for SFT. All codes are available at https://github.com/NEUIR/ORION.git. |
| title | Enhancing Long-Chain Reasoning Distillation through Error-Aware Self-Reflection |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2505.22131 |