Reverse-Engineered Reasoning for Open-Ended Generation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909775529246720 |
|---|---|
| author | Wang, Haozhe Que, Haoran Xu, Qixin Liu, Minghao Zhou, Wangchunshu Feng, Jiazhan Zhong, Wanjun Ye, Wei Yang, Tong Huang, Wenhao Zhang, Ge Lin, Fangzhen |
| author_facet | Wang, Haozhe Que, Haoran Xu, Qixin Liu, Minghao Zhou, Wangchunshu Feng, Jiazhan Zhong, Wanjun Ye, Wei Yang, Tong Huang, Wenhao Zhang, Ge Lin, Fangzhen |
| contents | While the ``deep reasoning'' paradigm has spurred significant advances in verifiable domains like mathematics, its application to open-ended, creative generation remains a critical challenge. The two dominant methods for instilling reasoning -- reinforcement learning (RL) and instruction distillation -- falter in this area; RL struggles with the absence of clear reward signals and high-quality reward models, while distillation is prohibitively expensive and capped by the teacher model's capabilities. To overcome these limitations, we introduce REverse-Engineered Reasoning (REER), a new paradigm that fundamentally shifts the approach. Instead of building a reasoning process ``forwards'' through trial-and-error or imitation, REER works ``backwards'' from known-good solutions to computationally discover the latent, step-by-step deep reasoning process that could have produced them. Using this scalable, gradient-free approach, we curate and open-source DeepWriting-20K, a large-scale dataset of 20,000 deep reasoning trajectories for open-ended tasks. Our model, DeepWriter-8B, trained on this data, not only surpasses strong open-source baselines but also achieves performance competitive with, and at times superior to, leading proprietary models like GPT-4o and Claude 3.5. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_06160 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Reverse-Engineered Reasoning for Open-Ended Generation Wang, Haozhe Que, Haoran Xu, Qixin Liu, Minghao Zhou, Wangchunshu Feng, Jiazhan Zhong, Wanjun Ye, Wei Yang, Tong Huang, Wenhao Zhang, Ge Lin, Fangzhen Artificial Intelligence Computation and Language While the ``deep reasoning'' paradigm has spurred significant advances in verifiable domains like mathematics, its application to open-ended, creative generation remains a critical challenge. The two dominant methods for instilling reasoning -- reinforcement learning (RL) and instruction distillation -- falter in this area; RL struggles with the absence of clear reward signals and high-quality reward models, while distillation is prohibitively expensive and capped by the teacher model's capabilities. To overcome these limitations, we introduce REverse-Engineered Reasoning (REER), a new paradigm that fundamentally shifts the approach. Instead of building a reasoning process ``forwards'' through trial-and-error or imitation, REER works ``backwards'' from known-good solutions to computationally discover the latent, step-by-step deep reasoning process that could have produced them. Using this scalable, gradient-free approach, we curate and open-source DeepWriting-20K, a large-scale dataset of 20,000 deep reasoning trajectories for open-ended tasks. Our model, DeepWriter-8B, trained on this data, not only surpasses strong open-source baselines but also achieves performance competitive with, and at times superior to, leading proprietary models like GPT-4o and Claude 3.5. |
| title | Reverse-Engineered Reasoning for Open-Ended Generation |
| topic | Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2509.06160 |