Adaptive Test-Time Reasoning via Reward-Guided Dual-Phase Search
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912615786086400 |
|---|---|
| author | Cui, Yingqian Dai, Zhenwei He, Pengfei He, Bing Liu, Hui Tang, Xianfeng Zeng, Jingying Wang, Suhang Xing, Yue Tang, Jiliang Dumoulin, Benoit |
| author_facet | Cui, Yingqian Dai, Zhenwei He, Pengfei He, Bing Liu, Hui Tang, Xianfeng Zeng, Jingying Wang, Suhang Xing, Yue Tang, Jiliang Dumoulin, Benoit |
| contents | Large Language Models (LLMs) have achieved significant advances in reasoning tasks. A key approach is tree-based search with verifiers, which expand candidate reasoning paths and use reward models to guide pruning and selection. Although effective in improving accuracy, these methods are not optimal in terms of efficiency: they perform simple decomposition on the reasoning process, but ignore the planning-execution nature of tasks such as math reasoning or code generation. This results in inefficient exploration of reasoning process. To address this, we propose a dual-phase test-time scaling framework that explicitly separates reasoning into planning and execution, and performs search over the two phases individually. Specifically, we decompose reasoning trajectories and develop reward models for each phase, enabling the search to explore and prune plans and executions separately. We further introduce a dynamic budget allocation mechanism that adaptively redistributes sampling effort based on reward feedback, allowing early stopping on confident steps and reallocation of computation to more challenging parts of the reasoning process. Experiments on both mathematical reasoning and code generation benchmarks demonstrate that our approach consistently improves accuracy while reducing redundant computation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_25420 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Adaptive Test-Time Reasoning via Reward-Guided Dual-Phase Search Cui, Yingqian Dai, Zhenwei He, Pengfei He, Bing Liu, Hui Tang, Xianfeng Zeng, Jingying Wang, Suhang Xing, Yue Tang, Jiliang Dumoulin, Benoit Artificial Intelligence Computation and Language Machine Learning Large Language Models (LLMs) have achieved significant advances in reasoning tasks. A key approach is tree-based search with verifiers, which expand candidate reasoning paths and use reward models to guide pruning and selection. Although effective in improving accuracy, these methods are not optimal in terms of efficiency: they perform simple decomposition on the reasoning process, but ignore the planning-execution nature of tasks such as math reasoning or code generation. This results in inefficient exploration of reasoning process. To address this, we propose a dual-phase test-time scaling framework that explicitly separates reasoning into planning and execution, and performs search over the two phases individually. Specifically, we decompose reasoning trajectories and develop reward models for each phase, enabling the search to explore and prune plans and executions separately. We further introduce a dynamic budget allocation mechanism that adaptively redistributes sampling effort based on reward feedback, allowing early stopping on confident steps and reallocation of computation to more challenging parts of the reasoning process. Experiments on both mathematical reasoning and code generation benchmarks demonstrate that our approach consistently improves accuracy while reducing redundant computation. |
| title | Adaptive Test-Time Reasoning via Reward-Guided Dual-Phase Search |
| topic | Artificial Intelligence Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2509.25420 |