Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2510.16449 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914102665805824 |
|---|---|
| author | Yu, Bin Wang, Xinming Lian, Shijie Li, Haotian Wu, Changti Hu, Ruina Wang, Bailing Wei, Yuliang Chen, Kai |
| author_facet | Yu, Bin Wang, Xinming Lian, Shijie Li, Haotian Wu, Changti Hu, Ruina Wang, Bailing Wei, Yuliang Chen, Kai |
| contents | Large language models (LLMs) have shown remarkable progress in complex reasoning tasks, largely enabled by test-time scaling (TTS) paradigms that allocate additional compute during inference. Among these, external TTS (particularly the Best-of-N selection paradigm) yields scalable performance improvements by selecting from multiple independently generated reasoning trajectories. However, this approach faces key limitations: (i) the high computational overhead of deploying process reward models, (ii) the underutilization of the LLM's intrinsic latent representations. We introduce TrajSelector, an efficient and effective Best-of-N framework that exploit the hidden states in the sampler LLM for process-level scoring. A lightweight verifier (with only 0.6B parameters) evaluates the quality of step-wise trajectory, and then aggregates these scores to identify the optimal reasoning trajectory. Our framework employs a fully data-driven, end-to-end training recipe that eliminates reliance on massive step-level annotations. Experiential results across five benchmarks demonstrate that TrajSelector delivers consistent performance gains. In Best-of-32 settings, it surpasses majority voting by 4.61% accuracy and outperforms existing process reward models by 4.31% to 12.21%, all while maintaining lower inference costs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_16449 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | TrajSelector: Harnessing Latent Representations for Efficient and Effective Best-of-N in Large Reasoning Model Yu, Bin Wang, Xinming Lian, Shijie Li, Haotian Wu, Changti Hu, Ruina Wang, Bailing Wei, Yuliang Chen, Kai Computation and Language Large language models (LLMs) have shown remarkable progress in complex reasoning tasks, largely enabled by test-time scaling (TTS) paradigms that allocate additional compute during inference. Among these, external TTS (particularly the Best-of-N selection paradigm) yields scalable performance improvements by selecting from multiple independently generated reasoning trajectories. However, this approach faces key limitations: (i) the high computational overhead of deploying process reward models, (ii) the underutilization of the LLM's intrinsic latent representations. We introduce TrajSelector, an efficient and effective Best-of-N framework that exploit the hidden states in the sampler LLM for process-level scoring. A lightweight verifier (with only 0.6B parameters) evaluates the quality of step-wise trajectory, and then aggregates these scores to identify the optimal reasoning trajectory. Our framework employs a fully data-driven, end-to-end training recipe that eliminates reliance on massive step-level annotations. Experiential results across five benchmarks demonstrate that TrajSelector delivers consistent performance gains. In Best-of-32 settings, it surpasses majority voting by 4.61% accuracy and outperforms existing process reward models by 4.31% to 12.21%, all while maintaining lower inference costs. |
| title | TrajSelector: Harnessing Latent Representations for Efficient and Effective Best-of-N in Large Reasoning Model |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2510.16449 |