HiPO: Hybrid Policy Optimization for Dynamic Reasoning in LLMs
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912662212837376 |
|---|---|
| author | Deng, Ken Zhan, Zizheng Xiang, Wen Zhu, Wenqiang Li, Weihao Xu, Jingxuan Peng, Tianhao Lei, Xinping Wu, Kun Yao, Yifan Huang, Haoyang Tang, Huaixi Lei, Kepeng Lai, Zhiyi Yu, Songwei Feng, Zongxian Gao, Zuchen Xie, Weihao Zhang, Chenchen Wu, Yanan Zhang, Yuanxing Huang, Lecheng Zhang, Yuqun Liu, Jie Zhang, Zhaoxiang Zhang, Haotian Chen, Bin Liu, Jiaheng |
| author_facet | Deng, Ken Zhan, Zizheng Xiang, Wen Zhu, Wenqiang Li, Weihao Xu, Jingxuan Peng, Tianhao Lei, Xinping Wu, Kun Yao, Yifan Huang, Haoyang Tang, Huaixi Lei, Kepeng Lai, Zhiyi Yu, Songwei Feng, Zongxian Gao, Zuchen Xie, Weihao Zhang, Chenchen Wu, Yanan Zhang, Yuanxing Huang, Lecheng Zhang, Yuqun Liu, Jie Zhang, Zhaoxiang Zhang, Haotian Chen, Bin Liu, Jiaheng |
| contents | Large Language Models (LLMs) increasingly rely on Chain-of-Thought (CoT) reasoning to improve accuracy on complex tasks. However, always generating lengthy reasoning traces is inefficient, leading to excessive token usage and higher inference costs. This paper introduces the Hybrid Policy Optimization (i.e., HiPO), a framework for adaptive reasoning control that enables LLMs to selectively decide when to engage in detailed reasoning (Think-on) and when to respond directly (Think-off). Specifically, HiPO combines a hybrid data pipelineproviding paired Think-on and Think-off responseswith a hybrid reinforcement learning reward system that balances accuracy and efficiency while avoiding over-reliance on detailed reasoning. Experiments across mathematics and coding benchmarks demonstrate that HiPO can substantially reduce token length while maintaining or improving accuracy. Finally, we hope HiPO a can be a principled approach for efficient adaptive reasoning, advancing the deployment of reasoning-oriented LLMs in real-world, resource-sensitive settings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_23967 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | HiPO: Hybrid Policy Optimization for Dynamic Reasoning in LLMs Deng, Ken Zhan, Zizheng Xiang, Wen Zhu, Wenqiang Li, Weihao Xu, Jingxuan Peng, Tianhao Lei, Xinping Wu, Kun Yao, Yifan Huang, Haoyang Tang, Huaixi Lei, Kepeng Lai, Zhiyi Yu, Songwei Feng, Zongxian Gao, Zuchen Xie, Weihao Zhang, Chenchen Wu, Yanan Zhang, Yuanxing Huang, Lecheng Zhang, Yuqun Liu, Jie Zhang, Zhaoxiang Zhang, Haotian Chen, Bin Liu, Jiaheng Computation and Language Large Language Models (LLMs) increasingly rely on Chain-of-Thought (CoT) reasoning to improve accuracy on complex tasks. However, always generating lengthy reasoning traces is inefficient, leading to excessive token usage and higher inference costs. This paper introduces the Hybrid Policy Optimization (i.e., HiPO), a framework for adaptive reasoning control that enables LLMs to selectively decide when to engage in detailed reasoning (Think-on) and when to respond directly (Think-off). Specifically, HiPO combines a hybrid data pipelineproviding paired Think-on and Think-off responseswith a hybrid reinforcement learning reward system that balances accuracy and efficiency while avoiding over-reliance on detailed reasoning. Experiments across mathematics and coding benchmarks demonstrate that HiPO can substantially reduce token length while maintaining or improving accuracy. Finally, we hope HiPO a can be a principled approach for efficient adaptive reasoning, advancing the deployment of reasoning-oriented LLMs in real-world, resource-sensitive settings. |
| title | HiPO: Hybrid Policy Optimization for Dynamic Reasoning in LLMs |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2509.23967 |