Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2601.05870 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914243417210880 |
|---|---|
| author | Deng, Huilin Luo, Hongchen Zhu, Yue Li, Long Chen, Zhuoyue Zhao, Xinghao Li, Ming Zhang, Jihai Wang, Mengchang Cao, Yang Kang, Yu |
| author_facet | Deng, Huilin Luo, Hongchen Zhu, Yue Li, Long Chen, Zhuoyue Zhao, Xinghao Li, Ming Zhang, Jihai Wang, Mengchang Cao, Yang Kang, Yu |
| contents | Recent advances in Reinforcement Learning with Verifiable Rewards (RLVR) for Large Language Model (LLM) reasoning have been hindered by a persistent challenge: exploration collapse. The semantic homogeneity of random rollouts often traps models in narrow, over-optimized behaviors. While existing methods leverage policy entropy to encourage exploration, they face inherent limitations. Global entropy regularization is susceptible to reward hacking, which can induce meaningless verbosity, whereas local token-selective updates struggle with the strong inductive bias of pre-trained models. To address this, we propose Latent Policy Optimization via Iterative Information Bottleneck (IIB-LPO), a novel approach that shifts exploration from statistical perturbation of token distributions to topological branching of reasoning trajectories. IIB-LPO triggers latent branching at high-entropy states to diversify reasoning paths and employs the Information Bottleneck principle both as a trajectory filter and a self-reward mechanism, ensuring concise and informative exploration. Empirical results across four mathematical reasoning benchmarks demonstrate that IIB-LPO achieves state-of-the-art performance, surpassing prior methods by margins of up to 5.3% in accuracy and 7.4% in diversity metrics. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_05870 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | IIB-LPO: Latent Policy Optimization via Iterative Information Bottleneck Deng, Huilin Luo, Hongchen Zhu, Yue Li, Long Chen, Zhuoyue Zhao, Xinghao Li, Ming Zhang, Jihai Wang, Mengchang Cao, Yang Kang, Yu Machine Learning Artificial Intelligence Recent advances in Reinforcement Learning with Verifiable Rewards (RLVR) for Large Language Model (LLM) reasoning have been hindered by a persistent challenge: exploration collapse. The semantic homogeneity of random rollouts often traps models in narrow, over-optimized behaviors. While existing methods leverage policy entropy to encourage exploration, they face inherent limitations. Global entropy regularization is susceptible to reward hacking, which can induce meaningless verbosity, whereas local token-selective updates struggle with the strong inductive bias of pre-trained models. To address this, we propose Latent Policy Optimization via Iterative Information Bottleneck (IIB-LPO), a novel approach that shifts exploration from statistical perturbation of token distributions to topological branching of reasoning trajectories. IIB-LPO triggers latent branching at high-entropy states to diversify reasoning paths and employs the Information Bottleneck principle both as a trajectory filter and a self-reward mechanism, ensuring concise and informative exploration. Empirical results across four mathematical reasoning benchmarks demonstrate that IIB-LPO achieves state-of-the-art performance, surpassing prior methods by margins of up to 5.3% in accuracy and 7.4% in diversity metrics. |
| title | IIB-LPO: Latent Policy Optimization via Iterative Information Bottleneck |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2601.05870 |