Small Generalizable Prompt Predictive Models Can Steer Efficient RL Post-Training of Large Reasoning Models
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913131505123328 |
|---|---|
| author | Qu, Yun Wang, Qi Mao, Yixiu Zou, Heming Jiang, Yuhang Liu, Weijie Bai, Clive Yang, Kai Chen, Yangkun Yang, Saiyong Ji, Xiangyang |
| author_facet | Qu, Yun Wang, Qi Mao, Yixiu Zou, Heming Jiang, Yuhang Liu, Weijie Bai, Clive Yang, Kai Chen, Yangkun Yang, Saiyong Ji, Xiangyang |
| contents | Reinforcement learning enhances the reasoning capabilities of large language models but often involves high computational costs due to rollout-intensive optimization. Online prompt selection presents a plausible solution by prioritizing informative prompts to improve training efficiency. However, current methods either depend on costly, exact evaluations or construct prompt-specific predictive models lacking generalization across prompts. This study introduces Generalizable Predictive Prompt Selection (GPS), which performs Bayesian inference towards prompt difficulty using a lightweight generative model trained on the shared optimization history. Intermediate-difficulty prioritization and history-anchored diversity are incorporated into the batch acquisition principle to select informative prompt batches. The small predictive model also generalizes at test-time for efficient computational allocation. Experiments across varied reasoning benchmarks indicate GPS's substantial improvements in training efficiency, final performance, and test-time efficiency over superior baseline methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_01970 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Small Generalizable Prompt Predictive Models Can Steer Efficient RL Post-Training of Large Reasoning Models Qu, Yun Wang, Qi Mao, Yixiu Zou, Heming Jiang, Yuhang Liu, Weijie Bai, Clive Yang, Kai Chen, Yangkun Yang, Saiyong Ji, Xiangyang Artificial Intelligence Machine Learning Reinforcement learning enhances the reasoning capabilities of large language models but often involves high computational costs due to rollout-intensive optimization. Online prompt selection presents a plausible solution by prioritizing informative prompts to improve training efficiency. However, current methods either depend on costly, exact evaluations or construct prompt-specific predictive models lacking generalization across prompts. This study introduces Generalizable Predictive Prompt Selection (GPS), which performs Bayesian inference towards prompt difficulty using a lightweight generative model trained on the shared optimization history. Intermediate-difficulty prioritization and history-anchored diversity are incorporated into the batch acquisition principle to select informative prompt batches. The small predictive model also generalizes at test-time for efficient computational allocation. Experiments across varied reasoning benchmarks indicate GPS's substantial improvements in training efficiency, final performance, and test-time efficiency over superior baseline methods. |
| title | Small Generalizable Prompt Predictive Models Can Steer Efficient RL Post-Training of Large Reasoning Models |
| topic | Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2602.01970 |