Cover Image

Optimizing Anytime Reasoning via Budget Relative Policy Optimization

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Qi, Penghui, Liu, Zichen, Pang, Tianyu, Du, Chao, Lee, Wee Sun, Lin, Min
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Defeating the Training-Inference Mismatch via FP16
by: Qi, Penghui, et al.
Published: (2025)

Understanding R1-Zero-Like Training: A Critical Perspective
by: Liu, Zichen, et al.
Published: (2025)

Sample-Efficient Alignment for LLMs
by: Liu, Zichen, et al.
Published: (2024)

Variational Reasoning for Language Models
by: Zhou, Xiangxin, et al.
Published: (2025)

SPIRAL: Self-Play on Zero-Sum Games Incentivizes Reasoning via Multi-Agent Multi-Turn Reinforcement Learning
by: Liu, Bo, et al.
Published: (2025)

Locality Sensitive Sparse Encoding for Learning World Models Online
by: Liu, Zichen, et al.
Published: (2024)

Language Models Can Learn from Verbal Feedback Without Scalar Rewards
by: Luo, Renjie, et al.
Published: (2025)

LongSpec: Long-Context Lossless Speculative Decoding with Efficient Drafting and Verification
by: Yang, Penghui, et al.
Published: (2025)

Continual Reinforcement Learning by Planning with Online World Models
by: Liu, Zichen, et al.
Published: (2025)

Graph Diffusion Policy Optimization
by: Liu, Yijing, et al.
Published: (2024)

Towards Improved Preference Optimization Pipeline: from Data Generation to Budget-Controlled Regularization
by: Chen, Zhuotong, et al.
Published: (2024)

A Closer Look at Machine Unlearning for Large Language Models
by: Yuan, Xiaojian, et al.
Published: (2024)

Improving Your Model Ranking on Chatbot Arena by Vote Rigging
by: Min, Rui, et al.
Published: (2025)

LightTransfer: Your Long-Context LLM is Secretly a Hybrid Model with Effortless Adaptation
by: Zhang, Xuan, et al.
Published: (2024)

When Attention Sink Emerges in Language Models: An Empirical View
by: Gu, Xiangming, et al.
Published: (2024)

Regressing the Relative Future: Efficient Policy Optimization for Multi-turn RLHF
by: Gao, Zhaolin, et al.
Published: (2024)

Scaf-GRPO: Scaffolded Group Relative Policy Optimization for Enhancing LLM Reasoning
by: Zhang, Xichen, et al.
Published: (2025)

Klear-Reasoner: Advancing Reasoning Capability via Gradient-Preserving Clipping Policy Optimization
by: Su, Zhenpeng, et al.
Published: (2025)

Knapsack RL: Unlocking Exploration of LLMs via Optimizing Budget Allocation
by: Li, Ziniu, et al.
Published: (2025)

Scalable Token-Level Hallucination Detection in Large Language Models
by: Min, Rui, et al.
Published: (2026)

Pyramid MoA: A Probabilistic Framework for Cost-Optimized Anytime Inference
by: Khaled, Arindam
Published: (2026)

Cheating Automatic LLM Benchmarks: Null Models Achieve High Win Rates
by: Zheng, Xiaosen, et al.
Published: (2024)

Improved Few-Shot Jailbreaking Can Circumvent Aligned Language Models and Their Defenses
by: Zheng, Xiaosen, et al.
Published: (2024)

Fibration Policy Optimization
by: Li, Chang, et al.
Published: (2026)

Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMs
by: Zhang, Xuan, et al.
Published: (2024)

Scaling up Masked Diffusion Models on Text
by: Nie, Shen, et al.
Published: (2024)

Slow-Fast Policy Optimization: Reposition-Before-Update for LLM Reasoning
by: Wang, Ziyan, et al.
Published: (2025)

Stepwise Guided Policy Optimization: Coloring your Incorrect Reasoning in GRPO
by: Chen, Peter, et al.
Published: (2025)

Purifying Large Language Models by Ensembling a Small Language Model
by: Li, Tianlin, et al.
Published: (2024)

GTPO: Stabilizing Group Relative Policy Optimization via Gradient and Entropy Control
by: Simoni, Marco, et al.
Published: (2025)

Dataset Reset Policy Optimization for RLHF
by: Chang, Jonathan D., et al.
Published: (2024)

Group Sequence Policy Optimization
by: Zheng, Chujie, et al.
Published: (2025)

HAPO: Training Language Models to Reason Concisely via History-Aware Policy Optimization
by: Huang, Chengyu, et al.
Published: (2025)

Soft Adaptive Policy Optimization
by: Gao, Chang, et al.
Published: (2025)

Orthogonal Finetuning for Direct Preference Optimization
by: Yang, Chenxu, et al.
Published: (2024)

Personalized Group Relative Policy Optimization for Heterogenous Preference Alignment
by: Wang, Jialu, et al.
Published: (2026)

RePO: Replay-Enhanced Policy Optimization
by: Li, Siheng, et al.
Published: (2025)

Demystifying the Slash Pattern in Attention: The Role of RoPE
by: Cheng, Yuan, et al.
Published: (2026)

BAPO: Stabilizing Off-Policy Reinforcement Learning for LLMs via Balanced Policy Optimization with Adaptive Clipping
by: Xi, Zhiheng, et al.
Published: (2025)

PrAg-PO: Prompt Augmented Policy Optimization for Robust and Diverse Mathematical Reasoning
by: Lu, Wenquan, et al.
Published: (2026)