Cover Image

Buffer Matters: Unleashing the Power of Off-Policy Reinforcement Learning in Large Language Model Reasoning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wan, Xu, Wang, Yansheng, Huang, Wenqi, Sun, Mingyang
Format:	Preprint
Published:	2026
Subjects:	Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Cooper: Co-Optimizing Policy and Reward Models in Reinforcement Learning for Large Language Models
by: Hong, Haitao, et al.
Published: (2025)

AdapThink: Adaptive Thinking Preferences for Reasoning Language Model
by: Wan, Xu, et al.
Published: (2025)

Mitigating Overthinking in Large Reasoning Models via Difficulty-aware Reinforcement Learning
by: Wan, Qian, et al.
Published: (2026)

MLCopilot: Unleashing the Power of Large Language Models in Solving Machine Learning Tasks
by: Zhang, Lei, et al.
Published: (2023)

Large Language Model Post-Training: A Unified View of Off-Policy and On-Policy Learning
by: Zhao, Shiwan, et al.
Published: (2026)

Exploringand Unleashing the Power of Large Language Models in CI/CD Configuration Translation
by: Wang, Chong, et al.
Published: (2025)

CogGPT: Unleashing the Power of Cognitive Dynamics on Large Language Models
by: Lv, Yaojia, et al.
Published: (2024)

Resource-Efficient Reinforcement for Reasoning Large Language Models via Dynamic One-Shot Policy Refinement
by: Zhang, Yunjian, et al.
Published: (2026)

Score-Based Diffusion Policy Compatible with Reinforcement Learning via Optimal Transport
by: Sun, Mingyang, et al.
Published: (2025)

Exploring and Unleashing the Power of Large Language Models in Automated Code Translation
by: Yang, Zhen, et al.
Published: (2024)

Towards Off-Policy Reinforcement Learning for Ranking Policies with Human Feedback
by: Xiao, Teng, et al.
Published: (2024)

m1: Unleash the Potential of Test-Time Scaling for Medical Reasoning with Large Language Models
by: Huang, Xiaoke, et al.
Published: (2025)

MindOmni: Unleashing Reasoning Generation in Vision Language Models with RGPO
by: Xiao, Yicheng, et al.
Published: (2025)

SATURN: SAT-based Reinforcement Learning to Unleash LLMs Reasoning
by: Liu, Huanyu, et al.
Published: (2025)

Think in Games: Learning to Reason in Games via Reinforcement Learning with Large Language Models
by: Liao, Yi, et al.
Published: (2025)

EvidenceMap: Learning Evidence Analysis to Unleash the Power of Small Language Models for Biomedical Question Answering
by: Zong, Chang, et al.
Published: (2025)

Think Twice, Act Once: A Co-Evolution Framework of LLM and RL for Large-Scale Decision Making
by: Wan, Xu, et al.
Published: (2025)

Effective Reinforcement Learning for Reasoning in Language Models
by: Huang, Lianghuan, et al.
Published: (2025)

SAMG: Offline-to-Online Reinforcement Learning via State-Action-Conditional Offline Model Guidance
by: Zhang, Liyu, et al.
Published: (2024)

Learning to Reason under Off-Policy Guidance
by: Yan, Jianhao, et al.
Published: (2025)

Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learning
by: Xie, Tian, et al.
Published: (2025)

Premise Order Matters in Reasoning with Large Language Models
by: Chen, Xinyun, et al.
Published: (2024)

Reinforcement Fine-Tuning Powers Reasoning Capability of Multimodal Large Language Models
by: Sun, Haoyuan, et al.
Published: (2025)

Nested-ReFT: Efficient Reinforcement Learning for Large Language Model Fine-Tuning via Off-Policy Rollouts
by: Heuillet, Maxime, et al.
Published: (2025)

Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Models
by: Xu, Fengli, et al.
Published: (2025)

Route-and-Reason: Scaling Large Language Model Reasoning with Reinforced Model Router
by: Shao, Chenyang, et al.
Published: (2025)

Training Large Language Models to Reason via EM Policy Gradient
by: Xu, Tianbing
Published: (2025)

On Predictability of Reinforcement Learning Dynamics for Large Language Models
by: Cai, Yuchen, et al.
Published: (2025)

MedAdapter: Efficient Test-Time Adaptation of Large Language Models towards Medical Reasoning
by: Shi, Wenqi, et al.
Published: (2024)

Large Language Model as a Policy Teacher for Training Reinforcement Learning Agents
by: Zhou, Zihao, et al.
Published: (2023)

AR$^2$: Adversarial Reinforcement Learning for Abstract Reasoning in Large Language Models
by: Yeh, Cheng-Kai, et al.
Published: (2025)

Learning to Check: Unleashing Potentials for Self-Correction in Large Language Models
by: Zhang, Che, et al.
Published: (2024)

Rationale-Grounded In-Context Learning for Time Series Reasoning with Multimodal Large Language Models
by: Liu, Qingxiang, et al.
Published: (2026)

Off-policy Reinforcement Learning with Model-based Exploration Augmentation
by: Wang, Likun, et al.
Published: (2025)

Think Before Recommend: Unleashing the Latent Reasoning Power for Sequential Recommendation
by: Tang, Jiakai, et al.
Published: (2025)

SrSv: Integrating Sequential Rollouts with Sequential Value Estimation for Multi-agent Reinforcement Learning
by: Wan, Xu, et al.
Published: (2025)

Training Large Language Models for Reasoning through Reverse Curriculum Reinforcement Learning
by: Xi, Zhiheng, et al.
Published: (2024)

Low-Rank Adaptation for Critic Learning in Off-Policy Reinforcement Learning
by: Zhuang, Yuan, et al.
Published: (2026)

Scaling Off-Policy Reinforcement Learning with Batch and Weight Normalization
by: Palenicek, Daniel, et al.
Published: (2025)

Reinforced MLLM: A Survey on RL-Based Reasoning in Multimodal Large Language Models
by: Zhou, Guanghao, et al.
Published: (2025)