Cover Image

Improving RL Exploration for LLM Reasoning through Retrospective Replay

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Dou, Shihan, Wu, Muling, Xu, Jingwen, Zheng, Rui, Gui, Tao, Zhang, Qi, Huang, Xuanjing
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Advancing Translation Preference Modeling with RLHF: A Step Towards Cost-Effective Solution
by: Xu, Nuo, et al.
Published: (2024)

MetaRM: Shifted Distributions Alignment via Meta-Learning
by: Dou, Shihan, et al.
Published: (2024)

Subspace Defense: Discarding Adversarial Perturbations by Learning a Subspace for Clean Signals
by: Zheng, Rui, et al.
Published: (2024)

EVPO: Explained Variance Policy Optimization for Adaptive Critic Utilization in LLM Post-Training
by: Pan, Chengjun, et al.
Published: (2026)

AgentGym-RL: Training LLM Agents for Long-Horizon Decision Making through Multi-Turn Reinforcement Learning
by: Xi, Zhiheng, et al.
Published: (2025)

Training Large Language Models for Reasoning through Reverse Curriculum Reinforcement Learning
by: Xi, Zhiheng, et al.
Published: (2024)

DSDR: Dual-Scale Diversity Regularization for Exploration in LLM Reasoning
by: Wan, Zhongwei, et al.
Published: (2026)

TemplateRL: Structured Template-Guided Reinforcement Learning for LLM Reasoning
by: Wu, Jinyang, et al.
Published: (2025)

Enhancing LLM Reasoning via Critique Models with Test-Time and Training-Time Supervision
by: Xi, Zhiheng, et al.
Published: (2024)

Promoting Data and Model Privacy in Federated Learning through Quantized LoRA
by: Zhu, JianHao, et al.
Published: (2024)

DFPO: Scaling Value Modeling via Distributional Flow towards Robust and Generalizable LLM Post-Training
by: Zhu, Dingwei, et al.
Published: (2026)

Advancing Parameter Efficiency in Fine-tuning via Representation Editing
by: Wu, Muling, et al.
Published: (2024)

VRPO: Rethinking Value Modeling for Robust RL Training under Noisy Supervision
by: Zhu, Dingwei, et al.
Published: (2025)

Beyond Markovian: Reflective Exploration via Bayes-Adaptive RL for LLM Reasoning
by: Zhang, Shenao, et al.
Published: (2025)

Prioritized Replay for RL Post-training
by: Fatemi, Mehdi
Published: (2026)

Two Minds Better Than One: Collaborative Reward Modeling for LLM Alignment
by: Zhang, Jiazheng, et al.
Published: (2025)

Internalizing LLM Reasoning via Discovery and Replay of Latent Actions
by: Shi, Zhenning, et al.
Published: (2026)

TinyV: Reducing False Negatives in Verification Improves RL for LLM Reasoning
by: Xu, Zhangchen, et al.
Published: (2025)

Attention as a Compass: Efficient Exploration for Process-Supervised RL in Reasoning Models
by: Liu, Runze, et al.
Published: (2025)

Mitigating Tail Narrowing in LLM Self-Improvement via Socratic-Guided Sampling
by: Ding, Yiwen, et al.
Published: (2024)

Part I: Tricks or Traps? A Deep Dive into RL for LLM Reasoning
by: Liu, Zihe, et al.
Published: (2025)

Semantic-Space Exploration and Exploitation in RLVR for LLM Reasoning
by: Huang, Fanding, et al.
Published: (2025)

Enhancing Model Privacy in Federated Learning with Random Masking and Quantization
by: Xu, Zhibo, et al.
Published: (2025)

Entropy Polarity in Reinforcement Fine-Tuning: Direction, Asymmetry, and Control
by: Zhang, Jiazheng, et al.
Published: (2026)

SpikeCLIP: A Contrastive Language-Image Pretrained Spiking Neural Network
by: Lv, Changze, et al.
Published: (2023)

On Designing Effective RL Reward at Training Time for LLM Reasoning
by: Gao, Jiaxuan, et al.
Published: (2024)

Outcome-based Exploration for LLM Reasoning
by: Song, Yuda, et al.
Published: (2025)

PFDial: A Structured Dialogue Instruction Fine-tuning Method Based on UML Flowcharts
by: Zhang, Ming, et al.
Published: (2025)

Sample-efficient LLM Optimization with Reset Replay
by: Liu, Zichuan, et al.
Published: (2025)

AgentPRM: Process Reward Models for LLM Agents via Step-Wise Promise and Progress
by: Xi, Zhiheng, et al.
Published: (2025)

FlowRL: Matching Reward Distributions for LLM Reasoning
by: Zhu, Xuekai, et al.
Published: (2025)

Interpreting and Controlling LLM Reasoning through Integrated Policy Gradient
by: Li, Changming, et al.
Published: (2026)

Save the Good Prefix: Precise Error Penalization via Process-Supervised RL to Enhance LLM Reasoning
by: Liu, Haolin, et al.
Published: (2026)

Reward Hacking in the Era of Large Models: Mechanisms, Emergent Misalignment, Challenges
by: Wang, Xiaohua, et al.
Published: (2026)

Counteracting Matthew Effect in Self-Improvement of LVLMs through Head-Tail Re-balancing
by: Guo, Xin, et al.
Published: (2025)

Steering LLMs via Scalable Interactive Oversight
by: Zhou, Enyu, et al.
Published: (2026)

UPLex: Fine-Grained Personality Control in Large Language Models via Unsupervised Lexical Modulation
by: Li, Tianlong, et al.
Published: (2023)

Pre-Trained Policy Discriminators are General Reward Models
by: Dou, Shihan, et al.
Published: (2025)

Enhancing LLM-based Search Agents via Contribution Weighted Group Relative Policy Optimization
by: Wang, Junzhe, et al.
Published: (2026)

BroRL: Scaling Reinforcement Learning via Broadened Exploration
by: Hu, Jian, et al.
Published: (2025)