Cover Image

Unlocking Exploration in RLVR: Uncertainty-aware Advantage Shaping for Deeper Reasoning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Xie, Can, Pan, Ruotong, Wu, Xiangyu, Zhang, Yunfei, Fu, Jiayi, Gao, Tingting, Zhou, Guorui
Format:	Preprint
Published:	2025
Subjects:	Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Quantile Advantage Estimation: Stabilizing RLVR for LLM Reasoning
by: Wu, Junkang, et al.
Published: (2025)

Depth-Breadth Synergy in RLVR: Unlocking LLM Reasoning Gains with Adaptive Exploration
by: Yang, Zhicheng, et al.
Published: (2025)

SAGE: Shaping Anchors for Guided Exploration in RLVR of LLMs
by: Lee, Chanuk, et al.
Published: (2026)

Asymmetric Advantage Modulation Calibrates Entropy Dynamics in RLVR
by: Gu, Hengrui, et al.
Published: (2026)

Controllable Exploration in Hybrid-Policy RLVR for Multi-Modal Reasoning
by: Huang, Zhuoxu, et al.
Published: (2026)

EFRame: Deeper Reasoning via Exploration-Filter-Replay Reinforcement Learning Framework
by: Wang, Chen, et al.
Published: (2025)

SmartSwitch: Advancing LLM Reasoning by Overcoming Underthinking via Promoting Deeper Thought Exploration
by: Zhang, Xichen, et al.
Published: (2025)

DIVA-GRPO: Enhancing Multimodal Reasoning through Difficulty-Adaptive Variant Advantage
by: Gao, Haowen, et al.
Published: (2026)

How You Begin is How You Reason: Driving Exploration in RLVR via Prefix-Tuned Priors
by: Xu, Yifan, et al.
Published: (2026)

On the Direction of RLVR Updates for LLM Reasoning: Identification and Exploitation
by: Huang, Kexin, et al.
Published: (2026)

Negative Advantages Is a Double-Edged Sword: Calibrating advantages in GRPO for Search Agents
by: Wu, Jiayi, et al.
Published: (2026)

Beyond High-Entropy Exploration: Correctness-Aware Low-Entropy Segment-Based Advantage Shaping for Reasoning LLMs
by: Chen, Xinzhu, et al.
Published: (2025)

Why Distillation can Outperform Zero-RL: The Role of Flexible Reasoning
by: Hu, Xiao, et al.
Published: (2025)

From Efficiency to Adaptivity: A Deeper Look at Adaptive Reasoning in Large Language Models
by: Wu, Chao, et al.
Published: (2025)

Detecting RLVR Training Data via Structural Convergence of Reasoning
by: Zhang, Hongbo, et al.
Published: (2026)

Learn More with Less: Uncertainty Consistency Guided Query Selection for RLVR
by: Yi, Hao, et al.
Published: (2026)

Unlocking Reasoning Capabilities in LLMs via Reinforcement Learning Exploration
by: Deng, Wenhao, et al.
Published: (2025)

Unlocking Feature Visualization for Deeper Networks with MAgnitude Constrained Optimization
by: Fel, Thomas, et al.
Published: (2023)

Recycling Failures: Salvaging Exploration in RLVR via Fine-Grained Off-Policy Guidance
by: Ren, Yanwei, et al.
Published: (2026)

Nudging Beyond the Comfort Zone: Efficient Strategy-Guided Exploration for RLVR
by: Lee, Chanuk, et al.
Published: (2026)

Incentivizing In-depth Reasoning over Long Contexts with Process Advantage Shaping
by: Peng, Miao, et al.
Published: (2026)

Towards Real-world Human Behavior Simulation: Benchmarking Large Language Models on Long-horizon, Cross-scenario, Heterogeneous Behavior Traces
by: Chen, Jiawei, et al.
Published: (2026)

Risk-aware Meta-level Decision Making for Exploration Under Uncertainty
by: Ott, Joshua, et al.
Published: (2022)

Evolving Deeper LLM Thinking
by: Lee, Kuang-Huei, et al.
Published: (2025)

SeePhys Pro: Diagnosing Modality Transfer and Blind-Training Effects in Multimodal RLVR for Physics Reasoning
by: Xiang, Kun, et al.
Published: (2026)

GUI-explorer: Autonomous Exploration and Mining of Transition-aware Knowledge for GUI Agent
by: Xie, Bin, et al.
Published: (2025)

SWE-Fuse: Empowering Software Agents via Issue-free Trajectory Learning and Entropy-aware RLVR Training
by: Wen, Xin-Cheng, et al.
Published: (2026)

JudgeRLVR: Judge First, Generate Second for Efficient Reasoning
by: Duo, Jiangshan, et al.
Published: (2026)

MEML-GRPO: Heterogeneous Multi-Expert Mutual Learning for RLVR Advancement
by: Jia, Weitao, et al.
Published: (2025)

Rethinking Entropy Interventions in RLVR: An Entropy Change Perspective
by: Hao, Zhezheng, et al.
Published: (2025)

SHAPE: Stage-aware Hierarchical Advantage via Potential Estimation for LLM Reasoning
by: Ai, Zhengyang, et al.
Published: (2026)

Accelerating RL for LLM Reasoning with Optimal Advantage Regression
by: Brantley, Kianté, et al.
Published: (2025)

Generalization of RLVR Using Causal Reasoning as a Testbed
by: Lu, Brian, et al.
Published: (2025)

Unlocking the Power of LLM Uncertainty for Active In-Context Example Selection
by: Huang, Hsiu-Yuan, et al.
Published: (2024)

Does RLVR Extend Reasoning Boundaries? Investigating Capability Expansion in Vision-Language Models
by: Shen, Minghe, et al.
Published: (2025)

Beyond Classification Accuracy: Neural-MedBench and the Need for Deeper Reasoning Benchmarks
by: Jing, Miao, et al.
Published: (2025)

Decoupling Knowledge and Reasoning in LLMs: An Exploration Using Cognitive Dual-System Theory
by: Yang, Mutian, et al.
Published: (2025)

Monitorability as a Free Gift: How RLVR Spontaneously Aligns Reasoning
by: Xiong, Zidi, et al.
Published: (2026)

SPADER: Step-wise Peer Advantage with Diversity-Aware Exploration Rewards for Multi-Answer Question Answering
by: Shi, Qiming, et al.
Published: (2026)

Exploration vs Exploitation: Rethinking RLVR through Clipping, Entropy, and Spurious Reward
by: Chen, Peter, et al.
Published: (2025)