Cover Image

ADORA: Training Reasoning Models with Dynamic Advantage Estimation on Reinforcement Learning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Ren, Qingnan, Huang, Shiting, Fang, Zhen, Chen, Zehui, Chen, Lin, Li, Lijun, Zhao, Feng
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Internalizing Meta-Experience into Memory for Guided Reinforcement Learning in Large Language Models
by: Huang, Shiting, et al.
Published: (2026)

An Advantage-based Optimization Method for Reinforcement Learning in Large Action Space
by: Lin, Hai, et al.
Published: (2024)

Quantile Advantage Estimation: Stabilizing RLVR for LLM Reasoning
by: Wu, Junkang, et al.
Published: (2025)

Advantage-Guided Diffusion for Model-Based Reinforcement Learning
by: Foffano, Daniele, et al.
Published: (2026)

Segmental Advantage Estimation: Enhancing PPO for Long-Context LLM Training
by: Gong, Xue, et al.
Published: (2026)

ACC: Compiling Agent Trajectories for Long-Context Training
by: Su, Qisheng, et al.
Published: (2026)

Reinforcement Learning for Reasoning in Large Language Models with One Training Example
by: Wang, Yiping, et al.
Published: (2025)

How to Allocate, How to Learn? Dynamic Rollout Allocation and Advantage Modulation for Policy Optimization
by: Fang, Yangyi, et al.
Published: (2026)

Relational Learning in Pre-Trained Models: A Theory from Hypergraph Recovery Perspective
by: Chen, Yang, et al.
Published: (2024)

RLVR-World: Training World Models with Reinforcement Learning
by: Wu, Jialong, et al.
Published: (2025)

Can One Domain Help Others? A Data-Centric Study on Multi-Domain Reasoning via Reinforcement Learning
by: Li, Yu, et al.
Published: (2025)

Accelerating RL for LLM Reasoning with Optimal Advantage Regression
by: Brantley, Kianté, et al.
Published: (2025)

TimeMaster: Training Time-Series Multimodal LLMs to Reason via Reinforcement Learning
by: Zhang, Junru, et al.
Published: (2025)

A2PO: Towards Effective Offline Reinforcement Learning from an Advantage-aware Perspective
by: Qing, Yunpeng, et al.
Published: (2024)

OPD+: Rethinking the Advantage Design for On-Policy Distillation
by: Zhao, Hanyang, et al.
Published: (2026)

On Predictability of Reinforcement Learning Dynamics for Large Language Models
by: Cai, Yuchen, et al.
Published: (2025)

DiFFPO: Training Diffusion LLMs to Reason Fast and Furious via Reinforcement Learning
by: Zhao, Hanyang, et al.
Published: (2025)

Memory-Based Advantage Shaping for LLM-Guided Reinforcement Learning
by: Nourzad, Narjes, et al.
Published: (2026)

Training Large Language Models for Reasoning through Reverse Curriculum Reinforcement Learning
by: Xi, Zhiheng, et al.
Published: (2024)

HER: Human-like Reasoning and Reinforcement Learning for LLM Role-playing
by: Du, Chengyu, et al.
Published: (2026)

Learning to Remember: End-to-End Training of Memory Agents for Long-Context Reasoning
by: Zhang, Kehao, et al.
Published: (2026)

Conditional Advantage Estimation for Reinforcement Learning in Large Reasoning Models
by: Chen, Guanxu, et al.
Published: (2025)

Revisiting Plasticity in Visual Reinforcement Learning: Data, Modules and Training Stages
by: Ma, Guozheng, et al.
Published: (2023)

RationalRewards: Reasoning Rewards Scale Visual Generation Both Training and Test Time
by: Wang, Haozhe, et al.
Published: (2026)

Revisiting Entropy in Reinforcement Learning for Large Reasoning Models
by: Jin, Renren, et al.
Published: (2025)

Scaling Behaviors of LLM Reinforcement Learning Post-Training: An Empirical Study in Mathematical Reasoning
by: Tan, Zelin, et al.
Published: (2025)

Self-Guided Process Reward Optimization with Redefined Step-wise Advantage for Process Reinforcement Learning
by: Fei, Wu, et al.
Published: (2025)

Quantum Advantage Actor-Critic for Reinforcement Learning
by: Kölle, Michael, et al.
Published: (2024)

Understanding the Role of Hallucination in Reinforcement Post-Training of Multimodal Reasoning Models
by: Zhang, Gengwei, et al.
Published: (2026)

Entropy-Guided Data-Efficient Training for Multimodal Reasoning Reward Models
by: Yang, Shidong, et al.
Published: (2026)

GAPO: Robust Advantage Estimation for Real-World Code LLMs
by: Zhang, Jianqing, et al.
Published: (2025)

GPG: A Simple and Strong Reinforcement Learning Baseline for Model Reasoning
by: Chu, Xiangxiang, et al.
Published: (2025)

Adaptive Advantage-Guided Policy Regularization for Offline Reinforcement Learning
by: Liu, Tenglong, et al.
Published: (2024)

SOAP-RL: Sequential Option Advantage Propagation for Reinforcement Learning in POMDP Environments
by: Ishida, Shu, et al.
Published: (2024)

TableGPT-R1: Advancing Tabular Reasoning Through Reinforcement Learning
by: Yang, Saisai, et al.
Published: (2025)

Breaking the Exploration Bottleneck: Rubric-Scaffolded Reinforcement Learning for General LLM Reasoning
by: Zhou, Yang, et al.
Published: (2025)

Echo: Decoupling Inference and Training for Large-Scale RL Alignment on Heterogeneous Swarms
by: Xiao, Jie, et al.
Published: (2025)

Offline Trajectory Optimization for Offline Reinforcement Learning
by: Zhao, Ziqi, et al.
Published: (2024)

VIPO: Value Function Inconsistency Penalized Offline Reinforcement Learning
by: Chen, Xuyang, et al.
Published: (2025)

GOPlan: Goal-conditioned Offline Reinforcement Learning by Planning with Learned Models
by: Wang, Mianchu, et al.
Published: (2023)