Cover Image

Demystifying Group Relative Policy Optimization: Its Policy Gradient is a U-Statistic

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhou, Hongyi, Ye, Kai, Xu, Erhan, Zhu, Jin, Yang, Ying, Gong, Shijin, Shi, Chengchun
Format:	Preprint
Published:	2026
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Demystifying the Paradox of Importance Sampling with an Estimated History-Dependent Behavior Policy in Off-Policy Evaluation
by: Zhou, Hongyi, et al.
Published: (2025)

Kernelized Advantage Estimation: From Nonparametric Statistics to LLM Reasoning
by: Gong, Shijin, et al.
Published: (2026)

Learn-to-Distance: Distance Learning for Detecting LLM-Generated Text
by: Zhou, Hongyi, et al.
Published: (2026)

BASIS: Batchwise Advantage Estimation from Single-Rollout Information Sharing for LLM Reasoning
by: Gong, Shijin, et al.
Published: (2026)

Doubly Robust Alignment for Large Language Models
by: Xu, Erhan, et al.
Published: (2025)

Detecting LLM-Generated Text with Performance Guarantees
by: Zhou, Hongyi, et al.
Published: (2026)

Robust Reinforcement Learning from Human Feedback for Large Language Models Fine-Tuning
by: Ye, Kai, et al.
Published: (2025)

AdaDetectGPT: Adaptive Detection of LLM-Generated Text with Statistical Guarantees
by: Zhou, Hongyi, et al.
Published: (2025)

Statistical Inference in Reinforcement Learning: A Selective Survey
by: Shi, Chengchun
Published: (2025)

Group-Relative REINFORCE Is Secretly an Off-Policy Algorithm: Demystifying Some Myths About GRPO and Its Friends
by: Yao, Chaorui, et al.
Published: (2025)

Revisiting Group Relative Policy Optimization: Insights into On-Policy and Off-Policy Training
by: Mroueh, Youssef, et al.
Published: (2025)

Group Policy Gradient
by: Chen, Junhua, et al.
Published: (2025)

NGRPO: Negative-enhanced Group Relative Policy Optimization
by: Nan, Gongrui, et al.
Published: (2025)

Balancing Interference and Correlation in Spatial Experimental Designs: A Causal Graph Cut Approach
by: Zhu, Jin, et al.
Published: (2025)

Combining Experimental and Historical Data for Policy Evaluation
by: Li, Ting, et al.
Published: (2024)

Constrained Group Relative Policy Optimization
by: Girgis, Roger, et al.
Published: (2026)

Relative Policy-Transition Optimization for Fast Policy Transfer
by: Xu, Jiawei, et al.
Published: (2022)

GTPO: Stabilizing Group Relative Policy Optimization via Gradient and Entropy Control
by: Simoni, Marco, et al.
Published: (2025)

Consensus Group Relative Policy Optimization for Text Generation
by: Ichihara, Yuki, et al.
Published: (2026)

Reinforcement Unlearning via Group Relative Policy Optimization
by: Zaradoukas, Efstratios, et al.
Published: (2026)

Double Fairness Policy Learning: Integrating Action Fairness and Outcome Fairness in Decision-making
by: Bian, Zeyu, et al.
Published: (2026)

Group Sequence Policy Optimization
by: Zheng, Chujie, et al.
Published: (2025)

Latent-GRPO: Group Relative Policy Optimization for Latent Reasoning
by: Deng, Jingcheng, et al.
Published: (2026)

GRPOformer: Advancing Hyperparameter Optimization via Group Relative Policy Optimization
by: Guo, Haoxin, et al.
Published: (2025)

F-GRPO: Factorized Group-Relative Policy Optimization for Unified Candidate Generation and Ranking
by: Surana, Rohan, et al.
Published: (2026)

Advantage Collapse in Group Relative Policy Optimization: Diagnosis and Mitigation
by: He, Xixiang, et al.
Published: (2026)

AGPO: Adaptive Group Policy Optimization with Dual Statistical Feedback
by: Hu, Miaobo, et al.
Published: (2026)

Reinforcement Learning from Human Feedback: A Statistical Perspective
by: Liu, Pangpang, et al.
Published: (2026)

Hybrid Group Relative Policy Optimization: A Multi-Sample Approach to Enhancing Policy Optimization
by: Sane, Soham
Published: (2025)

GAGPO: Generalized Advantage Grouped Policy Optimization
by: Zhu, Siyuan, et al.
Published: (2026)

Sharpness-Guided Group Relative Policy Optimization via Probability Shaping
by: Le, Tue, et al.
Published: (2025)

EBPO: Empirical Bayes Shrinkage for Stabilizing Group-Relative Policy Optimization
by: Han, Kevin, et al.
Published: (2026)

Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy
by: Li, Pingzhi, et al.
Published: (2023)

Group Orthogonalized Policy Optimization:Group Policy Optimization as Orthogonal Projection in Hilbert Space
by: Zixian, Wang
Published: (2026)

Algorithm-Relative Trajectory Valuation in Policy Gradient Control
by: Li, Shihao, et al.
Published: (2025)

Unifying Group-Relative and Self-Distillation Policy Optimization via Sample Routing
by: Li, Gengsheng, et al.
Published: (2026)

Wasserstein Proximal Policy Gradient
by: Zhu, Zhaoyu, et al.
Published: (2026)

Co-GRPO: Co-Optimized Group Relative Policy Optimization for Masked Diffusion Model
by: Zhou, Renping, et al.
Published: (2025)

Unraveling the Interplay between Carryover Effects and Reward Autocorrelations in Switchback Experiments
by: Wen, Qianglin, et al.
Published: (2024)

Log-Sum-Exponential Estimator for Off-Policy Evaluation and Learning
by: Behnamnia, Armin, et al.
Published: (2025)