Image de couverture de livre

f-GRPO and Beyond: Divergence-Based Reinforcement Learning Algorithms for General LLM Alignment

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Haldar, Rajdeep, Mei, Lantao, Lin, Guang, Xing, Yue, Song, Qifan
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

LLM Safety Alignment is Divergence Estimation in Disguise
par: Haldar, Rajdeep, et autres
Publié: (2025)

Adversarial Vulnerability as a Consequence of On-Manifold Inseparibility
par: Haldar, Rajdeep, et autres
Publié: (2024)

Effect of Ambient-Intrinsic Dimension Gap on Adversarial Vulnerability
par: Haldar, Rajdeep, et autres
Publié: (2024)

Fair Supervised Learning with A Simple Random Sampler of Sensitive Attributes
par: Sohn, Jinwon, et autres
Publié: (2023)

Bayesian Federated Learning with Hamiltonian Monte Carlo: Algorithm and Theory
par: Liang, Jiajun, et autres
Publié: (2024)

Task-tailored Pre-processing: Fair Downstream Supervised Learning
par: Sohn, Jinwon, et autres
Publié: (2026)

Theoretical Understanding of In-Context Learning in Shallow Transformers with Unstructured Data
par: Xing, Yue, et autres
Publié: (2024)

How Off-Policy Can GRPO Be? Mu-GRPO for Efficient LLM Reinforcement Learning
par: Tian, Minghao, et autres
Publié: (2026)

Better Representations via Adversarial Training in Pre-Training: A Theoretical Perspective
par: Xing, Yue, et autres
Publié: (2024)

Generalization Error of $f$-Divergence Stabilized Algorithms via Duality
par: Daunas, Francisco, et autres
Publié: (2025)

$f$-Divergence Based Classification: Beyond the Use of Cross-Entropy
par: Novello, Nicola, et autres
Publié: (2024)

What is the Alignment Objective of GRPO?
par: Vojnovic, Milan, et autres
Publié: (2025)

Kalman Filter Enhanced GRPO for Reinforcement Learning-Based Language Model Reasoning
par: Wang, Hu, et autres
Publié: (2025)

On Weak-to-Strong Generalization and f-Divergence
par: Yao, Wei, et autres
Publié: (2025)

LithoGRPO: Fast Inverse Lithography via GRPO Reinforced Flow Matching
par: Lai, Yao, et autres
Publié: (2026)

SQS: Bayesian DNN Compression through Sparse Quantized Sub-distributions
par: Wang, Ziyi, et autres
Publié: (2025)

Learning Without Critics? Revisiting GRPO in Classical Reinforcement Learning Environments
par: de Oliveira, Bryan L. M., et autres
Publié: (2025)

Graph-GRPO: Training Graph Flow Models with Reinforcement Learning
par: Zhu, Baoheng, et autres
Publié: (2026)

FairGRPO: Fair Reinforcement Learning for Equitable Clinical Reasoning
par: Dai, Shiqi, et autres
Publié: (2025)

Stratified GRPO: Handling Structural Heterogeneity in Reinforcement Learning of LLM Search Agents
par: Zhu, Mingkang, et autres
Publié: (2025)

A Unified Framework for Rethinking Policy Divergence Measures in GRPO
par: Wu, Qingyuan, et autres
Publié: (2026)

Loss Functions and Operators Generated by f-Divergences
par: Roulet, Vincent, et autres
Publié: (2025)

Robust Offline Reinforcement Learning with Linearly Structured f-Divergence Regularization
par: Tang, Cheng, et autres
Publié: (2024)

Generalized Discrete Diffusion with Self-Correction
par: Wang, Linxuan, et autres
Publié: (2026)

Minimizing $f$-Divergences by Interpolating Velocity Fields
par: Liu, Song, et autres
Publié: (2023)

Unified Algorithms for RL with Decision-Estimation Coefficients: PAC, Reward-Free, Preference-Based Learning, and Beyond
par: Chen, Fan, et autres
Publié: (2022)

Parallelly Tempered Generative Adversarial Nets: Toward Stabilized Gradients
par: Sohn, Jinwon, et autres
Publié: (2024)

Improving Visual Representation Alignment Generation with GRPO
par: Mo, Shentong, et autres
Publié: (2026)

GRPO-RM: Fine-Tuning Representation Models via GRPO-Driven Reinforcement Learning
par: Xu, Yanchen, et autres
Publié: (2025)

Rewarding the Unlikely: Lifting GRPO Beyond Distribution Sharpening
par: He, Andre, et autres
Publié: (2025)

S-GRPO: Early Exit via Reinforcement Learning in Reasoning Models
par: Dai, Muzhi, et autres
Publié: (2025)

Reinforcement Learning with Verifiable Rewards: GRPO's Effective Loss, Dynamics, and Success Amplification
par: Mroueh, Youssef
Publié: (2025)

TIC-GRPO: Provable and Efficient Optimization for Reinforcement Learning from Human Feedback
par: Pang, Lei, et autres
Publié: (2025)

GRPO with State Mutations: Improving LLM-Based Hardware Test Plan Generation
par: Kochar, Dimple Vijay, et autres
Publié: (2026)

Beyond KL Divergence: Policy Optimization with Flexible Bregman Divergences for LLM Reasoning
par: Yuan, Rui, et autres
Publié: (2026)

FedKL: Tackling Data Heterogeneity in Federated Reinforcement Learning by Penalizing KL Divergence
par: Xie, Zhijie, et autres
Publié: (2022)

Improving LLM Safety Alignment with Dual-Objective Optimization
par: Zhao, Xuandong, et autres
Publié: (2025)

Inference-Aware Meta-Alignment of LLMs via Non-Linear GRPO
par: Takakura, Shokichi, et autres
Publié: (2026)

Robust Semi-supervised Learning via $f$-Divergence and $α$-Rényi Divergence
par: Aminian, Gholamali, et autres
Publié: (2024)

Deep Generative Spatiotemporal Engression for Probabilistic Forecasting of Epidemics
par: Pathak, Rajdeep, et autres
Publié: (2026)