Imagen de Portada

Rewarding the Unlikely: Lifting GRPO Beyond Distribution Sharpening

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	He, Andre, Fried, Daniel, Welleck, Sean
Formato:	Preprint
Publicado:	2025
Materias:	Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Reasoning with Latent Tokens in Diffusion Language Models
por: He, Andre, et al.
Publicado: (2026)

Beyond Distribution Sharpening: The Importance of Task Rewards
por: Mittal, Sarthak, et al.
Publicado: (2026)

Programming with Pixels: Can Computer-Use Agents do Software Engineering?
por: Aggarwal, Pranjal, et al.
Publicado: (2025)

L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning
por: Aggarwal, Pranjal, et al.
Publicado: (2025)

miniCodeProps: a Minimal Benchmark for Proving Code Properties
por: Lohn, Evan, et al.
Publicado: (2024)

Beyond GRPO and On-Policy Distillation: An Empirical Sparse-to-Dense Reward Principle for Language-Model Post-Training
por: Xu, Yuanda, et al.
Publicado: (2026)

AlphaVerus: Bootstrapping Formally Verified Code Generation through Self-Improving Translation and Treefinement
por: Aggarwal, Pranjal, et al.
Publicado: (2024)

Gym-Anything: Turn any Software into an Agent Environment
por: Aggarwal, Pranjal, et al.
Publicado: (2026)

GRPO is Secretly a Process Reward Model
por: Sullivan, Michael, et al.
Publicado: (2025)

Optimizing Temperature for Language Models with Multi-Sample Inference
por: Du, Weihua, et al.
Publicado: (2025)

miniCTX: Neural Theorem Proving with (Long-)Contexts
por: Hu, Jiewen, et al.
Publicado: (2024)

Noise-corrected GRPO: From Noisy Rewards to Unbiased Gradients
por: Mansouri, Omar El, et al.
Publicado: (2025)

Reinforcement Learning with Verifiable Rewards: GRPO's Effective Loss, Dynamics, and Success Amplification
por: Mroueh, Youssef
Publicado: (2025)

When Sharpening Becomes Collapse: Sampling Bias and Semantic Coupling in RL with Verifiable Rewards
por: Fan, Mingyuan, et al.
Publicado: (2026)

MMR-GRPO: Accelerating GRPO-Style Training through Diversity-Aware Reward Reweighting
por: Wei, Kangda, et al.
Publicado: (2026)

FedMOA: Federated GRPO for Personalized Reasoning LLMs under Heterogeneous Rewards
por: Wang, Ziyao, et al.
Publicado: (2026)

Easy-to-Hard Generalization: Scalable Alignment Beyond Human Supervision
por: Sun, Zhiqing, et al.
Publicado: (2024)

Agentic-R1: Distilled Dual-Strategy Reasoning
por: Du, Weihua, et al.
Publicado: (2025)

MO-GRPO: Mitigating Reward Hacking of Group Relative Policy Optimization on Multi-Objective Problems
por: Ichihara, Yuki, et al.
Publicado: (2025)

FedGRPO: Privately Optimizing Foundation Models with Group-Relative Rewards from Domain Client
por: Zhu, Gongxi, et al.
Publicado: (2026)

Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domains
por: Gunjal, Anisha, et al.
Publicado: (2025)

Advances in GRPO for Generation Models: A Survey
por: Liu, Zexiang, et al.
Publicado: (2026)

Gradient Starvation in Binary-Reward GRPO: Why Group-Mean Centering Fails and Why the Simplest Fix Works
por: Nie, Wenhua, et al.
Publicado: (2026)

f-GRPO and Beyond: Divergence-Based Reinforcement Learning Algorithms for General LLM Alignment
por: Haldar, Rajdeep, et al.
Publicado: (2026)

ImProver 2: Iteratively Self-Improving LMs for Neurosymbolic Proof Optimization
por: Ahuja, Riyaz, et al.
Publicado: (2026)

LithoGRPO: Fast Inverse Lithography via GRPO Reinforced Flow Matching
por: Lai, Yao, et al.
Publicado: (2026)

AMIR-GRPO: Inducing Implicit Preference Signals into GRPO
por: Yari, Amir Hossein, et al.
Publicado: (2026)

Self-Consistency via Marginal Sharpening
por: Arzhantsev, Aleksei, et al.
Publicado: (2026)

A Minimalist Example of Edge-of-Stability and Progressive Sharpening
por: Liu, Liming, et al.
Publicado: (2025)

Differential Smoothing Mitigates Sharpening and Improves LLM Reasoning
por: Gai, Jingchu, et al.
Publicado: (2025)

GradAlign: Gradient-Aligned Data Selection for LLM Reinforcement Learning
por: Yang, Ningyuan, et al.
Publicado: (2026)

What Makes ImageNet Look Unlike LAION
por: Shirali, Ali, et al.
Publicado: (2023)

Scalable Power Sampling: Unlocking Efficient, Training-Free Reasoning for LLMs via Distribution Sharpening
por: Ji, Xiaotong, et al.
Publicado: (2026)

Soft-Label Caching and Sharpening for Communication-Efficient Federated Distillation
por: Azuma, Kitsuya, et al.
Publicado: (2025)

Graph Transductive Sharpening: Leveraging Unlabeled Predictions in Node Classification
por: Zaz, Brown, et al.
Publicado: (2026)

Mitigating Distribution Sharpening in Math RLVR via Distribution-Aligned Hint Synthesis and Backward Hint Annealing
por: Xie, Pei-Xi, et al.
Publicado: (2026)

Sharpened Lazy Incremental Quasi-Newton Method
por: Lahoti, Aakash, et al.
Publicado: (2023)

Beyond Scalar Rewards: Distributional Reinforcement Learning with Preordered Objectives for Safe and Reliable Autonomous Driving
por: Abouelazm, Ahmed, et al.
Publicado: (2026)

Premise Selection for a Lean Hammer
por: Zhu, Thomas, et al.
Publicado: (2025)

DRA-GRPO: Your GRPO Needs to Know Diverse Reasoning Paths for Mathematical Reasoning
por: Chen, Xiwen, et al.
Publicado: (2025)