Copertina

How Off-Policy Can GRPO Be? Mu-GRPO for Efficient LLM Reinforcement Learning

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Tian, Minghao, Xie, Yunfei, Wei, Chen
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Machine Learning Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Stratified GRPO: Handling Structural Heterogeneity in Reinforcement Learning of LLM Search Agents
di: Zhu, Mingkang, et al.
Pubblicazione: (2025)

Latent-GRPO: Group Relative Policy Optimization for Latent Reasoning
di: Deng, Jingcheng, et al.
Pubblicazione: (2026)

DRA-GRPO: Your GRPO Needs to Know Diverse Reasoning Paths for Mathematical Reasoning
di: Chen, Xiwen, et al.
Pubblicazione: (2025)

Can GRPO Help LLMs Transcend Their Pretraining Origin?
di: Ni, Kangqi, et al.
Pubblicazione: (2025)

Group-Relative REINFORCE Is Secretly an Off-Policy Algorithm: Demystifying Some Myths About GRPO and Its Friends
di: Yao, Chaorui, et al.
Pubblicazione: (2025)

Scaf-GRPO: Scaffolded Group Relative Policy Optimization for Enhancing LLM Reasoning
di: Zhang, Xichen, et al.
Pubblicazione: (2025)

MMR-GRPO: Accelerating GRPO-Style Training through Diversity-Aware Reward Reweighting
di: Wei, Kangda, et al.
Pubblicazione: (2026)

GRPO-CARE: Consistency-Aware Reinforcement Learning for Multimodal Reasoning
di: Chen, Yi, et al.
Pubblicazione: (2025)

Stepwise Guided Policy Optimization: Coloring your Incorrect Reasoning in GRPO
di: Chen, Peter, et al.
Pubblicazione: (2025)

7B Fully Open Source Moxin-LLM/VLM -- From Pretraining to GRPO-based Reinforcement Learning Enhancement
di: Zhao, Pu, et al.
Pubblicazione: (2024)

It Takes Two: Your GRPO Is Secretly DPO
di: Wu, Yihong, et al.
Pubblicazione: (2025)

GRAPH-GRPO-LEX: Contract Graph Modeling and Reinforcement Learning with Group Relative Policy Optimization
di: Dechtiar, Moriya, et al.
Pubblicazione: (2025)

ExGRPO: Learning to Reason from Experience
di: Zhan, Runzhe, et al.
Pubblicazione: (2025)

Multi-Task GRPO: Reliable LLM Reasoning Across Tasks
di: Ramesh, Shyam Sundhar, et al.
Pubblicazione: (2026)

MiGrATe: Mixed-Policy GRPO for Adaptation at Test-Time
di: Phan, Peter, et al.
Pubblicazione: (2025)

DACA-GRPO: Denoising-Aware Credit Assignment for Reinforcement Learning in Diffusion Language Models
di: Monsefi, Amin Karimi, et al.
Pubblicazione: (2026)

GRPO++: Enhancing Dermatological Reasoning under Low Resource Settings
di: Swapnil, Ismam Nur, et al.
Pubblicazione: (2025)

GRPO with State Mutations: Improving LLM-Based Hardware Test Plan Generation
di: Kochar, Dimple Vijay, et al.
Pubblicazione: (2026)

GanitLLM: Difficulty-Aware Bengali Mathematical Reasoning through Curriculum-GRPO
di: Dipta, Shubhashis Roy, et al.
Pubblicazione: (2026)

Off-Policy Value-Based Reinforcement Learning for Large Language Models
di: Wang, Peng-Yuan, et al.
Pubblicazione: (2026)

Balanced Aggregation: Understanding and Fixing Aggregation Bias in GRPO
di: Zeng, Zhiyuan, et al.
Pubblicazione: (2026)

GRPO-RM: Fine-Tuning Representation Models via GRPO-Driven Reinforcement Learning
di: Xu, Yanchen, et al.
Pubblicazione: (2025)

S-GRPO: Unified Post-Training for Large Vision-Language Models
di: Yan, Yuming, et al.
Pubblicazione: (2026)

GIFT: Group-Relative Implicit Fine-Tuning Integrates GRPO with DPO and UNA
di: Wang, Zhichao
Pubblicazione: (2025)

Adaptive-Boundary-Clipping GRPO: Ensuring Bounded Ratios for Stable and Generalizable Training
di: Liu, Chi, et al.
Pubblicazione: (2026)

Taming Extreme Tokens: Covariance-Aware GRPO with Gaussian-Kernel Advantage Reweighting
di: Wang, Cheng, et al.
Pubblicazione: (2026)

BranchGRPO: Stable and Efficient GRPO with Structured Branching in Diffusion Models
di: Li, Yuming, et al.
Pubblicazione: (2025)

GRPO-TTA: Test-Time Visual Tuning for Vision-Language Models via GRPO-Driven Reinforcement Learning
di: Li, Yujun, et al.
Pubblicazione: (2026)

Mitigating Selection Bias in Large Language Models via Permutation-Aware GRPO
di: Zheng, Jinquan, et al.
Pubblicazione: (2026)

Scalpel vs. Hammer: GRPO Amplifies Existing Capabilities, SFT Replaces Them
di: Rajani, Neel, et al.
Pubblicazione: (2025)

LithoGRPO: Fast Inverse Lithography via GRPO Reinforced Flow Matching
di: Lai, Yao, et al.
Pubblicazione: (2026)

Delving into RL for Image Generation with CoT: A Study on DPO vs. GRPO
di: Tong, Chengzhuo, et al.
Pubblicazione: (2025)

OP-GRPO: Efficient Off-Policy GRPO for Flow-Matching Models
di: Zhang, Liyu, et al.
Pubblicazione: (2026)

Advancing Speech Understanding in Speech-Aware Language Models with GRPO
di: Elmakies, Avishai, et al.
Pubblicazione: (2025)

FairGRPO: Fair Reinforcement Learning for Equitable Clinical Reasoning
di: Dai, Shiqi, et al.
Pubblicazione: (2025)

From Reasoning to Code: GRPO Optimization for Underrepresented Languages
di: Pennino, Federico, et al.
Pubblicazione: (2025)

Nested-ReFT: Efficient Reinforcement Learning for Large Language Model Fine-Tuning via Off-Policy Rollouts
di: Heuillet, Maxime, et al.
Pubblicazione: (2025)

Can GRPO Boost Complex Multimodal Table Understanding?
di: Kang, Xiaoqiang, et al.
Pubblicazione: (2025)

$λ$-GRPO: Unifying the GRPO Frameworks with Learnable Token Preferences
di: Wang, Yining, et al.
Pubblicazione: (2025)

BAPO: Stabilizing Off-Policy Reinforcement Learning for LLMs via Balanced Policy Optimization with Adaptive Clipping
di: Xi, Zhiheng, et al.
Pubblicazione: (2025)