Copertina

Balancing the Reasoning Load: Difficulty-Differentiated Policy Optimization with Length Redistribution for Efficient and Robust Reinforcement Learning

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Xia, Yinan, Zhang, Haotian, Wang, Huiming
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Machine Learning Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

From the Inside Out: Progressive Distribution Refinement for Confidence Calibration
di: Yang, Xizhong, et al.
Pubblicazione: (2026)

Efficient RLVR Training via Weighted Mutual Information Data Selection
di: Zhou, Xinyu, et al.
Pubblicazione: (2026)

HeaPA: Difficulty-Aware Heap Sampling and On-Policy Query Augmentation for LLM Reinforcement Learning
di: Wang, Weiqi, et al.
Pubblicazione: (2026)

LSPO: Length-aware Dynamic Sampling for Policy Optimization in LLM Reasoning
di: Chen, Weizhe, et al.
Pubblicazione: (2025)

DISCO Balances the Scales: Adaptive Domain- and Difficulty-Aware Reinforcement Learning on Imbalanced Data
di: Zhou, Yuhang, et al.
Pubblicazione: (2025)

BAPO: Stabilizing Off-Policy Reinforcement Learning for LLMs via Balanced Policy Optimization with Adaptive Clipping
di: Xi, Zhiheng, et al.
Pubblicazione: (2025)

IAPO: Information-Aware Policy Optimization for Token-Efficient Reasoning
di: He, Yinhan, et al.
Pubblicazione: (2026)

D$^2$Evo: Dual Difficulty-Aware Self-Evolution for Data-Efficient Reinforcement Learning
di: Zhang, Ru, et al.
Pubblicazione: (2026)

NFT: Bridging Supervised Learning and Reinforcement Learning in Math Reasoning
di: Chen, Huayu, et al.
Pubblicazione: (2025)

Internalizing Outcome Supervision into Process Supervision: A New Paradigm for Reinforcement Learning for Reasoning
di: Ding, Fei, et al.
Pubblicazione: (2026)

Learn to Reason Efficiently with Adaptive Length-based Reward Shaping
di: Liu, Wei, et al.
Pubblicazione: (2025)

Asymmetric REINFORCE for off-Policy Reinforcement Learning: Balancing positive and negative rewards
di: Arnal, Charles, et al.
Pubblicazione: (2025)

Group Distributionally Robust Optimization-Driven Reinforcement Learning for LLM Reasoning
di: Panaganti, Kishan, et al.
Pubblicazione: (2026)

Efficient Reasoning with Balanced Thinking
di: Li, Yulin, et al.
Pubblicazione: (2026)

REA-RL: Reflection-Aware Online Reinforcement Learning for Efficient Reasoning
di: Deng, Hexuan, et al.
Pubblicazione: (2025)

EPO: Entropy-regularized Policy Optimization for LLM Agents Reinforcement Learning
di: Xu, Wujiang, et al.
Pubblicazione: (2025)

Agentic Reinforced Policy Optimization
di: Dong, Guanting, et al.
Pubblicazione: (2025)

Balancing the Scales: Reinforcement Learning for Fair Classification
di: Eshuijs, Leon, et al.
Pubblicazione: (2024)

Stepwise Penalization for Length-Efficient Chain-of-Thought Reasoning
di: Li, Xintong, et al.
Pubblicazione: (2026)

Causally-Enhanced Reinforcement Policy Optimization
di: Wang, Xiangqi, et al.
Pubblicazione: (2025)

REINFORCE++: Stabilizing Critic-Free Policy Optimization with Global Advantage Normalization
di: Hu, Jian, et al.
Pubblicazione: (2025)

C$^2$GSPG: Confidence-calibrated Group Sequence Policy Gradient towards Self-aware Reasoning
di: Liu, Haotian, et al.
Pubblicazione: (2025)

Multi-Objective and Mixed-Reward Reinforcement Learning via Reward-Decorrelated Policy Optimization
di: Bai, Yang, et al.
Pubblicazione: (2026)

SmartThinker: Progressive Chain-of-Thought Length Calibration for Efficient Large Language Model Reasoning
di: Hu, Chenzhi, et al.
Pubblicazione: (2026)

SABER: Switchable and Balanced Training for Efficient LLM Reasoning
di: Zhao, Kai, et al.
Pubblicazione: (2025)

CE-GPPO: Coordinating Entropy via Gradient-Preserving Clipping Policy Optimization in Reinforcement Learning
di: Su, Zhenpeng, et al.
Pubblicazione: (2025)

Auxiliary-Loss-Free Load Balancing Strategy for Mixture-of-Experts
di: Wang, Lean, et al.
Pubblicazione: (2024)

Reasoning through Exploration: A Reinforcement Learning Framework for Robust Function Calling
di: Hao, Bingguang, et al.
Pubblicazione: (2025)

TACO-RL: Task Aware Prompt Compression Optimization with Reinforcement Learning
di: Shandilya, Shivam, et al.
Pubblicazione: (2024)

Internalize the Temperature: On-Policy Self-Distillation as Policy Reheater for Reinforcement Learning
di: Yang, Xuewei, et al.
Pubblicazione: (2026)

Not All Tokens Matter: Towards Efficient LLM Reasoning via Token Significance in Reinforcement Learning
di: Liu, Hanbing, et al.
Pubblicazione: (2025)

UFO-RL: Uncertainty-Focused Optimization for Efficient Reinforcement Learning Data Selection
di: Zhao, Yang, et al.
Pubblicazione: (2025)

Length Desensitization in Direct Preference Optimization
di: Liu, Wei, et al.
Pubblicazione: (2024)

Agentic Entropy-Balanced Policy Optimization
di: Dong, Guanting, et al.
Pubblicazione: (2025)

How Off-Policy Can GRPO Be? Mu-GRPO for Efficient LLM Reinforcement Learning
di: Tian, Minghao, et al.
Pubblicazione: (2026)

MHPO: Modulated Hazard-aware Policy Optimization for Stable Reinforcement Learning
di: Wang, Hongjun, et al.
Pubblicazione: (2026)

Thinking-Free Policy Initialization Makes Distilled Reasoning Models More Effective and Efficient Reasoners
di: Xu, Xin, et al.
Pubblicazione: (2025)

Scaling Reasoning Efficiently via Relaxed On-Policy Distillation
di: Ko, Jongwoo, et al.
Pubblicazione: (2026)

Tackling Length Inflation Without Trade-offs: Group Relative Reward Rescaling for Reinforcement Learning
di: Li, Zichao, et al.
Pubblicazione: (2026)

PrAg-PO: Prompt Augmented Policy Optimization for Robust and Diverse Mathematical Reasoning
di: Lu, Wenquan, et al.
Pubblicazione: (2026)