Copertina

GAC: Stabilizing Asynchronous RL Training for LLMs via Gradient Alignment Control

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Xu, Haofeng, Su, Junwei, Tian, Yukun, Diao, Lansong, Qian, Zhengping, Wu, Chuan
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Machine Learning Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

GAC: Noise-Aware Adaptive Mixing for Hybrid SFT-RL Post-Training
di: Hu, Yuelin, et al.
Pubblicazione: (2026)

When Do Multi-Agent Systems Outperform? Analysing the Learning Efficiency of Agentic Systems
di: Su, Junwei, et al.
Pubblicazione: (2026)

Imbalanced Gradients in RL Post-Training of Multi-Task LLMs
di: Wu, Runzhe, et al.
Pubblicazione: (2025)

Laminar: A Scalable Asynchronous RL Post-Training Framework
di: Sheng, Guangming, et al.
Pubblicazione: (2025)

LlamaRL: A Distributed Asynchronous Reinforcement Learning Framework for Efficient Large-scale LLM Training
di: Wu, Bo, et al.
Pubblicazione: (2025)

Partial Policy Gradients for RL in LLMs
di: Mathur, Puneet, et al.
Pubblicazione: (2026)

AsyncFlow: An Asynchronous Streaming RL Framework for Efficient LLM Post-Training
di: Han, Zhenyu, et al.
Pubblicazione: (2025)

Verifier-Free RL for LLMs via Intrinsic Gradient-Norm Reward
di: Wen, Xuexiang, et al.
Pubblicazione: (2026)

Scaling Up RL: Unlocking Diverse Reasoning in LLMs via Prolonged Training
di: Liu, Mingjie, et al.
Pubblicazione: (2025)

SortedRL: Accelerating RL Training for LLMs through Online Length-Aware Scheduling
di: Zhang, Yiqi, et al.
Pubblicazione: (2026)

EntroPIC: Towards Stable Long-Term Training of LLMs via Entropy Stabilization with Proportional-Integral Control
di: Yang, Kai, et al.
Pubblicazione: (2025)

Tool Zero: Training Tool-Augmented LLMs via Pure RL from Scratch
di: Zeng, Yirong, et al.
Pubblicazione: (2025)

Multi-Value Alignment for LLMs via Value Decorrelation and Extrapolation
di: Xu, Hefei, et al.
Pubblicazione: (2025)

CAP: Controllable Alignment Prompting for Unlearning in LLMs
di: Wang, Zhaokun, et al.
Pubblicazione: (2026)

A Comedy of Estimators: On KL Regularization in RL Training of LLMs
di: Shah, Vedant, et al.
Pubblicazione: (2025)

Temporal-Aware Evaluation and Learning for Temporal Graph Neural Networks
di: Su, Junwei, et al.
Pubblicazione: (2024)

EARL: Entropy-Aware RL Alignment of LLMs for Reliable RTL Code Generation
di: Shi, Jiahe, et al.
Pubblicazione: (2025)

GaLore 2: Large-Scale LLM Pre-Training by Gradient Low-Rank Projection
di: Su, DiJia, et al.
Pubblicazione: (2025)

The Two-Stage Decision-Sampling Hypothesis: Understanding the Emergence of Self-Reflection in RL-Trained LLMs
di: Zhao, Zibo, et al.
Pubblicazione: (2026)

How to Compress KV Cache in RL Post-Training? Shadow Mask Distillation for Memory-Efficient Alignment
di: Zhu, Rui, et al.
Pubblicazione: (2026)

Echo: Decoupling Inference and Training for Large-Scale RL Alignment on Heterogeneous Swarms
di: Xiao, Jie, et al.
Pubblicazione: (2025)

Missing Old Logits in Asynchronous Agentic RL: Semantic Mismatch and Repair Methods for Off-Policy Correction
di: Guan, Zhong, et al.
Pubblicazione: (2026)

MVCL-DAF++: Enhancing Multimodal Intent Recognition via Prototype-Aware Contrastive Alignment and Coarse-to-Fine Dynamic Attention Fusion
di: Huang, Haofeng, et al.
Pubblicazione: (2025)

DARA: Few-shot Budget Allocation in Online Advertising via In-Context Decision Making with RL-Finetuned LLMs
di: Song, Mingxuan, et al.
Pubblicazione: (2026)

Stable Asynchrony: Variance-Controlled Off-Policy RL for LLMs
di: Huang, Luke J., et al.
Pubblicazione: (2026)

COSMO-RL: Towards Trustworthy LMRMs via Joint Safety and Stability
di: Ding, Yizhuo, et al.
Pubblicazione: (2025)

DistRL: An Asynchronous Distributed Reinforcement Learning Framework for On-Device Control Agents
di: Wang, Taiyi, et al.
Pubblicazione: (2024)

Arithmetic Control of LLMs for Diverse User Preferences: Directional Preference Alignment with Multi-Objective Rewards
di: Wang, Haoxiang, et al.
Pubblicazione: (2024)

On Effectiveness and Efficiency of Agentic Tool-calling and RL Training
di: Liu, Tong, et al.
Pubblicazione: (2026)

Training Large Language Models to Reason via EM Policy Gradient
di: Xu, Tianbing
Pubblicazione: (2025)

Taming LLMs by Scaling Learning Rates with Gradient Grouping
di: Li, Siyuan, et al.
Pubblicazione: (2025)

ProRL: Effective Reinforcement Learning for Proactive Recommendation via Rectified Policy Gradient Estimation
di: Hou, Hongru, et al.
Pubblicazione: (2026)

VI-CuRL: Stabilizing Verifier-Independent RL Reasoning via Confidence-Guided Variance Reduction
di: Cai, Xin-Qiang, et al.
Pubblicazione: (2026)

Training LLMs for Honesty via Confessions
di: Joglekar, Manas, et al.
Pubblicazione: (2025)

Align and Filter: Improving Performance in Asynchronous On-Policy RL
di: Honari, Homayoun, et al.
Pubblicazione: (2026)

Knapsack RL: Unlocking Exploration of LLMs via Optimizing Budget Allocation
di: Li, Ziniu, et al.
Pubblicazione: (2025)

QaRL: Rollout-Aligned Quantization-Aware RL for Fast and Stable Training under Training--Inference Mismatch
di: Gu, Hao, et al.
Pubblicazione: (2026)

CoBA-RL: Capability-Oriented Budget Allocation for Reinforcement Learning in LLMs
di: Yao, Zhiyuan, et al.
Pubblicazione: (2026)

Consolidation or Adaptation? PRISM: Disentangling SFT and RL Data via Gradient Concentration
di: Zhao, Yang, et al.
Pubblicazione: (2026)

TimeMaster: Training Time-Series Multimodal LLMs to Reason via Reinforcement Learning
di: Zhang, Junru, et al.
Pubblicazione: (2025)