Image de couverture de livre

Variance-Adaptive Muon: Accelerating LLM Pretraining with NSR-Modulated and Variance-Scaled Momentum

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Li, Jingru, Fan, Yibo, Li, Huan
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Muon is Provably Faster with Momentum Variance Reduction
par: Qian, Xun, et autres
Publié: (2025)

Adaptive Optimization via Momentum on Variance-Normalized Gradients
par: Patitucci, Francisco, et autres
Publié: (2026)

Accelerating Byzantine-Robust Distributed Learning with Compressed Communication via Double Momentum and Variance Reduction
par: Li, Yanghao, et autres
Publié: (2026)

Adaptive Variance Reduction for Stochastic Optimization under Weaker Assumptions
par: Jiang, Wei, et autres
Publié: (2024)

TrasMuon: Trust-Region Adaptive Scaling for Orthogonalized Momentum Optimizers
par: Cheng, Peng, et autres
Publié: (2026)

Efficient Backpropagation with Variance-Controlled Adaptive Sampling
par: Wang, Ziteng, et autres
Publié: (2024)

Empirical Privacy Variance
par: Hu, Yuzheng, et autres
Publié: (2025)

Global Convergence of Natural Policy Gradient with Hessian-aided Momentum Variance Reduction
par: Feng, Jie, et autres
Publié: (2024)

Accelerating RLHF Training with Reward Variance Increase
par: Yang, Zonglin, et autres
Publié: (2025)

Variance-aware Reward Modeling with Anchor Guidance
par: Fang, Shuxing, et autres
Publié: (2026)

DP-Muon: Differentially Private Optimization via Matrix-Orthogonalized Momentum
par: Kim, Jihwan, et autres
Publié: (2026)

GPAS: Accelerating Convergence of LLM Pretraining via Gradient-Preserving Activation Scaling
par: Chen, Tianhao, et autres
Publié: (2025)

Ratio-Variance Regularized Policy Optimization for Efficient LLM Fine-tuning
par: Luo, Yu, et autres
Publié: (2026)

VASSO: Variance Suppression for Sharpness-Aware Minimization
par: Li, Bingcong, et autres
Publié: (2025)

The Optimal Token Baseline: Variance Reduction for Long-Horizon LLM-RL
par: Li, Yingru, et autres
Publié: (2026)

Practical Efficiency of Muon for Pretraining
par: AI, Essential, et autres
Publié: (2025)

On Vanishing Variance in Transformer Length Generalization
par: Li, Ruining, et autres
Publié: (2025)

Adaptive Variance-Penalized Continual Learning with Fisher Regularization
par: Sarkar, Krisanu
Publié: (2025)

Variance-Based Pruning for Accelerating and Compressing Trained Networks
par: Berisha, Uranik, et autres
Publié: (2025)

Enhanced Federated Optimization: Adaptive Unbiased Client Sampling with Reduced Variance
par: Zeng, Dun, et autres
Publié: (2023)

Allocating Variance to Maximize Expectation
par: Leme, Renato Purita Paes, et autres
Publié: (2025)

EVPO: Explained Variance Policy Optimization for Adaptive Critic Utilization in LLM Post-Training
par: Pan, Chengjun, et autres
Publié: (2026)

Efficient Sign-Based Optimization: Accelerating Convergence via Variance Reduction
par: Jiang, Wei, et autres
Publié: (2024)

TRSVR: An Adaptive Stochastic Trust-Region Method with Variance Reduction
par: Fang, Yuchen, et autres
Publié: (2026)

Data- and Variance-dependent Regret Bounds for Online Tabular MDPs
par: Li, Mingyi, et autres
Publié: (2026)

Variance-Aware Adaptive Weighting for Diffusion Model Training
par: Sun, Nanlong, et autres
Publié: (2026)

Variance Control via Weight Rescaling in LLM Pre-training
par: Owen, Louis, et autres
Publié: (2025)

Variance-Adaptive Optimal Algorithm for Reinforcement Learning with Multinomial Logit Function Approximation
par: Kim, Wonyoung, et autres
Publié: (2026)

Non-Convex Optimization in Federated Learning via Variance Reduction and Adaptive Learning
par: Thakur, Dipanwita, et autres
Publié: (2024)

Projection-Free Variance Reduction Methods for Stochastic Constrained Multi-Level Compositional Optimization
par: Jiang, Wei, et autres
Publié: (2024)

SAPPHIRE: Preconditioned Stochastic Variance Reduction for Faster Large-Scale Statistical Learning
par: Sun, Jingruo, et autres
Publié: (2025)

Variance-Reducing Couplings for Random Features
par: Reid, Isaac, et autres
Publié: (2024)

Fair CoVariance Neural Networks
par: Cavallo, Andrea, et autres
Publié: (2024)

Zero-Variance Gradients for Variational Autoencoders
par: Shao, Zilei, et autres
Publié: (2025)

Conformal Risk Minimization with Variance Reduction
par: Noorani, Sima, et autres
Publié: (2024)

On the Role of Preference Variance in Preference Optimization
par: Guo, Jiacheng, et autres
Publié: (2025)

STATE: A Robust ATE Estimator of Heavy-Tailed Metrics for Variance Reduction in Online Controlled Experiments
par: Zhou, Hao, et autres
Publié: (2024)

Stochastic Variance-Reduced Newton: Accelerating Finite-Sum Minimization with Large Batches
par: Dereziński, Michał
Publié: (2022)

Precise Asymptotics and Refined Regret of Variance-Aware UCB
par: Fan, Yingying, et autres
Publié: (2024)

Breaking the Stochasticity Barrier: An Adaptive Variance-Reduced Method for Variational Inequalities
par: Jeong, Yungi, et autres
Publié: (2026)