Cover Image

Muon$^2$: Boosting Muon via Adaptive Second-Moment Preconditioning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Liu, Ziyue, Zhang, Ruijie, Wang, Zhengyang, Zhao, Yequan, Su, Yupeng, Yang, Zi, Zhang, Zheng
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

TEON: Tensorized Orthonormalization Beyond Layer-Wise Muon for Large Language Model Pre-Training
by: Zhang, Ruijie, et al.
Published: (2026)

MuonQ: Enhancing Low-Bit Muon Quantization via Directional Fidelity Optimization
by: Su, Yupeng, et al.
Published: (2026)

MUON+: Towards More Effective Muon via One Additional Normalization Step for LLM Pre-training
by: Zhang, Ruijie, et al.
Published: (2026)

Preconditioning Benefits of Spectral Orthogonalization in Muon
by: Ma, Jianhao, et al.
Published: (2026)

Mousse: Rectifying the Geometry of Muon with Curvature-Aware Preconditioning
by: Zhang, Yechen, et al.
Published: (2026)

DynMuon: A Dynamic Spectral Shaping View of Muon
by: Wu, Fangzhou, et al.
Published: (2026)

Muon is Scalable for LLM Training
by: Liu, Jingyuan, et al.
Published: (2025)

The Newton-Muon Optimizer
by: Du, Zhehang, et al.
Published: (2026)

CoLA: Compute-Efficient Pre-Training of LLMs via Low-Rank Activation
by: Liu, Ziyue, et al.
Published: (2025)

TrasMuon: Trust-Region Adaptive Scaling for Orthogonalized Momentum Optimizers
by: Cheng, Peng, et al.
Published: (2026)

HTMuon: Improving Muon via Heavy-Tailed Spectral Correction
by: Pang, Tianyu, et al.
Published: (2026)

LaX: Boosting Low-Rank Training of Foundation Models via Latent Crossing
by: Zhang, Ruijie, et al.
Published: (2025)

MiMuon: Mixed Muon Optimizer with Improved Generalization for Large Models
by: Huang, Feihu, et al.
Published: (2026)

FedMuon: Accelerating Federated Learning with Matrix Orthogonalization
by: Liu, Junkang, et al.
Published: (2025)

CoMERA: Computing- and Memory-Efficient Training via Rank-Adaptive Tensor Optimization
by: Yang, Zi, et al.
Published: (2024)

Muon Optimizer Accelerates Grokking
by: Tveit, Amund, et al.
Published: (2025)

Optimizer-Induced Mode Connectivity: From AdamW to Muon
by: Zhang, Fangzhao, et al.
Published: (2026)

Muon Outperforms Adam in Tail-End Associative Memory Learning
by: Wang, Shuche, et al.
Published: (2025)

Intrinsic Muon: Spectral Optimization on Riemannian Matrix Manifolds
by: Li, Yibang, et al.
Published: (2026)

AdaMuon: Adaptive Muon Optimizer
by: Si, Chongjie, et al.
Published: (2025)

DeMuon: A Decentralized Muon for Matrix Optimization over Graphs
by: He, Chuan, et al.
Published: (2025)

QuZO: Quantized Zeroth-Order Fine-Tuning for Large Language Models
by: Zhou, Jiajun, et al.
Published: (2025)

Muon Dynamics as a Spectral Wasserstein Flow
by: Peyré, Gabriel
Published: (2026)

Delving into Muon and Beyond: Deep Analysis and Extensions
by: Qi, Xianbiao, et al.
Published: (2026)

Advancing Model Refinement: Muon-Optimized Distillation and Quantization for LLM Deployment
by: Sander, Jacob, et al.
Published: (2026)

Spectral Flattening Is All Muon Needs: How Orthogonalization Controls Learning Rate and Convergence
by: Nguyen, Tien-Phat, et al.
Published: (2026)

FuRA: Full-Rank Parameter-Efficient Fine-Tuning with Spectral Preconditioning
by: Zhao, Yequan, et al.
Published: (2026)

AMO: Adaptive Muon Orthogonalization
by: Zhuang, Xinlin, et al.
Published: (2026)

Muon is Not That Special: Random or Inverted Spectra Work Just as Well
by: Shumaylov, Zakhar, et al.
Published: (2026)

Tensor-Compressed and Fully-Quantized Training of Neural PDE Solvers
by: Lu, Jinming, et al.
Published: (2025)

Finetune Once: Decoupling General & Domain Learning with Dynamic Boosted Annealing
by: Tang, Yang, et al.
Published: (2025)

MuonEq: Balancing Before Orthogonalization with Lightweight Equilibration
by: Chang, Da, et al.
Published: (2026)

HARP: Hadamard-Preconditioned Adaptive Rotation Processor for Extreme LLM Quantization
by: Zagitov, Artur, et al.
Published: (2026)

Adam Improves Muon: Adaptive Moment Estimation with Orthogonalized Momentum
by: Zhang, Minxin, et al.
Published: (2026)

Online Boosting Adaptive Learning under Concept Drift for Multistream Classification
by: Yu, En, et al.
Published: (2023)

Position-Aware Parameter Efficient Fine-Tuning Approach for Reducing Positional Bias in LLMs
by: Zhang, Zheng, et al.
Published: (2024)

NorMuon: Making Muon more efficient and scalable
by: Li, Zichong, et al.
Published: (2025)

Why Adam Can Beat SGD: Second-Moment Normalization Yields Sharper Tails
by: Jin, Ruinan, et al.
Published: (2026)

CompilerKV: Risk-Adaptive KV Compression via Offline Experience Compilation
by: Yang, Ning, et al.
Published: (2026)

Deep Reinforcement Learning with Task-Adaptive Retrieval via Hypernetwork
by: Jin, Yonggang, et al.
Published: (2023)