Cover Image

On the Convergence of Muon and Beyond

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Chang, Da, Liu, Yongxiang, Yuan, Ganzhao
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

MuonEq: Balancing Before Orthogonalization with Lightweight Equilibration
by: Chang, Da, et al.
Published: (2026)

AlphaAdam:Asynchronous Masked Optimization with Dynamic Alpha for Selective Updates
by: Chang, Da, et al.
Published: (2025)

OptEMA: Adaptive Exponential Moving Average for Stochastic Optimization with Zero-Noise Optimality
by: Yuan, Ganzhao
Published: (2026)

ADMM for Structured Fractional Minimization
by: Yuan, Ganzhao
Published: (2024)

Adaptive Lipschitz-Free Conditional Gradient Methods for Stochastic Composite Nonconvex Optimization
by: Yuan, Ganzhao
Published: (2026)

A Block Coordinate Descent Method for Nonsmooth Composite Optimization under Orthogonality Constraints
by: Yuan, Ganzhao
Published: (2023)

Convergence of Muon with Newton-Schulz
by: Kim, Gyu Yeol, et al.
Published: (2026)

On the Convergence Analysis of Muon
by: Shen, Wei, et al.
Published: (2025)

Convergence Bound and Critical Batch Size of Muon Optimizer
by: Sato, Naoki, et al.
Published: (2025)

Drop-Muon: Update Less, Converge Faster
by: Gruntkowska, Kaja, et al.
Published: (2025)

Muon Does Not Converge on Convex Lipschitz Functions
by: Parshakova, Tetiana, et al.
Published: (2026)

Improved Convergence Rates of Muon Optimizer for Nonconvex Optimization
by: Nagashima, Shuntaro, et al.
Published: (2026)

Rethinking Muon Beyond Pretraining: Spectral Failures and High-Pass Remedies for VLA and RLVR
by: Fan, Chongyu, et al.
Published: (2026)

Uniform Spectral Growth and Convergence of Muon in LoRA-Style Matrix Factorization
by: Kang, Changmin, et al.
Published: (2026)

Beyond the Ideal: Analyzing the Inexact Muon Update
by: Shulgin, Egor, et al.
Published: (2025)

Calibrating and Rotating: A Unified Framework for Weight Conditioning in PEFT
by: Chang, Da, et al.
Published: (2025)

AdaMuon: Adaptive Muon Optimizer
by: Si, Chongjie, et al.
Published: (2025)

Muon-OGD: Muon-based Spectral Orthogonal Gradient Projection for LLM Continual Learning
by: Lu, Binghang, et al.
Published: (2026)

Delving into Muon and Beyond: Deep Analysis and Extensions
by: Qi, Xianbiao, et al.
Published: (2026)

NorMuon: Making Muon more efficient and scalable
by: Li, Zichong, et al.
Published: (2025)

MuonQ: Enhancing Low-Bit Muon Quantization via Directional Fidelity Optimization
by: Su, Yupeng, et al.
Published: (2026)

Outlier detection in mixed-attribute data: a semi-supervised approach with fuzzy approximations and relative entropy
by: Chen, Baiyang, et al.
Published: (2025)

AMO: Adaptive Muon Orthogonalization
by: Zhuang, Xinlin, et al.
Published: (2026)

Spectral Flattening Is All Muon Needs: How Orthogonalization Controls Learning Rate and Convergence
by: Nguyen, Tien-Phat, et al.
Published: (2026)

TEON: Tensorized Orthonormalization Beyond Layer-Wise Muon for Large Language Model Pre-Training
by: Zhang, Ruijie, et al.
Published: (2026)

NuMuon: Nuclear-Norm-Constrained Muon for Compressible LLM Training
by: Dolatabadi, Hadi Mohaghegh, et al.
Published: (2026)

AlphaDecay: Module-wise Weight Decay for Heavy-Tailed Balancing in LLMs
by: He, Di, et al.
Published: (2025)

Muon Converges under Heavy-Tailed Noise: Nonconvex Hölder-Smooth Empirical Risk Minimization
by: Iiduka, Hideaki
Published: (2026)

Muon$^2$: Boosting Muon via Adaptive Second-Moment Preconditioning
by: Liu, Ziyue, et al.
Published: (2026)

Uniform Convergence Beyond Glivenko-Cantelli
by: Devale, Tanmay, et al.
Published: (2025)

Phases of Muon: When Muon Eclipses SignSGD
by: Paquette, Elliot, et al.
Published: (2026)

Practical Efficiency of Muon for Pretraining
by: AI, Essential, et al.
Published: (2025)

Beyond Muon: MUD (MomentUm Decorrelation) for Faster Transformer Training
by: Southworth, Ben S., et al.
Published: (2026)

MuonBP: Faster Muon via Block-Periodic Orthogonalization
by: Khaled, Ahmed, et al.
Published: (2025)

LiMuon: Light and Fast Muon Optimizer for Large Models
by: Huang, Feihu, et al.
Published: (2025)

DynMuon: A Dynamic Spectral Shaping View of Muon
by: Wu, Fangzhou, et al.
Published: (2026)

MuonAll: Muon Variant for Efficient Finetuning of Large Language Models
by: Page, Saurabh, et al.
Published: (2025)

FedMuon: Accelerating Federated Learning with Matrix Orthogonalization
by: Liu, Junkang, et al.
Published: (2025)

On Provable Benefits of Muon in Federated Learning
by: Zhang, Xinwen, et al.
Published: (2025)

Effective Quantization of Muon Optimizer States
by: Gupta, Aman, et al.
Published: (2025)