Image de couverture de livre

Improving Adaptive Moment Optimization via Preconditioner Diagonalization

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Nguyen, Son, Liu, Bo, Chen, Lizhang, Liu, Qiang
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Memory-Efficient Optimization with Factorized Hamiltonian Descent
par: Nguyen, Son, et autres
Publié: (2024)

Cautious Optimizers: Improving Training with One Line of Code
par: Liang, Kaizhao, et autres
Publié: (2024)

Lion Secretly Solves Constrained Optimization: As Lyapunov Predicts
par: Chen, Lizhang, et autres
Publié: (2023)

Muon Optimizes Under Spectral Norm Constraints
par: Chen, Lizhang, et autres
Publié: (2025)

Memory-Efficient LLM Training with Online Subspace Descent
par: Liang, Kaizhao, et autres
Publié: (2024)

Structured Preconditioners in Adaptive Optimization: A Unified Analysis
par: Xie, Shuo, et autres
Publié: (2025)

DeMo: Decoupled Momentum Optimization
par: Peng, Bowen, et autres
Publié: (2024)

Training-Free Looped Transformers
par: Chen, Lizhang, et autres
Publié: (2026)

Adaptive Preconditioners Trigger Loss Spikes in Adam
par: Bai, Zhiwei, et autres
Publié: (2025)

Taming Preconditioner Drift: Unlocking the Potential of Second-Order Optimizers for Federated Learning on Non-IID Data
par: Liu, Junkang, et autres
Publié: (2026)

Momentum Guidance: Plug-and-Play Guidance for Flow Models
par: Liao, Runlong, et autres
Publié: (2026)

Communication Efficient Distributed Training with Distributed Lion
par: Liu, Bo, et autres
Publié: (2024)

$ϕ$-Balancing for Mixture-of-Experts Training
par: Chen, Lizhang, et autres
Publié: (2026)

AdaFlow: Imitation Learning with Variance-Adaptive Flow-Based Policies
par: Hu, Xixi, et autres
Publié: (2024)

Curvature-Informed SGD via General Purpose Lie-Group Preconditioners
par: Pooladzandi, Omead, et autres
Publié: (2024)

Muon$^2$: Boosting Muon via Adaptive Second-Moment Preconditioning
par: Liu, Ziyue, et autres
Publié: (2026)

SAMix: Calibrated and Accurate Continual Learning via Sphere-Adaptive Mixup and Neural Collapse
par: Dang, Trung-Anh, et autres
Publié: (2025)

CUPID in the Model Zoo: Online Matchmaking for Selecting Your Dream LLM
par: Nguyen, Son, et autres
Publié: (2026)

Graph Neural Preconditioners for Iterative Solutions of Sparse Linear Systems
par: Chen, Jie
Publié: (2024)

Regime-Adaptive Bayesian Optimization via Dirichlet Process Mixtures of Gaussian Processes
par: Zhang, Yan, et autres
Publié: (2026)

Improving Probabilistic Diffusion Models With Optimal Diagonal Covariance Matching
par: Ou, Zijing, et autres
Publié: (2024)

Adaptive Estimation and Inference in Conditional Moment Models via the Discrepancy Principle
par: Tan, Jiyuan, et autres
Publié: (2026)

Gaussian Processes Sampling with Sparse Grids under Additive Schwarz Preconditioner
par: Chen, Haoyuan, et autres
Publié: (2024)

A Self-Attentive Meta-Optimizer with Group-Adaptive Learning Rates and Weight Decay
par: Zhao, JiangBo, et autres
Publié: (2026)

Learning Sparse Approximate Inverse Preconditioners for Conjugate Gradient Solvers on GPUs
par: Yang, Zherui, et autres
Publié: (2025)

Cautious Weight Decay
par: Chen, Lizhang, et autres
Publié: (2025)

A New Perspective on Shampoo's Preconditioner
par: Morwani, Depen, et autres
Publié: (2024)

Diagonal Over-parameterization in Reproducing Kernel Hilbert Spaces as an Adaptive Feature Model: Generalization and Adaptivity
par: Li, Yicheng, et autres
Publié: (2025)

Improving Deep Knowledge Tracing via Gated Architectures and Adaptive Optimization
par: Shukurlu, Altun
Publié: (2025)

Spectral Embeddings Leak Graph Topology: Theory, Benchmark, and Adaptive Reconstruction
par: Nguyen-Cong, Thinh, et autres
Publié: (2026)

Optimization Insights into Deep Diagonal Linear Networks
par: Labarrière, Hippolyte, et autres
Publié: (2024)

Diagonally-Weighted Generalized Method of Moments Estimation for Gaussian Mixture Modeling
par: Zhang, Liu, et autres
Publié: (2025)

Generative modeling of Sparse Approximate Inverse Preconditioners
par: Li, Mou, et autres
Publié: (2024)

Adaptive Moment Estimation Optimization Algorithm Using Projection Gradient for Deep Learning
par: Li, Yongqi, et autres
Publié: (2025)

Diagonal Adaptive Non-local Observables on Quantum Neural Networks
par: Tseng, Huan-Hsin, et autres
Publié: (2026)

Improving Rectified Flow with Boundary Conditions
par: Hu, Xixi, et autres
Publié: (2025)

GADPN: Graph Adaptive Denoising and Perturbation Networks via Singular Value Decomposition
par: Deng, Hao, et autres
Publié: (2026)

SKIM: Any-bit Quantization Pushing The Limits of Post-Training Quantization
par: Bai, Runsheng, et autres
Publié: (2024)

Adam Improves Muon: Adaptive Moment Estimation with Orthogonalized Momentum
par: Zhang, Minxin, et autres
Publié: (2026)

Preconditioners for the Stochastic Training of Neural Fields
par: Chng, Shin-Fang, et autres
Publié: (2024)