Copertina

Implicit Bias of AdamW: $\ell_\infty$ Norm Constrained Optimization

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Xie, Shuo, Li, Zhiyuan
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Machine Learning Optimization and Control
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

On the $O(\frac{\sqrt{d}}{K^{1/4}})$ Convergence Rate of AdamW Measured by $\ell_1$ Norm
di: Li, Huan, et al.
Pubblicazione: (2025)

Optimizer-Induced Mode Connectivity: From AdamW to Muon
di: Zhang, Fangzhao, et al.
Pubblicazione: (2026)

HomeAdam: Adam and AdamW Algorithms Sometimes Go Home to Obtain Better Provable Generalization
di: Huang, Feihu, et al.
Pubblicazione: (2026)

Convergence Rate Analysis of the AdamW-Style Shampoo: Unifying One-Sided and Two-Sided Preconditioning
di: Li, Huan, et al.
Pubblicazione: (2026)

The Rich and the Simple: On the Implicit Bias of Adam and SGD
di: Vasudeva, Bhavya, et al.
Pubblicazione: (2025)

Towards The Implicit Bias on Multiclass Separable Data Under Norm Constraints
di: Xie, Shengping, et al.
Pubblicazione: (2026)

On the Implicit Bias of Adam
di: Cattaneo, Matias D., et al.
Pubblicazione: (2023)

The Effect of Mini-Batch Noise on the Implicit Bias of Adam
di: Cattaneo, Matias D., et al.
Pubblicazione: (2026)

A Determinantal Approach to a Sharp $\ell^1-\ell^\infty-\ell^2$ Norm Inequality
di: Benitez, Jose Antonio Lara
Pubblicazione: (2026)

Clipping Improves Adam-Norm and AdaGrad-Norm when the Noise Is Heavy-Tailed
di: Chezhegov, Savelii, et al.
Pubblicazione: (2024)

Training Deep Learning Models with Norm-Constrained LMOs
di: Pethick, Thomas, et al.
Pubblicazione: (2025)

Implicit Bias of Per-sample Adam on Separable Data: Departure from the Full-batch Regime
di: Baek, Beomhan, et al.
Pubblicazione: (2025)

De-singularity Subgradient for the $q$-th-Powered $\ell_p$-Norm Weber Location Problem
di: Lai, Zhao-Rong, et al.
Pubblicazione: (2024)

Where Does Warm-Up Come From? Adaptive Scheduling for Norm-Constrained Optimizers
di: Riabinin, Artem, et al.
Pubblicazione: (2026)

Near-Linear Time Projection onto the $\ell_{1,\infty}$ Ball; Application to Sparse Autoencoders
di: Perez, Guillaume, et al.
Pubblicazione: (2023)

Understanding Adam Optimizer via Online Learning of Updates: Adam is FTRL in Disguise
di: Ahn, Kwangjun, et al.
Pubblicazione: (2024)

Adam-SHANG: A Convergent Adam-Type Method for Stochastic Smooth Convex Optimization
di: Yu, Yaxin, et al.
Pubblicazione: (2026)

Implicit Bias of Mirror Flow on Separable Data
di: Pesme, Scott, et al.
Pubblicazione: (2024)

Muon Optimizes Under Spectral Norm Constraints
di: Chen, Lizhang, et al.
Pubblicazione: (2025)

Dynamic Regret via Discounted-to-Dynamic Reduction with Applications to Curved Losses and Adam Optimizer
di: Xie, Yan-Feng, et al.
Pubblicazione: (2026)

Implicit Bias and Fast Convergence Rates for Self-attention
di: Vasudeva, Bhavya, et al.
Pubblicazione: (2024)

Implicit Bias and Convergence of Matrix Stochastic Mirror Descent
di: Akhtiamov, Danil, et al.
Pubblicazione: (2026)

Old Optimizer, New Norm: An Anthology
di: Bernstein, Jeremy, et al.
Pubblicazione: (2024)

On Convergence of Adam for Stochastic Optimization under Relaxed Assumptions
di: Hong, Yusu, et al.
Pubblicazione: (2024)

Adam-family Methods for Nonsmooth Optimization with Convergence Guarantees
di: Xiao, Nachuan, et al.
Pubblicazione: (2023)

Implicit Bias of Gradient Descent for Non-Homogeneous Deep Networks
di: Cai, Yuhang, et al.
Pubblicazione: (2025)

Implicit Bias of Spectral Descent and Muon on Multiclass Separable Data
di: Fan, Chen, et al.
Pubblicazione: (2025)

Convergence and Implicit Bias of Gradient Descent on Continual Linear Classification
di: Jung, Hyunji, et al.
Pubblicazione: (2025)

Small Gradient Norm Regret for Online Convex Optimization
di: Gao, Wenzhi, et al.
Pubblicazione: (2026)

Adam-HNAG: A Convergent Reformulation of Adam with Accelerated Rate
di: Yu, Yaxin, et al.
Pubblicazione: (2026)

On the Convergence of Adam-Type Algorithm for Bilevel Optimization under Unbounded Smoothness
di: Gong, Xiaochuan, et al.
Pubblicazione: (2025)

Implicit Bias in Matrix Factorization and its Explicit Realization in a New Architecture
di: Hou, Yikun, et al.
Pubblicazione: (2025)

Adam on Local Time: Addressing Nonstationarity in RL with Relative Adam Timesteps
di: Ellis, Benjamin, et al.
Pubblicazione: (2024)

The Implicit Bias of Heterogeneity towards Invariance: A Study of Multi-Environment Matrix Sensing
di: Xu, Yang, et al.
Pubblicazione: (2024)

An Iteratively Reweighted Method for Sparse Optimization on Nonconvex $\ell_{p}$ Ball
di: Wang, Hao, et al.
Pubblicazione: (2021)

Convergence Guarantees for RMSProp and Adam in Generalized-smooth Non-convex Optimization with Affine Noise Variance
di: Zhang, Qi, et al.
Pubblicazione: (2024)

Scale-Invariant Neural Network Optimization: Norm Geometry and Heavy-Tailed Noise
di: Zhang, Jiayu, et al.
Pubblicazione: (2026)

The Ky Fan Norms and Beyond: Dual Norms and Combinations for Matrix Optimization
di: Kravatskiy, Alexey, et al.
Pubblicazione: (2025)

A Sequential Quadratic Programming Method with High Probability Complexity Bounds for Nonlinear Equality Constrained Stochastic Optimization
di: Berahas, Albert S., et al.
Pubblicazione: (2023)

HUANet: Hard-Constrained Unrolled ADMM for Constrained Convex Optimization
di: Tran, Trinh, et al.
Pubblicazione: (2026)