Cover Image

Provable Adaptivity of Adam under Non-uniform Smoothness

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wang, Bohan, Zhang, Yushun, Zhang, Huishuai, Meng, Qi, Sun, Ruoyu, Ma, Zhi-Ming, Liu, Tie-Yan, Luo, Zhi-Quan, Chen, Wei
Format:	Preprint
Published:	2022
Subjects:	Machine Learning Optimization and Control
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

On the Convergence of Adam under Non-uniform Smoothness: Separability from SGDM and Beyond
by: Wang, Bohan, et al.
Published: (2024)

Adam Converges Without Any Modification On Update Rules
by: Zhang, Yushun, et al.
Published: (2026)

A Single-Loop Smoothed Gradient Descent-Ascent Algorithm for Nonconvex-Concave Min-Max Problems
by: Zhang, Jiawei, et al.
Published: (2020)

Finite Horizon Optimization: Framework and Applications
by: Zhang, Yushun, et al.
Published: (2024)

MGDA Converges under Generalized Smoothness, Provably
by: Zhang, Qi, et al.
Published: (2024)

Convergence of Steepest Descent and Adam under Non-Uniform Smoothness
by: Vaswani, Sharan, et al.
Published: (2026)

A Stochastic Quasi-Newton Method for Non-convex Optimization with Non-uniform Smoothness
by: Sun, Zhenyu, et al.
Published: (2024)

Towards Quantifying the Hessian Structure of Neural Networks
by: Dong, Zhaorui, et al.
Published: (2025)

HomeAdam: Adam and AdamW Algorithms Sometimes Go Home to Obtain Better Provable Generalization
by: Huang, Feihu, et al.
Published: (2026)

Gradient-Variation Online Learning under Generalized Smoothness
by: Xie, Yan-Feng, et al.
Published: (2024)

Gradient Descent with Polyak's Momentum Finds Flatter Minima via Large Catapults
by: Phunyaphibarn, Prin, et al.
Published: (2023)

Dynamic Regret via Discounted-to-Dynamic Reduction with Applications to Curved Losses and Adam Optimizer
by: Xie, Yan-Feng, et al.
Published: (2026)

On the Convergence of Adam-Type Algorithm for Bilevel Optimization under Unbounded Smoothness
by: Gong, Xiaochuan, et al.
Published: (2025)

Provable Reduction in Communication Rounds for Non-Smooth Convex Federated Learning
by: Palenzuela, Karlo, et al.
Published: (2025)

Nonlinearly Preconditioned Gradient Methods under Generalized Smoothness
by: Oikonomidis, Konstantinos, et al.
Published: (2025)

Solving contextual chance-constrained programming under decision-dependent uncertainty
by: Liu, Xiangting, et al.
Published: (2026)

Provably Convergent Decentralized Optimization over Directed Graphs under Generalized Smoothness
by: Bo, Yanan, et al.
Published: (2026)

Convergence Guarantees for RMSProp and Adam in Generalized-smooth Non-convex Optimization with Affine Noise Variance
by: Zhang, Qi, et al.
Published: (2024)

Adam-SHANG: A Convergent Adam-Type Method for Stochastic Smooth Convex Optimization
by: Yu, Yaxin, et al.
Published: (2026)

Adaptive Methods for Variational Inequalities under Relaxed Smoothness Assumption
by: Vankov, Daniil, et al.
Published: (2024)

Distributed Stochastic Optimization for Non-Smooth and Weakly Convex Problems under Heavy-Tailed Noise
by: Hu, Jun, et al.
Published: (2025)

An Adaptive Smoothing Algorithm for Non-Lipschitz Optimization on Manifolds with Complexity Guarantees
by: Wang, Lei, et al.
Published: (2026)

Optimal Abort Policy for Mission-Critical Systems under Imperfect Condition Monitoring
by: Sun, Qiuzhuang, et al.
Published: (2025)

Understanding Adam Optimizer via Online Learning of Updates: Adam is FTRL in Disguise
by: Ahn, Kwangjun, et al.
Published: (2024)

Decentralized Gradient-Free Methods for Stochastic Non-Smooth Non-Convex Optimization
by: Lin, Zhenwei, et al.
Published: (2023)

A decomposition method in the multivariate feedback particle filter via tensor product Hermite polynomials
by: Wang, Ruoyu, et al.
Published: (2025)

How to Set $β_1, β_2$ in Adam: An Online Learning Perspective
by: Nguyen, Quan
Published: (2025)

Complexity Lower Bounds of Adaptive Gradient Algorithms for Non-convex Stochastic Optimization under Relaxed Smoothness
by: Crawshaw, Michael, et al.
Published: (2025)

Decentralized Stochastic Nonconvex Optimization under the Relaxed Smoothness
by: Luo, Luo, et al.
Published: (2025)

AdaGrad under Anisotropic Smoothness
by: Liu, Yuxing, et al.
Published: (2024)

Smoothed Proximal Lagrangian Method for Nonlinear Constrained Programs
by: Pu, Wenqiang, et al.
Published: (2024)

On the Complexity of Finite-Sum Smooth Optimization under the Polyak-Łojasiewicz Condition
by: Bai, Yunyan, et al.
Published: (2024)

Newsvendor under Ambiguity and Misspecification
by: Liu, Feng, et al.
Published: (2024)

ROS: A GNN-based Relax-Optimize-and-Sample Framework for Max-k-Cut Problems
by: Qiu, Yeqing, et al.
Published: (2024)

Error Estimates of the Gain Approximation by Hermite-Galerkin Method in Feedback Particle Filter
by: Wang, Ruoyu, et al.
Published: (2026)

On Relatively Smooth Optimization over Riemannian Manifolds
by: He, Chang, et al.
Published: (2025)

Adaptive Robust Control for Uncertain Systems with Ellipsoid-Set Learning
by: Ma, Xuehui, et al.
Published: (2026)

AdamFlow: Adam-based Wasserstein Gradient Flows for Surface Registration in Medical Imaging
by: Ma, Qiang, et al.
Published: (2026)

Modeling the Curbside Congestion Effects of Ride-hailing Services for Morning Commute using Bi-modal Two-Tandem Bottlenecks
by: Deng, Yao, et al.
Published: (2025)

Adaptive Accelerated Gradient Method for Smooth Convex Optimization
by: Wang, Zepeng, et al.
Published: (2025)