Saved in:
| Main Authors: | Zhang, Qi, Zhou, Yi, Zou, Shaofeng |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2404.01436 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Similar Items
Modeling AdaGrad, RMSProp, and Adam with Integro-Differential Equations
by: Heredia, Carlos
Published: (2024)
by: Heredia, Carlos
Published: (2024)
MGDA Converges under Generalized Smoothness, Provably
by: Zhang, Qi, et al.
Published: (2024)
by: Zhang, Qi, et al.
Published: (2024)
Adam-family Methods for Nonsmooth Optimization with Convergence Guarantees
by: Xiao, Nachuan, et al.
Published: (2023)
by: Xiao, Nachuan, et al.
Published: (2023)
Random Scaling and Momentum for Non-smooth Non-convex Optimization
by: Zhang, Qinzi, et al.
Published: (2024)
by: Zhang, Qinzi, et al.
Published: (2024)
Decentralized Non-convex Stochastic Optimization with Heterogeneous Variance
by: Chen, Hongxu, et al.
Published: (2026)
by: Chen, Hongxu, et al.
Published: (2026)
Convergence of Adam for Non-convex Objectives: Relaxed Hyperparameters and Non-ergodic Case
by: He, Meixuan, et al.
Published: (2023)
by: He, Meixuan, et al.
Published: (2023)
Optimal Stochastic Non-smooth Non-convex Optimization through Online-to-Non-convex Conversion
by: Cutkosky, Ashok, et al.
Published: (2023)
by: Cutkosky, Ashok, et al.
Published: (2023)
Adaptive Gradient Normalization and Independent Sampling for (Stochastic) Generalized-Smooth Optimization
by: Yang, Yufeng, et al.
Published: (2024)
by: Yang, Yufeng, et al.
Published: (2024)
Convergence of Spectral Descent for Non-smooth Optimization
by: Yang, Yixuan, et al.
Published: (2026)
by: Yang, Yixuan, et al.
Published: (2026)
On the Convergence of Adam under Non-uniform Smoothness: Separability from SGDM and Beyond
by: Wang, Bohan, et al.
Published: (2024)
by: Wang, Bohan, et al.
Published: (2024)
Riemannian Optimization for Non-convex Euclidean Distance Geometry with Global Recovery Guarantees
by: Smith, Chandler, et al.
Published: (2024)
by: Smith, Chandler, et al.
Published: (2024)
Adam-SHANG: A Convergent Adam-Type Method for Stochastic Smooth Convex Optimization
by: Yu, Yaxin, et al.
Published: (2026)
by: Yu, Yaxin, et al.
Published: (2026)
On Convergence of Adam for Stochastic Optimization under Relaxed Assumptions
by: Hong, Yusu, et al.
Published: (2024)
by: Hong, Yusu, et al.
Published: (2024)
Convergence of Steepest Descent and Adam under Non-Uniform Smoothness
by: Vaswani, Sharan, et al.
Published: (2026)
by: Vaswani, Sharan, et al.
Published: (2026)
Adam-HNAG: A Convergent Reformulation of Adam with Accelerated Rate
by: Yu, Yaxin, et al.
Published: (2026)
by: Yu, Yaxin, et al.
Published: (2026)
Efficient Sign-Based Optimization: Accelerating Convergence via Variance Reduction
by: Jiang, Wei, et al.
Published: (2024)
by: Jiang, Wei, et al.
Published: (2024)
On the Convergence of Adam-Type Algorithm for Bilevel Optimization under Unbounded Smoothness
by: Gong, Xiaochuan, et al.
Published: (2025)
by: Gong, Xiaochuan, et al.
Published: (2025)
Stochastic Compositional Minimax Optimization with Provable Convergence Guarantees
by: Deng, Yuyang, et al.
Published: (2024)
by: Deng, Yuyang, et al.
Published: (2024)
Subspace Optimization for Large Language Models with Convergence Guarantees
by: He, Yutong, et al.
Published: (2024)
by: He, Yutong, et al.
Published: (2024)
Online Non-convex Optimization with Long-term Non-convex Constraints
by: Pan, Shijie, et al.
Published: (2023)
by: Pan, Shijie, et al.
Published: (2023)
Convergence rates for the Adam optimizer
by: Dereich, Steffen, et al.
Published: (2024)
by: Dereich, Steffen, et al.
Published: (2024)
Adam Converges Without Any Modification On Update Rules
by: Zhang, Yushun, et al.
Published: (2026)
by: Zhang, Yushun, et al.
Published: (2026)
Quantization through Piecewise-Affine Regularization: Optimization and Statistical Guarantees
by: Ma, Jianhao, et al.
Published: (2025)
by: Ma, Jianhao, et al.
Published: (2025)
Beyond Bounded Variance: Variance-Reduced Normalized Methods for Nonconvex Optimization under Blum-Gladyshev Noise
by: Upadhyay, Antesh, et al.
Published: (2026)
by: Upadhyay, Antesh, et al.
Published: (2026)
Provable Adaptivity of Adam under Non-uniform Smoothness
by: Wang, Bohan, et al.
Published: (2022)
by: Wang, Bohan, et al.
Published: (2022)
Convergence and Complexity Guarantee for Inexact First-order Riemannian Optimization Algorithms
by: Li, Yuchen, et al.
Published: (2024)
by: Li, Yuchen, et al.
Published: (2024)
Understanding Adam Optimizer via Online Learning of Updates: Adam is FTRL in Disguise
by: Ahn, Kwangjun, et al.
Published: (2024)
by: Ahn, Kwangjun, et al.
Published: (2024)
HomeAdam: Adam and AdamW Algorithms Sometimes Go Home to Obtain Better Provable Generalization
by: Huang, Feihu, et al.
Published: (2026)
by: Huang, Feihu, et al.
Published: (2026)
Extended convexity and smoothness and their applications in deep learning
by: Qi, Binchuan, et al.
Published: (2024)
by: Qi, Binchuan, et al.
Published: (2024)
A Theoretical and Empirical Study on the Convergence of Adam with an "Exact" Constant Step Size in Non-Convex Settings
by: Mazumder, Alokendu, et al.
Published: (2023)
by: Mazumder, Alokendu, et al.
Published: (2023)
Nonconvex Stochastic Optimization under Heavy-Tailed Noises: Optimal Convergence without Gradient Clipping
by: Liu, Zijian, et al.
Published: (2024)
by: Liu, Zijian, et al.
Published: (2024)
Non-convex Stochastic Composite Optimization with Polyak Momentum
by: Gao, Yuan, et al.
Published: (2024)
by: Gao, Yuan, et al.
Published: (2024)
Memory-Reduced Meta-Learning with Guaranteed Convergence
by: Yang, Honglin, et al.
Published: (2024)
by: Yang, Honglin, et al.
Published: (2024)
MAP Estimation with Denoisers: Convergence Rates and Guarantees
by: Pesme, Scott, et al.
Published: (2025)
by: Pesme, Scott, et al.
Published: (2025)
A Comprehensive Framework for Analyzing the Convergence of Adam: Bridging the Gap with SGD
by: Jin, Ruinan, et al.
Published: (2024)
by: Jin, Ruinan, et al.
Published: (2024)
Retraction-Free Decentralized Non-convex Optimization with Orthogonal Constraints
by: Sun, Youbang, et al.
Published: (2024)
by: Sun, Youbang, et al.
Published: (2024)
Divergence Results and Convergence of a Variance Reduced Version of ADAM
by: Wang, Ruiqi, et al.
Published: (2022)
by: Wang, Ruiqi, et al.
Published: (2022)
Dynamic Regret via Discounted-to-Dynamic Reduction with Applications to Curved Losses and Adam Optimizer
by: Xie, Yan-Feng, et al.
Published: (2026)
by: Xie, Yan-Feng, et al.
Published: (2026)
Learning Over-Relaxation Policies for ADMM with Convergence Guarantees
by: Lin, Junan, et al.
Published: (2026)
by: Lin, Junan, et al.
Published: (2026)
Learning to Optimize for Mixed-Integer Non-linear Programming with Feasibility Guarantees
by: Tang, Bo, et al.
Published: (2024)
by: Tang, Bo, et al.
Published: (2024)
Similar Items
-
Modeling AdaGrad, RMSProp, and Adam with Integro-Differential Equations
by: Heredia, Carlos
Published: (2024) -
MGDA Converges under Generalized Smoothness, Provably
by: Zhang, Qi, et al.
Published: (2024) -
Adam-family Methods for Nonsmooth Optimization with Convergence Guarantees
by: Xiao, Nachuan, et al.
Published: (2023) -
Random Scaling and Momentum for Non-smooth Non-convex Optimization
by: Zhang, Qinzi, et al.
Published: (2024) -
Decentralized Non-convex Stochastic Optimization with Heterogeneous Variance
by: Chen, Hongxu, et al.
Published: (2026)