Image de couverture de livre

A Theoretical and Empirical Study on the Convergence of Adam with an "Exact" Constant Step Size in Non-Convex Settings

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Mazumder, Alokendu, Sabharwal, Rishabh, Tayal, Manan, Kumar, Bhartendu, Rathore, Punit
Format:	Preprint
Publié:	2023
Sujets:	Machine Learning Optimization and Control
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

On the Convergence of Projected Policy Gradient for Any Constant Step Sizes
par: Liu, Jiacai, et autres
Publié: (2023)

Adam-SHANG: A Convergent Adam-Type Method for Stochastic Smooth Convex Optimization
par: Yu, Yaxin, et autres
Publié: (2026)

Zeroth-Order Non-Convex Optimization for Cooperative Multi-Agent Systems with Diminishing Step Size and Smoothing Radius
par: Zheng, Xinran, et autres
Publié: (2024)

An Exact Penalty Approach for Equality Constrained Optimization over a Convex Set
par: Xiao, Nachuan, et autres
Publié: (2025)

Monitoring the Convergence Speed of PDHG to Find Better Primal and Dual Step Sizes
par: Fercoq, Olivier
Publié: (2024)

Convergence of Steepest Descent and Adam under Non-Uniform Smoothness
par: Vaswani, Sharan, et autres
Publié: (2026)

Global Convergence of Control-Based Lagrangian Flows for Non-Convex Optimization
par: Pirrera, Simone, et autres
Publié: (2026)

On Column sufficiency and Extended Horizontal Linear Complementarity Problem
par: Yadav, Punit Kumar, et autres
Publié: (2025)

Almost Sure Convergence of Stochastic Approximation: An Interplay of Noise and Step Size
par: Nguyen, Quang Dinh Thien, et autres
Publié: (2026)

Adaptive Open-Loop Step-Sizes for Accelerated Convergence Rates of the Frank-Wolfe Algorithm
par: Wirth, Elias, et autres
Publié: (2025)

Convex and Non-convex Federated Learning with Stale Stochastic Gradients: Diminishing Step Size is All You Need
par: Zheng, Xinran, et autres
Publié: (2026)

Exact Instance Compression for Convex Empirical Risk Minimization via Color Refinement
par: Zhu, Bryan, et autres
Publié: (2026)

Non-asymptotic Global Convergence Rates of BFGS with Exact Line Search
par: Jin, Qiujiang, et autres
Publié: (2024)

Adam-HNAG: A Convergent Reformulation of Adam with Accelerated Rate
par: Yu, Yaxin, et autres
Publié: (2026)

Last Iterate Convergence of AdaGrad-Norm for Convex Non-Smooth Optimization
par: Preobrazhenskaia, Margarita, et autres
Publié: (2026)

Accelerated Affine-Invariant Convergence Rates of the Frank-Wolfe Algorithm with Open-Loop Step-Sizes
par: Wirth, Elias, et autres
Publié: (2023)

Optimal Convergence Rate for Mirror Descent Methods with special Time-Varying Step Sizes Rules
par: Alkousa, Mohammad, et autres
Publié: (2024)

On Convergence of Incremental Gradient for Non-Convex Smooth Functions
par: Koloskova, Anastasia, et autres
Publié: (2023)

On the Convergence of Adam under Non-uniform Smoothness: Separability from SGDM and Beyond
par: Wang, Bohan, et autres
Publié: (2024)

Exact Flatness Constant for One-Point Convex Bodies and the Discrete Isominwidth Problem: The Planar Case
par: Averkov, Gennadiy, et autres
Publié: (2026)

MOSS: Multi-Objective Optimization for Stable Rule Sets
par: Liu, Brian, et autres
Publié: (2025)

What are Capra-Convex Sets?
par: Chancelier, Jean-Philippe, et autres
Publié: (2025)

Convergence rates for the Adam optimizer
par: Dereich, Steffen, et autres
Publié: (2024)

On Convergence and Stability of Two Extended BB-like Step Sizes
par: Xu, Xin
Publié: (2025)

Convergence of Adam for Non-convex Objectives: Relaxed Hyperparameters and Non-ergodic Case
par: He, Meixuan, et autres
Publié: (2023)

Structured eigenvalue backward errors of Rosenbrock systems and related $μ$-value problems
par: Prajapati, Anshul, et autres
Publié: (2024)

UAdam: Unified Adam-Type Algorithmic Framework for Non-Convex Stochastic Optimization
par: Jiang, Yiming, et autres
Publié: (2023)

Stochastic Gradient Descent in Non-Convex Problems: Asymptotic Convergence with Relaxed Step-Size via Stopping Time Methods
par: Jin, Ruinan, et autres
Publié: (2025)

Strong Convexity of Sets in Riemannian Manifolds
par: Scieur, Damien, et autres
Publié: (2023)

Convergence Guarantees for RMSProp and Adam in Generalized-smooth Non-convex Optimization with Affine Noise Variance
par: Zhang, Qi, et autres
Publié: (2024)

Adam-family Methods for Nonsmooth Optimization with Convergence Guarantees
par: Xiao, Nachuan, et autres
Publié: (2023)

On Convergence of Adam for Stochastic Optimization under Relaxed Assumptions
par: Hong, Yusu, et autres
Publié: (2024)

Adam Converges Without Any Modification On Update Rules
par: Zhang, Yushun, et autres
Publié: (2026)

Linear Convergence of Plug-and-Play Algorithms with Kernel Denoisers
par: Sinha, Arghya, et autres
Publié: (2025)

Generalized Alternating Projections on Manifolds and Convex Sets
par: Fält, Mattias, et autres
Publié: (2021)

A Proof of the Exact Convergence Rate of Gradient Descent
par: Kim, Jungbin
Publié: (2024)

Global Stability and Step Size Robustness of RMSProp
par: Dimitrieski, Naum, et autres
Publié: (2026)

Non-Convex Robust Hypothesis Testing using Sinkhorn Uncertainty Sets
par: Wang, Jie, et autres
Publié: (2024)

Last-Iterate Convergence of Randomized Kaczmarz and SGD with Greedy Step Size
par: Dereziński, Michał, et autres
Publié: (2026)

Multichannel Frequency Estimation with Constant Amplitude via Convex Structured Low-Rank Approximation
par: Wu, Xunmeng, et autres
Publié: (2024)