Cover Image

Convergence of the Iterates for Momentum and RMSProp for Local Smooth Functions: Adaptation is the Key

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Bensaid, Bilel, Poëtte, Gaël, Turpault, Rodolphe
Format:	Preprint
Published:	2024
Subjects:	Optimization and Control
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

An Abstract Lyapunov Control Optimizer: Local Stabilization and Global Convergence
by: Bensaid, Bilel, et al.
Published: (2024)

Complexities of Armijo-like algorithms in Deep Learning context
by: Bilel, Bensaid
Published: (2024)

On the $O(\frac{\sqrt{d}}{T^{1/4}})$ Convergence Rate of RMSProp and Its Momentum Extension Measured by $\ell_1$ Norm
by: Li, Huan, et al.
Published: (2024)

Locally Linear Convergence for Nonsmooth Convex Optimization via Coupled Smoothing and Momentum
by: Baghbadorani, Reza Rahimi, et al.
Published: (2025)

Global Stability and Step Size Robustness of RMSProp
by: Dimitrieski, Naum, et al.
Published: (2026)

Convergence Guarantees for RMSProp and Adam in Generalized-smooth Non-convex Optimization with Affine Noise Variance
by: Zhang, Qi, et al.
Published: (2024)

Last Iterate Convergence of AdaGrad-Norm for Convex Non-Smooth Optimization
by: Preobrazhenskaia, Margarita, et al.
Published: (2026)

Fast Last-Iterate Convergence of SGD in the Smooth Interpolation Regime
by: Attia, Amit, et al.
Published: (2025)

Modeling AdaGrad, RMSProp, and Adam with Integro-Differential Equations
by: Heredia, Carlos
Published: (2024)

Convergence of Clipped SGD on Convex $(L_0,L_1)$-Smooth Functions
by: Gaash, Ofir, et al.
Published: (2025)

Clipped Stochastic Gradient Tracking For Locally Smooth Functions
by: Mei, Leilei, et al.
Published: (2026)

Random Reshuffling with Momentum for Nonconvex Problems: Iteration Complexity and Last Iterate Convergence
by: Qiu, Junwen, et al.
Published: (2024)

Generalized Stochastic Gradient Descent with Momentum Methods for Smooth Optimization
by: Wang, Zimeng, et al.
Published: (2026)

On Convergence of Incremental Gradient for Non-Convex Smooth Functions
by: Koloskova, Anastasia, et al.
Published: (2023)

On the Convergence of the Accelerated Riccati Iteration Method
by: Rajasingam, Prasanthan, et al.
Published: (2017)

Improved Iteration Complexity in Black-Box Optimization Problems under Higher Order Smoothness Function Condition
by: Lobanov, Aleksandr
Published: (2024)

Generalized Smooth Stochastic Variational Inequalities: Almost Sure Convergence and Convergence Rates
by: Vankov, Daniil, et al.
Published: (2024)

Convergence Analysis of the Last Iterate in Distributed Stochastic Gradient Descent with Momentum
by: Cheng, Difei, et al.
Published: (2025)

Convergence of the Iterates of the Stochastic Proximal Gradient Method
by: Madariaga, Javier I.
Published: (2026)

Last-Iterate Convergence of Anchored Gradient Descent
by: Cai, Yang, et al.
Published: (2026)

Dual Averaging Converges for Nonconvex Smooth Stochastic Optimization
by: Liu, Tuo, et al.
Published: (2025)

Last-Iterate Complexity of SGD for Convex and Smooth Stochastic Problems
by: Garrigos, Guillaume, et al.
Published: (2025)

On Convergence of the Iteratively Preconditioned Gradient-Descent (IPG) Observer
by: Chakrabarti, Kushal, et al.
Published: (2024)

The Iterates of Nesterov's Accelerated Algorithm Converge in The Critical Regimes
by: Bot, Radu Ioan, et al.
Published: (2025)

Learning-rate-free Momentum SGD with Reshuffling Converges in Nonsmooth Nonconvex Optimization
by: Hu, Xiaoyin, et al.
Published: (2024)

Information Geometry of Exponentiated Gradient: Convergence beyond L-Smoothness
by: Elshiaty, Yara, et al.
Published: (2025)

An Effective Iterative Solution for Independent Vector Analysis with Convergence Guarantees
by: Cosserat, Clément, et al.
Published: (2024)

Computing Competitive Equilibrium for Chores: Linear Convergence and Lightweight Iteration
by: Chen, He, et al.
Published: (2024)

Convergence of First-Order Algorithms with Momentum from the Perspective of an Inexact Gradient Descent Method
by: Khanh, Pham Duy, et al.
Published: (2025)

The Anytime Convergence of Stochastic Gradient Descent with Momentum: From a Continuous-Time Perspective
by: Feng, Yasong, et al.
Published: (2023)

Abs-Smooth Frank-Wolfe Method: Primal-Dual Analysis, Heavy Ball Momentum, and Inexact Oracles
by: Tadinada, Sri Harshitha, et al.
Published: (2026)

Convergence Analysis of Stochastic Accelerated Gradient Methods for Generalized Smooth Optimizations
by: Yu, Chenhao, et al.
Published: (2025)

Physics-Informed Neural Network Policy Iteration: Algorithms, Convergence, and Verification
by: Meng, Yiming, et al.
Published: (2024)

Distributed Stochastic Momentum Tracking with Local Updates: Achieving Optimal Communication and Iteration Complexities
by: Huang, Kun, et al.
Published: (2025)

MGDA Converges under Generalized Smoothness, Provably
by: Zhang, Qi, et al.
Published: (2024)

Directional Smoothness and Gradient Methods: Convergence and Adaptivity
by: Mishkin, Aaron, et al.
Published: (2024)

Improved Convergence in Parameter-Agnostic Error Feedback through Momentum
by: Sadiev, Abdurakhmon, et al.
Published: (2025)

Passive and reciprocal linear time-and-space-invariant systems
by: Shali, Brayan M., et al.
Published: (2025)

Last Iterate Convergence of Popov Method for Non-monotone Stochastic Variational Inequalities
by: Vankov, Daniil, et al.
Published: (2023)

Probabilistic Framework of Howard's Policy Iteration: BML Evaluation and Robust Convergence Analysis
by: Wang, Yutian, et al.
Published: (2022)