Cover Image

A short proof of near-linear convergence of adaptive gradient descent under fourth-order growth and convexity

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Davis, Damek, Drusvyatskiy, Dmitriy
Format:	Preprint
Published:	2026
Subjects:	Optimization and Control Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Gradient descent with adaptive stepsize converges (nearly) linearly under fourth-order growth
by: Davis, Damek, et al.
Published: (2024)

When do spectral gradient updates help in deep learning?
by: Davis, Damek, et al.
Published: (2025)

Iteratively reweighted kernel machines efficiently learn sparse functions
by: Zhu, Libin, et al.
Published: (2025)

Online Covariance Estimation in Nonsmooth Stochastic Approximation
by: Jiang, Liwei, et al.
Published: (2025)

What is the objective of reasoning with reinforcement learning?
by: Davis, Damek, et al.
Published: (2025)

Stochastic optimization over proximally smooth sets
by: Davis, Damek, et al.
Published: (2020)

On the global convergence of gradient descent for wide shallow models with bounded nonlinearities
by: Petit, Romain, et al.
Published: (2026)

Global convergence of gradient descent for phase retrieval
by: Fougereux, Théodore, et al.
Published: (2024)

On the stability of gradient descent with second order dynamics for time-varying cost functions
by: Gibson, Travis E., et al.
Published: (2024)

High-dimensional Limit of SGD for Diagonal Linear Networks
by: Malaxechebarría, Begoña García, et al.
Published: (2026)

Invariant Kernels: Rank Stabilization and Generalization Across Dimensions
by: Díaz, Mateo, et al.
Published: (2025)

Local linear convergence of gradient methods for overparameterized Gaussian mixtures
by: Wang, Jingxing, et al.
Published: (2026)

Problem-dependent convergence bounds for randomized linear gradient compression
by: Flynn, Thomas, et al.
Published: (2024)

Almost sure convergence rates of stochastic gradient methods under gradient domination
by: Weissmann, Simon, et al.
Published: (2024)

Stochastic Approximation with Decision-Dependent Distributions: Asymptotic Normality and Optimality
by: Cutler, Joshua, et al.
Published: (2022)

Non-convergence to global minimizers for Adam and stochastic gradient descent optimization and constructions of local minimizers in the training of artificial neural networks
by: Jentzen, Arnulf, et al.
Published: (2024)

Learning rate adaptive stochastic gradient descent optimization methods: numerical simulations for deep learning methods for partial differential equations and convergence analyses
by: Dereich, Steffen, et al.
Published: (2024)

Convergence of stochastic gradient descent under a local Lojasiewicz condition for deep neural networks
by: An, Jing, et al.
Published: (2023)

New logarithmic step size for stochastic gradient descent
by: Shamaee, M. Soheil, et al.
Published: (2024)

Linear convergence of proximal descent schemes on the Wasserstein space
by: Lascu, Razvan-Andrei, et al.
Published: (2024)

The duality structure gradient descent algorithm: analysis and applications to neural networks
by: Flynn, Thomas
Published: (2017)

A stochastic gradient descent algorithm with random search directions
by: Gbaguidi, Eméric
Published: (2025)

Convergence of continuous-time stochastic gradient descent with applications to deep neural networks
by: Lugosi, Gabor, et al.
Published: (2024)

Error dynamics of mini-batch gradient descent with random reshuffling for least squares regression
by: Lok, Jackie, et al.
Published: (2024)

A block-coordinate descent framework for non-convex composite optimization. Application to sparse precision matrix estimation
by: Lauga, Guillaume
Published: (2026)

Non-ergodic linear convergence property of the delayed gradient descent under the strongly convexity and the Polyak-Łojasiewicz condition
by: Choi, Hyung Jun, et al.
Published: (2023)

Learning linear dynamical systems under convex constraints
by: Tyagi, Hemant, et al.
Published: (2023)

Worst-case convergence analysis of relatively inexact gradient descent on smooth convex functions
by: Vernimmen, Pierre, et al.
Published: (2025)

A second-order-like optimizer with adaptive gradient scaling for deep learning
by: Bolte, Jérôme, et al.
Published: (2024)

Fast Spawn\&Prune (FS\&P): Global convergence of stochastic conic particle gradient descent via birth/death process
by: De Castro, Yohann, et al.
Published: (2026)

When majority rules, minority loses: bias amplification of gradient descent
by: Bachoc, François, et al.
Published: (2025)

Reinforcement learning for adaptive interior point methods in convex quadratic programming
by: Bertoncini, Jeremy, et al.
Published: (2025)

A simple and improved algorithm for noisy, convex, zeroth-order optimisation
by: Carpentier, Alexandra
Published: (2024)

Average Gradient Outer Product in kernel regression provably recovers the central subspace for multi-index models
by: Zhu, Libin, et al.
Published: (2026)

An accelerated first-order regularized momentum descent ascent algorithm for stochastic nonconvex-concave minimax problems
by: Zhang, Huiling, et al.
Published: (2023)

Manifold constrained steepest descent
by: Yang, Kaiwei, et al.
Published: (2026)

On the convergence analysis of the decentralized projected gradient descent method
by: Choi, Woocheol, et al.
Published: (2023)

Learning to optimize with guarantees: a complete characterization of linearly convergent algorithms
by: Martin, Andrea, et al.
Published: (2025)

Flattened one-bit stochastic gradient descent: compressed distributed optimization with controlled variance
by: Stollenwerk, Alexander, et al.
Published: (2024)

Convergence of flow-based generative models via proximal gradient descent in Wasserstein space
by: Cheng, Xiuyuan, et al.
Published: (2023)