Image de couverture de livre

Convergence rates of stochastic gradient method with independent sequences of step-size and momentum weight

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteur principal:	Hwang, Wen-Liang
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning Optimization and Control
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

New logarithmic step size for stochastic gradient descent
par: Shamaee, M. Soheil, et autres
Publié: (2024)

Convergence of projected stochastic natural gradient variational inference for various step size and sample or batch size schedules
par: Guilmeau, Thomas, et autres
Publié: (2026)

Almost sure convergence rates of stochastic gradient methods under gradient domination
par: Weissmann, Simon, et autres
Publié: (2024)

A stochastic gradient method for trilevel optimization
par: Giovannelli, Tommaso, et autres
Publié: (2025)

Convergence of continuous-time stochastic gradient descent with applications to deep neural networks
par: Lugosi, Gabor, et autres
Publié: (2024)

Convergence and concentration properties of constant step-size SGD through Markov chains
par: Merad, Ibrahim, et autres
Publié: (2023)

Convergence of stochastic gradient descent under a local Lojasiewicz condition for deep neural networks
par: An, Jing, et autres
Publié: (2023)

Unregularized limit of stochastic gradient method for Wasserstein distributionally robust optimization
par: Le, Tam
Publié: (2025)

A theoretical and empirical study of new adaptive algorithms with additional momentum steps and shifted updates for stochastic non-convex optimization
par: Alecsa, Cristian Daniel
Publié: (2021)

The generator gradient estimator is an adjoint state method for stochastic differential equations
par: Badolle, Quentin, et autres
Publié: (2024)

Projected gradient methods for nonconvex and stochastic smooth optimization: new complexities and auto-conditioned stepsizes
par: Lan, Guanghui, et autres
Publié: (2024)

Generalized EXTRA stochastic gradient Langevin dynamics
par: Gurbuzbalaban, Mert, et autres
Publié: (2024)

Learning rate adaptive stochastic gradient descent optimization methods: numerical simulations for deep learning methods for partial differential equations and convergence analyses
par: Dereich, Steffen, et autres
Publié: (2024)

Dealing with unbounded gradients in stochastic saddle-point optimization
par: Neu, Gergely, et autres
Publié: (2024)

Convergence of gradient flow for learning convolutional neural networks
par: Diederen, Jona-Maria, et autres
Publié: (2026)

Convergence of SGD with momentum in the nonconvex case: A time window-based analysis
par: Qiu, Junwen, et autres
Publié: (2024)

Convergence and sample complexity of natural policy gradient primal-dual methods for constrained MDPs
par: Ding, Dongsheng, et autres
Publié: (2022)

Exact Convergence rate of the subgradient method by using Polyak step size
par: Zamani, Moslem, et autres
Publié: (2024)

Shuffling the Data, Stretching the Step-size: Sharper Bias in constant step-size SGD
par: Emmanouilidis, Konstantinos, et autres
Publié: (2026)

An accelerated first-order regularized momentum descent ascent algorithm for stochastic nonconvex-concave minimax problems
par: Zhang, Huiling, et autres
Publié: (2023)

Non-asymptotic convergence analysis of the stochastic gradient Hamiltonian Monte Carlo algorithm with discontinuous stochastic gradient with applications to training of ReLU neural networks
par: Liang, Luxu, et autres
Publié: (2024)

Convergence of the momentum method for semialgebraic functions with locally Lipschitz gradients
par: Josz, Cédric, et autres
Publié: (2023)

A stochastic gradient descent algorithm with random search directions
par: Gbaguidi, Eméric
Publié: (2025)

Convergence rates for the Adam optimizer
par: Dereich, Steffen, et autres
Publié: (2024)

Auto-conditioned primal-dual hybrid gradient method and alternating direction method of multipliers
par: Lan, Guanghui, et autres
Publié: (2024)

Sparse Polyak: an adaptive step size rule for high-dimensional M-estimation
par: Qiao, Tianqi, et autres
Publié: (2025)

Local linear convergence of gradient methods for overparameterized Gaussian mixtures
par: Wang, Jingxing, et autres
Publié: (2026)

Glocal Smoothness: Line search and adaptive step sizes can help in theory too!
par: Fox, Curtis, et autres
Publié: (2025)

Robust stochastic first order methods in heavy-tailed noise via medoid mini-batch gradient sampling
par: Vukovic, Manojlo, et autres
Publié: (2026)

Stochastic Polyak Step-sizes and Momentum: Convergence Guarantees and Practical Performance
par: Oikonomou, Dimitris, et autres
Publié: (2024)

Empirical and computer-aided robustness analysis of long-step and accelerated methods in smooth convex optimization
par: Vernimmen, Pierre, et autres
Publié: (2025)

Non-Asymptotic Global Convergence of PPO-Clip
par: Liu, Yin, et autres
Publié: (2025)

Non-convergence to global minimizers for Adam and stochastic gradient descent optimization and constructions of local minimizers in the training of artificial neural networks
par: Jentzen, Arnulf, et autres
Publié: (2024)

Adaptive multi-gradient methods for quasiconvex vector optimization and applications to multi-task learning
par: Minh, Nguyen Anh, et autres
Publié: (2024)

Convergence Analysis of Stochastic Gradient Descent with MCMC Estimators
par: Li, Tianyou, et autres
Publié: (2023)

A Provably Convergent and Practical Algorithm for Gromov--Wasserstein Optimal Transport
par: Liang, Ling, et autres
Publié: (2026)

On improving generalization in a class of learning problems with the method of small parameters for weakly-controlled optimal gradient systems
par: Befekadu, Getachew K.
Publié: (2024)

Exponential convergence rates for momentum stochastic gradient descent in the overparametrized setting
par: Gess, Benjamin, et autres
Publié: (2023)

Flattened one-bit stochastic gradient descent: compressed distributed optimization with controlled variance
par: Stollenwerk, Alexander, et autres
Publié: (2024)

Tradeoffs between convergence rate and noise amplification for momentum-based accelerated optimization algorithms
par: Mohammadi, Hesameddin, et autres
Publié: (2022)