Copertina

First and Second Order Approximations to Stochastic Gradient Descent Methods with Momentum Terms

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autore principale:	Lu, Eric
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning Optimization and Control
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Momentum Does Not Reduce Stochastic Noise in Stochastic Gradient Descent
di: Sato, Naoki, et al.
Pubblicazione: (2024)

Policy Gradient with Second Order Momentum
di: Sun, Tianyu
Pubblicazione: (2025)

First-ish Order Methods: Hessian-aware Scalings of Gradient Descent
di: Smee, Oscar, et al.
Pubblicazione: (2025)

On Penalty Methods for Nonconvex Bilevel Optimization and First-Order Stochastic Approximation
di: Kwon, Jeongyeol, et al.
Pubblicazione: (2023)

Better LMO-based Momentum Methods with Second-Order Information
di: Khirirat, Sarit, et al.
Pubblicazione: (2025)

Algorithmic Stability of Stochastic Gradient Descent with Momentum under Heavy-Tailed Noise
di: Dang, Thanh, et al.
Pubblicazione: (2025)

Stochastic Adaptive Gradient Descent Without Descent
di: Aujol, Jean-François, et al.
Pubblicazione: (2025)

Compressed Decentralized Momentum Stochastic Gradient Methods for Nonconvex Optimization
di: Liu, Wei, et al.
Pubblicazione: (2025)

More Optimal Fractional-Order Stochastic Gradient Descent for Non-Convex Optimization Problems
di: Partohaghighi, Mohammad, et al.
Pubblicazione: (2025)

Effective Dimension Aware Fractional-Order Stochastic Gradient Descent for Convex Optimization Problems
di: Partohaghighi, Mohammad, et al.
Pubblicazione: (2025)

Stochastic Gradient Descent with Strategic Querying
di: Jiang, Nanfei, et al.
Pubblicazione: (2025)

Stochastic Gradient Descent with Adaptive Data
di: Che, Ethan, et al.
Pubblicazione: (2024)

Derivatives of Stochastic Gradient Descent in parametric optimization
di: Iutzeler, Franck, et al.
Pubblicazione: (2024)

Weighted Low-rank Approximation via Stochastic Gradient Descent on Manifolds
di: Xu, Conglong, et al.
Pubblicazione: (2025)

Gradient Descent with Polyak's Momentum Finds Flatter Minima via Large Catapults
di: Phunyaphibarn, Prin, et al.
Pubblicazione: (2023)

On the Complexity of First-Order Methods in Stochastic Bilevel Optimization
di: Kwon, Jeongyeol, et al.
Pubblicazione: (2024)

Gradient Descent, Stochastic Optimization, and Other Tales
di: Lu, Jun
Pubblicazione: (2022)

On the Inherent Privacy of Zeroth Order Projected Gradient Descent
di: Gupta, Devansh, et al.
Pubblicazione: (2025)

Convergence Analysis of Stochastic Gradient Descent with MCMC Estimators
di: Li, Tianyou, et al.
Pubblicazione: (2023)

The Sample Complexity of Gradient Descent in Stochastic Convex Optimization
di: Livni, Roi
Pubblicazione: (2024)

Adaptive Step Sizes for Preconditioned Stochastic Gradient Descent
di: Köhne, Frederik, et al.
Pubblicazione: (2023)

Functional Central Limit Theorem for Stochastic Gradient Descent
di: Flamand, Kessang, et al.
Pubblicazione: (2026)

Parameter Symmetry and Noise Equilibrium of Stochastic Gradient Descent
di: Ziyin, Liu, et al.
Pubblicazione: (2024)

Adaptive Batch Size and Learning Rate Scheduler for Stochastic Gradient Descent Based on Minimization of Stochastic First-order Oracle Complexity
di: Umeda, Hikaru, et al.
Pubblicazione: (2025)

Rapid Overfitting of Multi-Pass Stochastic Gradient Descent in Stochastic Convex Optimization
di: Vansover-Hager, Shira, et al.
Pubblicazione: (2025)

Exponential Convergence of (Stochastic) Gradient Descent for Separable Logistic Regression
di: Kale, Sacchit, et al.
Pubblicazione: (2026)

Faster Convergence of Stochastic Accelerated Gradient Descent under Interpolation
di: Mishkin, Aaron, et al.
Pubblicazione: (2024)

On the Convergence of Stochastic Gradient Descent with Perturbed Forward-Backward Passes
di: Kong, Boao, et al.
Pubblicazione: (2026)

On Penalty-based Bilevel Gradient Descent Method
di: Shen, Han, et al.
Pubblicazione: (2023)

Biased Stochastic First-Order Methods for Conditional Stochastic Optimization and Applications in Meta Learning
di: Hu, Yifan, et al.
Pubblicazione: (2020)

Convergence of First-Order Algorithms with Momentum from the Perspective of an Inexact Gradient Descent Method
di: Khanh, Pham Duy, et al.
Pubblicazione: (2025)

On the Convergence of the Gradient Descent Method with Stochastic Fixed-point Rounding Errors under the Polyak-Lojasiewicz Inequality
di: Xia, Lu, et al.
Pubblicazione: (2023)

RanSOM: Second-Order Momentum with Randomized Scaling for Constrained and Unconstrained Optimization
di: Chayti, El Mahdi
Pubblicazione: (2026)

Faster Convergence of Riemannian Stochastic Gradient Descent with Increasing Batch Size
di: Oowada, Kanata, et al.
Pubblicazione: (2025)

Coupling-based Convergence Diagnostic and Stepsize Scheme for Stochastic Gradient Descent
di: Li, Xiang, et al.
Pubblicazione: (2024)

Towards Noise-adaptive, Problem-adaptive (Accelerated) Stochastic Gradient Descent
di: Vaswani, Sharan, et al.
Pubblicazione: (2021)

Decentralized Stochastic Gradient Descent Ascent for Finite-Sum Minimax Problems
di: Gao, Hongchang
Pubblicazione: (2022)

Can We Remove the Square-Root in Adaptive Gradient Methods? A Second-Order Perspective
di: Lin, Wu, et al.
Pubblicazione: (2024)

Generalized Stochastic Gradient Descent with Momentum Methods for Smooth Optimization
di: Wang, Zimeng, et al.
Pubblicazione: (2026)

Armijo Line-search Can Make (Stochastic) Gradient Descent Provably Faster
di: Vaswani, Sharan, et al.
Pubblicazione: (2025)