Cover Image

Towards Noise-adaptive, Problem-adaptive (Accelerated) Stochastic Gradient Descent

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Vaswani, Sharan, Dubois-Taine, Benjamin, Babanezhad, Reza
Format:	Preprint
Published:	2021
Subjects:	Optimization and Control Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

(Accelerated) Noise-adaptive Stochastic Heavy-Ball Momentum
by: Dang, Anh, et al.
Published: (2024)

Armijo Line-search Can Make (Stochastic) Gradient Descent Provably Faster
by: Vaswani, Sharan, et al.
Published: (2025)

Convergence of Steepest Descent and Adam under Non-Uniform Smoothness
by: Vaswani, Sharan, et al.
Published: (2026)

Fast Stochastic Composite Minimization and an Accelerated Frank-Wolfe Algorithm under Parallelization
by: Dubois-Taine, Benjamin, et al.
Published: (2022)

Glocal Smoothness: Line search and adaptive step sizes can help in theory too!
by: Fox, Curtis, et al.
Published: (2025)

Momentum Does Not Reduce Stochastic Noise in Stochastic Gradient Descent
by: Sato, Naoki, et al.
Published: (2024)

Parameter Symmetry and Noise Equilibrium of Stochastic Gradient Descent
by: Ziyin, Liu, et al.
Published: (2024)

On the Inherent Privacy of Zeroth Order Projected Gradient Descent
by: Gupta, Devansh, et al.
Published: (2025)

From Inverse Optimization to Feasibility to ERM
by: Mishra, Saurabh, et al.
Published: (2024)

Faster Convergence of Stochastic Accelerated Gradient Descent under Interpolation
by: Mishkin, Aaron, et al.
Published: (2024)

Stochastic Adaptive Gradient Descent Without Descent
by: Aujol, Jean-François, et al.
Published: (2025)

Decentralized Stochastic Gradient Descent Ascent for Finite-Sum Minimax Problems
by: Gao, Hongchang
Published: (2022)

Increasing Both Batch Size and Learning Rate Accelerates Stochastic Gradient Descent
by: Umeda, Hikaru, et al.
Published: (2024)

Anytime Acceleration of Gradient Descent
by: Zhang, Zihan, et al.
Published: (2024)

Stochastic Gradient Descent with Adaptive Data
by: Che, Ethan, et al.
Published: (2024)

Stochastic Gradient Descent with Strategic Querying
by: Jiang, Nanfei, et al.
Published: (2025)

Algorithmic Stability of Stochastic Gradient Descent with Momentum under Heavy-Tailed Noise
by: Dang, Thanh, et al.
Published: (2025)

More Optimal Fractional-Order Stochastic Gradient Descent for Non-Convex Optimization Problems
by: Partohaghighi, Mohammad, et al.
Published: (2025)

Effective Dimension Aware Fractional-Order Stochastic Gradient Descent for Convex Optimization Problems
by: Partohaghighi, Mohammad, et al.
Published: (2025)

Nonlinear Stochastic Gradient Descent and Heavy-tailed Noise: A Unified Framework and High-probability Guarantees
by: Armacki, Aleksandar, et al.
Published: (2024)

Derivatives of Stochastic Gradient Descent in parametric optimization
by: Iutzeler, Franck, et al.
Published: (2024)

Accelerated Convergence of Stochastic Heavy Ball Method under Anisotropic Gradient Noise
by: Pan, Rui, et al.
Published: (2023)

Convergence Analysis of Stochastic Gradient Descent with MCMC Estimators
by: Li, Tianyou, et al.
Published: (2023)

The Sample Complexity of Gradient Descent in Stochastic Convex Optimization
by: Livni, Roi
Published: (2024)

Adaptive Step Sizes for Preconditioned Stochastic Gradient Descent
by: Köhne, Frederik, et al.
Published: (2023)

Functional Central Limit Theorem for Stochastic Gradient Descent
by: Flamand, Kessang, et al.
Published: (2026)

Matching the Statistical Query Lower Bound for $k$-Sparse Parity Problems with Sign Stochastic Gradient Descent
by: Kou, Yiwen, et al.
Published: (2024)

Rapid Overfitting of Multi-Pass Stochastic Gradient Descent in Stochastic Convex Optimization
by: Vansover-Hager, Shira, et al.
Published: (2025)

Implicit Bias and Convergence of Matrix Stochastic Mirror Descent
by: Akhtiamov, Danil, et al.
Published: (2026)

Exponential Convergence of (Stochastic) Gradient Descent for Separable Logistic Regression
by: Kale, Sacchit, et al.
Published: (2026)

On the Convergence of Stochastic Gradient Descent with Perturbed Forward-Backward Passes
by: Kong, Boao, et al.
Published: (2026)

On Gradient Descent Ascent for Nonconvex-Concave Minimax Problems
by: Lin, Tianyi, et al.
Published: (2019)

Open Problem: Anytime Convergence Rate of Gradient Descent
by: Kornowski, Guy, et al.
Published: (2024)

Dissecting Discrete Soft Actor-Critic: Limitations and Principled Alternatives
by: Asad, Reza, et al.
Published: (2025)

High-probability Convergence Bounds for Nonlinear Stochastic Gradient Descent Under Heavy-tailed Noise
by: Armacki, Aleksandar, et al.
Published: (2023)

Faster Convergence of Riemannian Stochastic Gradient Descent with Increasing Batch Size
by: Oowada, Kanata, et al.
Published: (2025)

Coupling-based Convergence Diagnostic and Stepsize Scheme for Stochastic Gradient Descent
by: Li, Xiang, et al.
Published: (2024)

Stochastic Subspace Descent Accelerated via Bi-fidelity Line Search
by: Cheng, Nuojin, et al.
Published: (2025)

Enhancing Stochastic Gradient Descent: A Unified Framework and Novel Acceleration Methods for Faster Convergence
by: Deng, Yichuan, et al.
Published: (2024)

GeoAdaLer: Geometric Insights into Adaptive Stochastic Gradient Descent Algorithms
by: Eleh, Chinedu, et al.
Published: (2024)