Cover Image

(Accelerated) Noise-adaptive Stochastic Heavy-Ball Momentum

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Dang, Anh, Babanezhad, Reza, Vaswani, Sharan
Format:	Preprint
Published:	2024
Subjects:	Optimization and Control Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Towards Noise-adaptive, Problem-adaptive (Accelerated) Stochastic Gradient Descent
by: Vaswani, Sharan, et al.
Published: (2021)

Armijo Line-search Can Make (Stochastic) Gradient Descent Provably Faster
by: Vaswani, Sharan, et al.
Published: (2025)

Convergence of Steepest Descent and Adam under Non-Uniform Smoothness
by: Vaswani, Sharan, et al.
Published: (2026)

Accelerated Convergence of Stochastic Heavy Ball Method under Anisotropic Gradient Noise
by: Pan, Rui, et al.
Published: (2023)

Glocal Smoothness: Line search and adaptive step sizes can help in theory too!
by: Fox, Curtis, et al.
Published: (2025)

Algorithmic Stability of Stochastic Gradient Descent with Momentum under Heavy-Tailed Noise
by: Dang, Thanh, et al.
Published: (2025)

From Inverse Optimization to Feasibility to ERM
by: Mishra, Saurabh, et al.
Published: (2024)

On the Provable Suboptimality of Momentum SGD in Nonstationary Stochastic Optimization
by: Sahu, Sharan, et al.
Published: (2026)

Momentum Does Not Reduce Stochastic Noise in Stochastic Gradient Descent
by: Sato, Naoki, et al.
Published: (2024)

Muon with Nesterov Momentum: Heavy-Tailed Noise and (Randomized) Inexact Polar Decomposition
by: Choudhury, Sayantan, et al.
Published: (2026)

Dissecting Discrete Soft Actor-Critic: Limitations and Principled Alternatives
by: Asad, Reza, et al.
Published: (2025)

Accelerated Stochastic Min-Max Optimization Based on Bias-corrected Momentum
by: Cai, Haoyuan, et al.
Published: (2024)

Stochastic Weakly Convex Optimization Under Heavy-Tailed Noises
by: Zhu, Tianxi, et al.
Published: (2025)

Near-Optimal Decentralized Stochastic Nonconvex Optimization with Heavy-Tailed Noise
by: Wang, Menglian, et al.
Published: (2026)

Optimal Asynchronous Stochastic Nonconvex Optimization under Heavy-Tailed Noise
by: Wu, Yidong, et al.
Published: (2026)

In-Expectation Convergence of Stochastic Gradient Methods under Heavy-Tailed Noise
by: Liu, Zijian
Published: (2026)

SHANG++: Robust Stochastic Acceleration under Multiplicative Noise
by: Yu, Yaxin, et al.
Published: (2026)

AltGDmin: Alternating GD and Minimization for Partly-Decoupled (Federated) Optimization
by: Vaswani, Namrata
Published: (2025)

Dimension-adapted Momentum Outscales SGD
by: Ferbach, Damien, et al.
Published: (2025)

High Probability Complexity Bounds for Non-Smooth Stochastic Optimization with Heavy-Tailed Noise
by: Gorbunov, Eduard, et al.
Published: (2021)

Concentration of General Stochastic Approximation Under Heavy-Tailed Markovian Noise
by: Agrawal, Shubhada, et al.
Published: (2026)

Improving Stochastic Cubic Newton with Momentum
by: Chayti, El Mahdi, et al.
Published: (2024)

Stochastic Difference-of-Convex Optimization with Momentum
by: Chayti, El Mahdi, et al.
Published: (2025)

Nonconvex Stochastic Optimization under Heavy-Tailed Noises: Optimal Convergence without Gradient Clipping
by: Liu, Zijian, et al.
Published: (2024)

High-Probability Convergence for Composite and Distributed Stochastic Minimization and Variational Inequalities with Heavy-Tailed Noise
by: Gorbunov, Eduard, et al.
Published: (2023)

Provable Acceleration of Nesterov's Accelerated Gradient Method over Heavy Ball Method in Training Over-Parameterized Neural Networks
by: Liu, Xin, et al.
Published: (2022)

Heavy-Tailed and Long-Range Dependent Noise in Stochastic Approximation: A Finite-Time Analysis
by: Chandak, Siddharth, et al.
Published: (2026)

Nonlinear Stochastic Gradient Descent and Heavy-tailed Noise: A Unified Framework and High-probability Guarantees
by: Armacki, Aleksandar, et al.
Published: (2024)

Revisiting Gradient Normalization and Clipping for Nonconvex SGD under Heavy-Tailed Noise: Necessity, Sufficiency, and Acceleration
by: Sun, Tao, et al.
Published: (2024)

Non-convex Stochastic Composite Optimization with Polyak Momentum
by: Gao, Yuan, et al.
Published: (2024)

Dynamics of Stochastic Momentum with Sparse Updates in High Dimensions
by: Everett, Katie, et al.
Published: (2026)

SGD with Adaptive Preconditioning: Unified Analysis and Momentum Acceleration
by: Kovalev, Dmitry
Published: (2025)

Can SGD Handle Heavy-Tailed Noise?
by: Fatkhullin, Ilyas, et al.
Published: (2025)

Stochastic Compositional Optimization via Hybrid Momentum Frank--Wolfe
by: Chayti, El Mahdi
Published: (2026)

Compressed Decentralized Momentum Stochastic Gradient Methods for Nonconvex Optimization
by: Liu, Wei, et al.
Published: (2025)

Stochastic Polyak Step-sizes and Momentum: Convergence Guarantees and Practical Performance
by: Oikonomou, Dimitris, et al.
Published: (2024)

Understanding Outer Optimizers in Local SGD: Learning Rates, Momentum, and Acceleration
by: Khaled, Ahmed, et al.
Published: (2025)

Towards Parameter-Free Temporal Difference Learning
by: Li, Yunxiang, et al.
Published: (2026)

A Novel Approach in Solving Stochastic Generalized Linear Regression via Nonconvex Programming
by: Anh, Vu Duc, et al.
Published: (2024)

Stability and Generalization for Stochastic Recursive Momentum-based Algorithms for (Strongly-)Convex One to $K$-Level Stochastic Optimizations
by: Pan, Xiaokang, et al.
Published: (2024)