Imagen de Portada

Accelerated Convergence of Stochastic Heavy Ball Method under Anisotropic Gradient Noise

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Pan, Rui, Liu, Yuxing, Wang, Xiaoyu, Zhang, Tong
Formato:	Preprint
Publicado:	2023
Materias:	Machine Learning Optimization and Control
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

AdaGrad under Anisotropic Smoothness
por: Liu, Yuxing, et al.
Publicado: (2024)

In-Expectation Convergence of Stochastic Gradient Methods under Heavy-Tailed Noise
por: Liu, Zijian
Publicado: (2026)

(Accelerated) Noise-adaptive Stochastic Heavy-Ball Momentum
por: Dang, Anh, et al.
Publicado: (2024)

Theoretical Analysis on how Learning Rate Warmup Accelerates Convergence
por: Liu, Yuxing, et al.
Publicado: (2025)

Nonconvex Stochastic Optimization under Heavy-Tailed Noises: Optimal Convergence without Gradient Clipping
por: Liu, Zijian, et al.
Publicado: (2024)

Provable Acceleration of Nesterov's Accelerated Gradient Method over Heavy Ball Method in Training Over-Parameterized Neural Networks
por: Liu, Xin, et al.
Publicado: (2022)

Can Adaptive Gradient Methods Converge under Heavy-Tailed Noise? A Case Study of AdaGrad
por: Liu, Zijian
Publicado: (2026)

Faster Convergence of Stochastic Accelerated Gradient Descent under Interpolation
por: Mishkin, Aaron, et al.
Publicado: (2024)

Convergence of the Stochastic Heavy Ball Method With Approximate Gradients and/or Block Updating
por: Tadipatri, Uday Kiran Reddy, et al.
Publicado: (2023)

Revisiting Gradient Normalization and Clipping for Nonconvex SGD under Heavy-Tailed Noise: Necessity, Sufficiency, and Acceleration
por: Sun, Tao, et al.
Publicado: (2024)

ASGO: Adaptive Structured Gradient Optimization
por: An, Kang, et al.
Publicado: (2025)

Algorithmic Stability of Stochastic Gradient Descent with Momentum under Heavy-Tailed Noise
por: Dang, Thanh, et al.
Publicado: (2025)

Revisiting the Last-Iterate Convergence of Stochastic Gradient Methods
por: Liu, Zijian, et al.
Publicado: (2023)

Clipped Gradient Methods for Nonsmooth Convex Optimization under Heavy-Tailed Noise: A Refined Analysis
por: Liu, Zijian
Publicado: (2025)

StoSignSGD: Unbiased Structural Stochasticity Fixes SignSGD for Training Large Language Models
por: Yu, Dingzhi, et al.
Publicado: (2026)

Unbiased Gradient Low-Rank Projection
por: Pan, Rui, et al.
Publicado: (2025)

High-probability Convergence Bounds for Nonlinear Stochastic Gradient Descent Under Heavy-tailed Noise
por: Armacki, Aleksandar, et al.
Publicado: (2023)

Almost Sure Convergence Analysis of Differentially Private Stochastic Gradient Methods
por: Mukherjee, Amartya, et al.
Publicado: (2025)

SHANG++: Robust Stochastic Acceleration under Multiplicative Noise
por: Yu, Yaxin, et al.
Publicado: (2026)

Towards Noise-adaptive, Problem-adaptive (Accelerated) Stochastic Gradient Descent
por: Vaswani, Sharan, et al.
Publicado: (2021)

Optimal Asynchronous Stochastic Nonconvex Optimization under Heavy-Tailed Noise
por: Wu, Yidong, et al.
Publicado: (2026)

High-Probability Convergence for Composite and Distributed Stochastic Minimization and Variational Inequalities with Heavy-Tailed Noise
por: Gorbunov, Eduard, et al.
Publicado: (2023)

Near-Optimal Decentralized Stochastic Nonconvex Optimization with Heavy-Tailed Noise
por: Wang, Menglian, et al.
Publicado: (2026)

Near-Optimal Convergence of Accelerated Gradient Methods under Generalized and $(L_0, L_1)$-Smoothness
por: Tyurin, Alexander
Publicado: (2025)

Parameter Symmetry and Noise Equilibrium of Stochastic Gradient Descent
por: Ziyin, Liu, et al.
Publicado: (2024)

Beyond Stationarity: Convergence Analysis of Stochastic Softmax Policy Gradient Methods
por: Klein, Sara, et al.
Publicado: (2023)

Muon Converges under Heavy-Tailed Noise: Nonconvex Hölder-Smooth Empirical Risk Minimization
por: Iiduka, Hideaki
Publicado: (2026)

Nonlinear Stochastic Gradient Descent and Heavy-tailed Noise: A Unified Framework and High-probability Guarantees
por: Armacki, Aleksandar, et al.
Publicado: (2024)

Enhancing Stochastic Gradient Descent: A Unified Framework and Novel Acceleration Methods for Faster Convergence
por: Deng, Yichuan, et al.
Publicado: (2024)

On the Last-Iterate Convergence of Shuffling Gradient Methods
por: Liu, Zijian, et al.
Publicado: (2024)

Momentum Does Not Reduce Stochastic Noise in Stochastic Gradient Descent
por: Sato, Naoki, et al.
Publicado: (2024)

On the Convergence of Stochastic Gradient Descent with Perturbed Forward-Backward Passes
por: Kong, Boao, et al.
Publicado: (2026)

Almost Sure Convergence Rates and Concentration of Stochastic Approximation and Reinforcement Learning with Markovian Noise
por: Qian, Xiaochi, et al.
Publicado: (2024)

Stochastic Weakly Convex Optimization Under Heavy-Tailed Noises
por: Zhu, Tianxi, et al.
Publicado: (2025)

Directional Smoothness and Gradient Methods: Convergence and Adaptivity
por: Mishkin, Aaron, et al.
Publicado: (2024)

Convergence of Decentralized Stochastic Subgradient-based Methods for Nonsmooth Nonconvex functions
por: Zhang, Siyuan, et al.
Publicado: (2024)

Convergence Analysis of Stochastic Gradient Descent with MCMC Estimators
por: Li, Tianyou, et al.
Publicado: (2023)

The Ball-Proximal (="Broximal") Point Method: a New Algorithm, Convergence Theory, and Applications
por: Gruntkowska, Kaja, et al.
Publicado: (2025)

Stochastic Gradients under Nuisances
por: Yu, Facheng, et al.
Publicado: (2025)

On the Convergence of the Gradient Descent Method with Stochastic Fixed-point Rounding Errors under the Polyak-Lojasiewicz Inequality
por: Xia, Lu, et al.
Publicado: (2023)