Image de couverture de livre

Tight Long-Term Tail Decay of (Clipped) SGD in Non-Convex Optimization

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Armacki, Aleksandar, Bajović, Dragana, Jakovetić, Dušan, Kar, Soummya, Sayed, Ali H.
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning Optimization and Control
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Sharp High-Probability Rates for Nonlinear SGD under Heavy-Tailed Noise via Symmetrization
par: Armacki, Aleksandar, et autres
Publié: (2025)

Large Deviation Upper Bounds and Improved MSE Rates of Nonlinear SGD: Heavy-tailed Noise and Power of Symmetry
par: Armacki, Aleksandar, et autres
Publié: (2024)

High-probability Convergence Bounds for Nonlinear Stochastic Gradient Descent Under Heavy-tailed Noise
par: Armacki, Aleksandar, et autres
Publié: (2023)

Nonlinear Stochastic Gradient Descent and Heavy-tailed Noise: A Unified Framework and High-probability Guarantees
par: Armacki, Aleksandar, et autres
Publié: (2024)

A Unified Framework for Center-based Clustering of Distributed Data
par: Armacki, Aleksandar, et autres
Publié: (2024)

Distributed gradient methods under heavy-tailed communication noise
par: Vukovic, Manojlo, et autres
Publié: (2025)

High-Probability Convergence Guarantees of Decentralized SGD
par: Armacki, Aleksandar, et autres
Publié: (2025)

Smoothed Gradient Clipping and Error Feedback for Decentralized Optimization under Symmetric Heavy-Tailed Noise
par: Yu, Shuhua, et autres
Publié: (2023)

Distributed Gradient Clustering: Convergence and the Effect of Initialization
par: Armacki, Aleksandar, et autres
Publié: (2026)

Tackling heavy-tailed noise in distributed estimation: Asymptotic performance and tradeoffs
par: Bajovic, Dragana, et autres
Publié: (2026)

Decentralized Nonconvex Optimization under Heavy-Tailed Noise: Normalization and Optimal Convergence
par: Yu, Shuhua, et autres
Publié: (2025)

High-Probability Convergence in Decentralized Stochastic Optimization with Gradient Tracking
par: Armacki, Aleksandar, et autres
Publié: (2026)

Convergence of Clipped-SGD for Convex $(L_0,L_1)$-Smooth Optimization with Heavy-Tailed Noise
par: Chezhegov, Savelii, et autres
Publié: (2025)

From Gradient Clipping to Normalization for Heavy Tailed SGD
par: Hübler, Florian, et autres
Publié: (2024)

Clipped SGD Algorithms for Performative Prediction: Tight Bounds for Clipping Bias and Remedies
par: Li, Qiang, et autres
Publié: (2024)

Robust stochastic first order methods in heavy-tailed noise via medoid mini-batch gradient sampling
par: Vukovic, Manojlo, et autres
Publié: (2026)

Revisiting Gradient Normalization and Clipping for Nonconvex SGD under Heavy-Tailed Noise: Necessity, Sufficiency, and Acceleration
par: Sun, Tao, et autres
Publié: (2024)

Clipped Gradient Methods for Nonsmooth Convex Optimization under Heavy-Tailed Noise: A Refined Analysis
par: Liu, Zijian
Publié: (2025)

Differentially Private Clipped-SGD: High-Probability Convergence with Arbitrary Clipping Level
par: Khah, Saleh Vatan, et autres
Publié: (2025)

SLowcal-SGD: Slow Query Points Improve Local-SGD for Stochastic Convex Optimization
par: Dahan, Tehila, et autres
Publié: (2023)

On the Convergence of DP-SGD with Adaptive Clipping
par: Shulgin, Egor, et autres
Publié: (2024)

Heavy-Tail Phenomenon in Decentralized SGD
par: Gurbuzbalaban, Mert, et autres
Publié: (2022)

Can SGD Handle Heavy-Tailed Noise?
par: Fatkhullin, Ilyas, et autres
Publié: (2025)

Tight Bounds for Online Convex Optimization with Adversarial Constraints
par: Sinha, Abhishek, et autres
Publié: (2024)

Provable Complexity Improvement of AdaGrad over SGD: Upper and Lower Bounds in Stochastic Non-Convex Optimization
par: Jiang, Ruichen, et autres
Publié: (2024)

Asynchronous Decentralized SGD under Non-Convexity: A Block-Coordinate Descent Framework
par: Zhou, Yijie, et autres
Publié: (2025)

Lower Bounds and Proximally Anchored SGD for Non-Convex Minimization Under Unbounded Variance
par: Fazla, Arda, et autres
Publié: (2026)

Nonconvex Stochastic Optimization under Heavy-Tailed Noises: Optimal Convergence without Gradient Clipping
par: Liu, Zijian, et autres
Publié: (2024)

Second-order Optimization under Heavy-Tailed Noise: Hessian Clipping and Sample Complexity Limits
par: Sadiev, Abdurakhmon, et autres
Publié: (2025)

Methods for Convex $(L_0,L_1)$-Smooth Optimization: Clipping, Acceleration, and Adaptivity
par: Gorbunov, Eduard, et autres
Publié: (2024)

Stochastic Weakly Convex Optimization Under Heavy-Tailed Noises
par: Zhu, Tianxi, et autres
Publié: (2025)

Efficient Private SCO for Heavy-Tailed Data via Averaged Clipping
par: Jin, Chenhan, et autres
Publié: (2022)

Byzantine-Robust Distributed SGD: A Unified Analysis and Tight Error Bounds
par: Ruan, Boyuan, et autres
Publié: (2026)

Online Convex Optimization with Heavy Tails: Old Algorithms, New Regrets, and Applications
par: Liu, Zijian
Publié: (2025)

Non-Euclidean SGD for Structured Optimization: Unified Analysis and Improved Rates
par: Kovalev, Dmitry, et autres
Publié: (2025)

Clipping Improves Adam-Norm and AdaGrad-Norm when the Noise Is Heavy-Tailed
par: Chezhegov, Savelii, et autres
Publié: (2024)

Non-Asymptotic Global Convergence of PPO-Clip
par: Liu, Yin, et autres
Publié: (2025)

Sign Operator for Coping with Heavy-Tailed Noise in Non-Convex Optimization: High Probability Bounds Under $(L_0, L_1)$-Smoothness
par: Kornilov, Nikita, et autres
Publié: (2025)

Diffusion Stochastic Optimization for Min-Max Problems
par: Cai, Haoyuan, et autres
Publié: (2024)

High-Probability Bounds for SGD under the Polyak-Lojasiewicz Condition with Markovian Noise
par: Kar, Avik, et autres
Publié: (2026)