Revisiting Gradient Normalization and Clipping for Nonconvex SGD under Heavy-Tailed Noise: Necessity, Sufficiency, and Acceleration

Documenti analoghi

• From Gradient Clipping to Normalization for Heavy Tailed SGD
  di: Hübler, Florian, et al.
  Pubblicazione: (2024)
• Nonconvex Stochastic Optimization under Heavy-Tailed Noises: Optimal Convergence without Gradient Clipping
  di: Liu, Zijian, et al.
  Pubblicazione: (2024)
• Clipped Gradient Methods for Nonsmooth Convex Optimization under Heavy-Tailed Noise: A Refined Analysis
  di: Liu, Zijian
  Pubblicazione: (2025)
• Heavy-Tail Phenomenon in Decentralized SGD
  di: Gurbuzbalaban, Mert, et al.
  Pubblicazione: (2022)
• Convergence of Clipped-SGD for Convex $(L_0,L_1)$-Smooth Optimization with Heavy-Tailed Noise
  di: Chezhegov, Savelii, et al.
  Pubblicazione: (2025)