Image de couverture de livre

To Clip or not to Clip: the Dynamics of SGD with Gradient Clipping in High-Dimensions

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Marshall, Noah, Xiao, Ke Liang, Agarwala, Atish, Paquette, Elliot
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Exact Risk Curves of signSGD in High-Dimensions: Quantifying Preconditioning and Noise-Compression Effects
par: Xiao, Ke Liang, et autres
Publié: (2024)

Phases of Muon: When Muon Eclipses SignSGD
par: Paquette, Elliot, et autres
Publié: (2026)

Differentially Private Clipped-SGD: High-Probability Convergence with Arbitrary Clipping Level
par: Khah, Saleh Vatan, et autres
Publié: (2025)

GeoClip: Geometry-Aware Clipping for Differentially Private SGD
par: Gilani, Atefeh, et autres
Publié: (2025)

From Gradient Clipping to Normalization for Heavy Tailed SGD
par: Hübler, Florian, et autres
Publié: (2024)

Clipped SGD Algorithms for Performative Prediction: Tight Bounds for Clipping Bias and Remedies
par: Li, Qiang, et autres
Publié: (2024)

On the Convergence of DP-SGD with Adaptive Clipping
par: Shulgin, Egor, et autres
Publié: (2024)

DC-SGD: Differentially Private SGD with Dynamic Clipping through Gradient Norm Distribution Estimation
par: Wei, Chengkun, et autres
Publié: (2025)

DP-SGD Without Clipping: The Lipschitz Neural Network Way
par: Bethune, Louis, et autres
Publié: (2023)

Near-Optimal Streaming Heavy-Tailed Statistical Estimation with Clipped SGD
par: Das, Aniket, et autres
Publié: (2024)

Adaptive Gradient Clipping for Robust Federated Learning
par: Allouah, Youssef, et autres
Publié: (2024)

Privacy of SGD under Gaussian or Heavy-Tailed Noise: Guarantees without Gradient Clipping
par: Şimşekli, Umut, et autres
Publié: (2024)

PPO-Clip Attains Global Optimality: Towards Deeper Understandings of Clipping
par: Huang, Nai-Chieh, et autres
Publié: (2023)

ConfClip: Confidence-Weighted and Clipped Reward for Reinforcement Learning in LLMs
par: Zhang, Bonan, et autres
Publié: (2025)

Clip-Low Increases Entropy and Clip-High Decreases Entropy in Reinforcement Learning of Large Language Models
par: Park, Jaesung R., et autres
Publié: (2025)

Differentially Private SGD Without Clipping Bias: An Error-Feedback Approach
par: Zhang, Xinwei, et autres
Publié: (2023)

SoftAdaClip: A Smooth Clipping Strategy for Fair and Private Model Training
par: Soleymani, Dorsa, et autres
Publié: (2025)

Robust Stochastic Optimization via Gradient Quantile Clipping
par: Merad, Ibrahim, et autres
Publié: (2023)

Revisiting Gradient Normalization and Clipping for Nonconvex SGD under Heavy-Tailed Noise: Necessity, Sufficiency, and Acceleration
par: Sun, Tao, et autres
Publié: (2024)

Tight Long-Term Tail Decay of (Clipped) SGD in Non-Convex Optimization
par: Armacki, Aleksandar, et autres
Publié: (2026)

ClipFormer: Key-Value Clipping of Transformers on Memristive Crossbars for Write Noise Mitigation
par: Bhattacharjee, Abhiroop, et autres
Publié: (2024)

DCPO: Dynamic Clipping Policy Optimization
par: Yang, Shihui, et autres
Publié: (2025)

Parameter-free Clipped Gradient Descent Meets Polyak
par: Takezawa, Yuki, et autres
Publié: (2024)

Dimension-adapted Momentum Outscales SGD
par: Ferbach, Damien, et autres
Publié: (2025)

AdaDPIGU: Differentially Private SGD with Adaptive Clipping and Importance-Based Gradient Updates for Deep Neural Networks
par: Zhang, Huiqi, et autres
Publié: (2025)

Feature Clipping for Uncertainty Calibration
par: Tao, Linwei, et autres
Publié: (2024)

High dimensional theory of two-phase optimizers
par: Agarwala, Atish
Publié: (2026)

Clip-and-Verify: Linear Constraint-Driven Domain Clipping for Accelerating Neural Network Verification
par: Zhou, Duo, et autres
Publié: (2025)

Can Entry-Wise Clipping Give Spectral Control of Stochastic Gradients?
par: Song, Zitao, et autres
Publié: (2026)

Regularized Gradient Clipping Provably Trains Wide and Deep Neural Networks
par: Tucat, Matteo, et autres
Publié: (2024)

Gradient Shaping Beyond Clipping: A Functional Perspective on Update Magnitude Control
par: You, Haochen, et autres
Publié: (2025)

Optimized Gradient Clipping for Noisy Label Learning
par: Ye, Xichen, et autres
Publié: (2024)

Convergence of Clipped-SGD for Convex $(L_0,L_1)$-Smooth Optimization with Heavy-Tailed Noise
par: Chezhegov, Savelii, et autres
Publié: (2025)

Dynamics of Stochastic Momentum with Sparse Updates in High Dimensions
par: Everett, Katie, et autres
Publié: (2026)

Weight Clipping for Deep Continual and Reinforcement Learning
par: Elsayed, Mohamed, et autres
Publié: (2024)

Non-Asymptotic Global Convergence of PPO-Clip
par: Liu, Yin, et autres
Publié: (2025)

Clipping-Free Policy Optimization for Large Language Models
par: Çağatan, Ömer Veysel, et autres
Publié: (2026)

MuCon: Clipped Muon Updates for LLM Training
par: Yi, Albert
Publié: (2026)

AGGC: Adaptive Group Gradient Clipping for Stabilizing Large Language Model Training
par: Li, Zhiyuan, et autres
Publié: (2026)

Generalized Gradient Norm Clipping & Non-Euclidean $(L_0,L_1)$-Smoothness
par: Pethick, Thomas, et autres
Publié: (2025)