Cover Image

Gradient Shaping Beyond Clipping: A Functional Perspective on Update Magnitude Control

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	You, Haochen, Liu, Baojing
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Metric Embedding Initialization-Based Differentially Private and Explainable Graph Clustering
by: You, Haochen, et al.
Published: (2025)

MCIGLE: Multimodal Exemplar-Free Class-Incremental Graph Learning
by: You, Haochen, et al.
Published: (2025)

Modular MeanFlow: Towards Stable and Scalable One-Step Generative Modeling
by: You, Haochen, et al.
Published: (2025)

Generalized Radius and Integrated Codebook Transforms for Differentiable Vector Quantization
by: You, Haochen, et al.
Published: (2026)

To Clip or not to Clip: the Dynamics of SGD with Gradient Clipping in High-Dimensions
by: Marshall, Noah, et al.
Published: (2024)

ReSSFormer: A Recursive Sparse Structured Transformer for Scalable and Long-Context Reasoning
by: You, Haochen, et al.
Published: (2025)

Can Entry-Wise Clipping Give Spectral Control of Stochastic Gradients?
by: Song, Zitao, et al.
Published: (2026)

MOVER: Multimodal Optimal Transport with Volume-based Embedding Regularization
by: You, Haochen, et al.
Published: (2025)

MuCon: Clipped Muon Updates for LLM Training
by: Yi, Albert
Published: (2026)

Gradient Clipping Beyond Vector Norms: A Spectral Approach for Matrix-Valued Parameters
by: Yukhimchuk, Alexander, et al.
Published: (2026)

Adaptive Gradient Clipping for Robust Federated Learning
by: Allouah, Youssef, et al.
Published: (2024)

When Gradient Clipping Becomes a Control Mechanism for Differential Privacy in Deep Learning
by: Partohaghighi, Mohammad, et al.
Published: (2026)

AdaDPIGU: Differentially Private SGD with Adaptive Clipping and Importance-Based Gradient Updates for Deep Neural Networks
by: Zhang, Huiqi, et al.
Published: (2025)

Robust Stochastic Optimization via Gradient Quantile Clipping
by: Merad, Ibrahim, et al.
Published: (2023)

Exploring the Impact of Parameter Update Magnitude on Forgetting and Generalization of Continual Learning
by: He, JinLi, et al.
Published: (2026)

From Gradient Clipping to Normalization for Heavy Tailed SGD
by: Hübler, Florian, et al.
Published: (2024)

Parameter-free Clipped Gradient Descent Meets Polyak
by: Takezawa, Yuki, et al.
Published: (2024)

Clipped Gradient Methods for Nonsmooth Convex Optimization under Heavy-Tailed Noise: A Refined Analysis
by: Liu, Zijian
Published: (2025)

Beyond Softmax: A New Perspective on Gradient Bandits
by: Melo, Emerson, et al.
Published: (2025)

Rethinking LLM Unlearning Objectives: A Gradient Perspective and Go Beyond
by: Wang, Qizhou, et al.
Published: (2025)

Merging Beyond: Streaming LLM Updates via Activation-Guided Rotations
by: Yao, Yuxuan, et al.
Published: (2026)

Nonconvex Stochastic Optimization under Heavy-Tailed Noises: Optimal Convergence without Gradient Clipping
by: Liu, Zijian, et al.
Published: (2024)

Generalized Gradient Norm Clipping & Non-Euclidean $(L_0,L_1)$-Smoothness
by: Pethick, Thomas, et al.
Published: (2025)

Regularized Gradient Clipping Provably Trains Wide and Deep Neural Networks
by: Tucat, Matteo, et al.
Published: (2024)

SoftAdaClip: A Smooth Clipping Strategy for Fair and Private Model Training
by: Soleymani, Dorsa, et al.
Published: (2025)

Revisiting Gradient Normalization and Clipping for Nonconvex SGD under Heavy-Tailed Noise: Necessity, Sufficiency, and Acceleration
by: Sun, Tao, et al.
Published: (2024)

AGGC: Adaptive Group Gradient Clipping for Stabilizing Large Language Model Training
by: Li, Zhiyuan, et al.
Published: (2026)

HELENE: Hessian Layer-wise Clipping and Gradient Annealing for Accelerating Fine-tuning LLM with Zeroth-order Optimization
by: Zhao, Huaqin, et al.
Published: (2024)

A Bootstrap Perspective on Stochastic Gradient Descent
by: Lan, Hongjian, et al.
Published: (2025)

Gradient Transformer: Learning to Generate Updates for LLMs
by: Nguyen, Binh-Nguyen, et al.
Published: (2026)

Klear-Reasoner: Advancing Reasoning Capability via Gradient-Preserving Clipping Policy Optimization
by: Su, Zhenpeng, et al.
Published: (2025)

GaLLoP: Gradient-based Sparse Learning on Low-Magnitude Parameters
by: Choudhary, Anand, et al.
Published: (2025)

ProAct: Progressive Training for Hybrid Clipped Activation Function to Enhance Resilience of DNNs
by: Mousavi, Seyedhamidreza, et al.
Published: (2024)

ConfClip: Confidence-Weighted and Clipped Reward for Reinforcement Learning in LLMs
by: Zhang, Bonan, et al.
Published: (2025)

PPO-Clip Attains Global Optimality: Towards Deeper Understandings of Clipping
by: Huang, Nai-Chieh, et al.
Published: (2023)

CE-GPPO: Coordinating Entropy via Gradient-Preserving Clipping Policy Optimization in Reinforcement Learning
by: Su, Zhenpeng, et al.
Published: (2025)

Revisit Micro-batch Clipping: Adaptive Data Pruning via Gradient Manipulation
by: Wang, Lun
Published: (2024)

Enabling Differentially Private Federated Learning for Speech Recognition: Benchmarks, Adaptive Optimizers and Gradient Clipping
by: Pelikan, Martin, et al.
Published: (2023)

Clip-Low Increases Entropy and Clip-High Decreases Entropy in Reinforcement Learning of Large Language Models
by: Park, Jaesung R., et al.
Published: (2025)

Improving Unlearning with Model Updates Probably Aligned with Gradients
by: Dine, Virgile, et al.
Published: (2025)