Image de couverture de livre

Gradient Clipping Beyond Vector Norms: A Spectral Approach for Matrix-Valued Parameters

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Yukhimchuk, Alexander, Kolar, Mladen, Takáč, Martin, Choudhury, Sayantan
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning Optimization and Control
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Muon with Nesterov Momentum: Heavy-Tailed Noise and (Randomized) Inexact Polar Decomposition
par: Choudhury, Sayantan, et autres
Publié: (2026)

Methods for Convex $(L_0,L_1)$-Smooth Optimization: Clipping, Acceleration, and Adaptivity
par: Gorbunov, Eduard, et autres
Publié: (2024)

Clipping Improves Adam-Norm and AdaGrad-Norm when the Noise Is Heavy-Tailed
par: Chezhegov, Savelii, et autres
Publié: (2024)

AdAdaGrad: Adaptive Batch Size Schemes for Adaptive Gradient Methods
par: Lau, Tim Tsz-Kit, et autres
Publié: (2024)

Parameter-free Clipped Gradient Descent Meets Polyak
par: Takezawa, Yuki, et autres
Publié: (2024)

Remove that Square Root: A New Efficient Scale-Invariant Version of AdaGrad
par: Choudhury, Sayantan, et autres
Publié: (2024)

Communication-Efficient Adaptive Batch Size Strategies for Distributed Local Gradient Methods
par: Lau, Tim Tsz-Kit, et autres
Publié: (2024)

Extragradient Method for $(L_0, L_1)$-Lipschitz Root-finding Problems
par: Choudhury, Sayantan, et autres
Publié: (2025)

The Ky Fan Norms and Beyond: Dual Norms and Combinations for Matrix Optimization
par: Kravatskiy, Alexey, et autres
Publié: (2025)

Communication-Efficient Gradient Descent-Accent Methods for Distributed Variational Inequalities: Unified Analysis and Local Updates
par: Zhang, Siqi, et autres
Publié: (2023)

Where Does Warm-Up Come From? Adaptive Scheduling for Norm-Constrained Optimizers
par: Riabinin, Artem, et autres
Publié: (2026)

Preconditioned Norms: A Unified Framework for Steepest Descent, Quasi-Newton and Adaptive Methods
par: Veprikov, Andrey, et autres
Publié: (2025)

Fully Stochastic Trust-Region Sequential Quadratic Programming for Equality-Constrained Optimization Problems
par: Fang, Yuchen, et autres
Publié: (2022)

Random-reshuffled SARAH does not need a full gradient computations
par: Beznosikov, Aleksandr, et autres
Publié: (2021)

From Gradient Clipping to Normalization for Heavy Tailed SGD
par: Hübler, Florian, et autres
Publié: (2024)

Multiplayer Federated Learning: Reaching Equilibrium with Less Communication
par: Yoon, TaeHo, et autres
Publié: (2025)

Stochastic Gradient Methods with Preconditioned Updates
par: Sadiev, Abdurakhmon, et autres
Publié: (2022)

Adaptive Batch Size Schedules for Distributed Training of Language Models with Data and Model Parallelism
par: Lau, Tim Tsz-Kit, et autres
Publié: (2024)

Regularized Gradient Clipping Provably Trains Wide and Deep Neural Networks
par: Tucat, Matteo, et autres
Publié: (2024)

Pessimism Meets Risk: Risk-Sensitive Offline Reinforcement Learning
par: Zhang, Dake, et autres
Publié: (2024)

Muon Optimizes Under Spectral Norm Constraints
par: Chen, Lizhang, et autres
Publié: (2025)

Trust-Region Sequential Quadratic Programming for Stochastic Optimization with Random Models
par: Fang, Yuchen, et autres
Publié: (2024)

Clipped Gradient Methods for Nonsmooth Convex Optimization under Heavy-Tailed Noise: A Refined Analysis
par: Liu, Zijian
Publié: (2025)

Small Gradient Norm Regret for Online Convex Optimization
par: Gao, Wenzhi, et autres
Publié: (2026)

Nonconvex Stochastic Optimization under Heavy-Tailed Noises: Optimal Convergence without Gradient Clipping
par: Liu, Zijian, et autres
Publié: (2024)

Differentially Private Clipped-SGD: High-Probability Convergence with Arbitrary Clipping Level
par: Khah, Saleh Vatan, et autres
Publié: (2025)

Revisiting Gradient Normalization and Clipping for Nonconvex SGD under Heavy-Tailed Noise: Necessity, Sufficiency, and Acceleration
par: Sun, Tao, et autres
Publié: (2024)

Federated Learning Can Find Friends That Are Advantageous
par: Tupitsa, Nazarii, et autres
Publié: (2024)

AdaBatchGrad: Combining Adaptive Batch Size and Adaptive Step Size
par: Ostroukhov, Petr, et autres
Publié: (2024)

Simple Stepsize for Quasi-Newton Methods with Global Convergence Guarantees
par: Agafonov, Artem, et autres
Publié: (2025)

Convergence of Alternating Gradient Descent for Matrix Factorization
par: Ward, Rachel, et autres
Publié: (2023)

SANIA: Polyak-type Optimization Framework Leads to Scale Invariant Stochastic Algorithms
par: Abdukhakimov, Farshed, et autres
Publié: (2023)

LoFT: Low-Rank Adaptation That Behaves Like Full Fine-Tuning
par: Tastan, Nurbek, et autres
Publié: (2025)

Non-Asymptotic Global Convergence of PPO-Clip
par: Liu, Yin, et autres
Publié: (2025)

Bayesian Optimization with Structured Measurements: A Vector-Valued RKHS Framework
par: Wang, Wenbin, et autres
Publié: (2026)

Robust and Fast Training via Per-Sample Clipping
par: Nobile, Davide, et autres
Publié: (2026)

Exploring New Frontiers in Vertical Federated Learning: the Role of Saddle Point Reformulation
par: Beznosikov, Aleksandr, et autres
Publié: (2026)

Preconditioned Gradient Descent for Over-Parameterized Nonconvex Matrix Factorization
par: Zhang, Gavin, et autres
Publié: (2025)

Convergence of Gradient Descent with Small Initialization for Unregularized Matrix Completion
par: Ma, Jianhao, et autres
Publié: (2024)

Gradient-Free Approaches is a Key to an Efficient Interaction with Markovian Stochasticity
par: Prokhorov, Boris, et autres
Publié: (2026)