Copertina

Gradient Clipping Beyond Vector Norms: A Spectral Approach for Matrix-Valued Parameters

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Yukhimchuk, Alexander, Kolar, Mladen, Takáč, Martin, Choudhury, Sayantan
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Machine Learning Optimization and Control
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Muon with Nesterov Momentum: Heavy-Tailed Noise and (Randomized) Inexact Polar Decomposition
di: Choudhury, Sayantan, et al.
Pubblicazione: (2026)

Methods for Convex $(L_0,L_1)$-Smooth Optimization: Clipping, Acceleration, and Adaptivity
di: Gorbunov, Eduard, et al.
Pubblicazione: (2024)

Clipping Improves Adam-Norm and AdaGrad-Norm when the Noise Is Heavy-Tailed
di: Chezhegov, Savelii, et al.
Pubblicazione: (2024)

AdAdaGrad: Adaptive Batch Size Schemes for Adaptive Gradient Methods
di: Lau, Tim Tsz-Kit, et al.
Pubblicazione: (2024)

Parameter-free Clipped Gradient Descent Meets Polyak
di: Takezawa, Yuki, et al.
Pubblicazione: (2024)

Remove that Square Root: A New Efficient Scale-Invariant Version of AdaGrad
di: Choudhury, Sayantan, et al.
Pubblicazione: (2024)

Communication-Efficient Adaptive Batch Size Strategies for Distributed Local Gradient Methods
di: Lau, Tim Tsz-Kit, et al.
Pubblicazione: (2024)

Extragradient Method for $(L_0, L_1)$-Lipschitz Root-finding Problems
di: Choudhury, Sayantan, et al.
Pubblicazione: (2025)

The Ky Fan Norms and Beyond: Dual Norms and Combinations for Matrix Optimization
di: Kravatskiy, Alexey, et al.
Pubblicazione: (2025)

Communication-Efficient Gradient Descent-Accent Methods for Distributed Variational Inequalities: Unified Analysis and Local Updates
di: Zhang, Siqi, et al.
Pubblicazione: (2023)

Where Does Warm-Up Come From? Adaptive Scheduling for Norm-Constrained Optimizers
di: Riabinin, Artem, et al.
Pubblicazione: (2026)

Preconditioned Norms: A Unified Framework for Steepest Descent, Quasi-Newton and Adaptive Methods
di: Veprikov, Andrey, et al.
Pubblicazione: (2025)

Fully Stochastic Trust-Region Sequential Quadratic Programming for Equality-Constrained Optimization Problems
di: Fang, Yuchen, et al.
Pubblicazione: (2022)

Random-reshuffled SARAH does not need a full gradient computations
di: Beznosikov, Aleksandr, et al.
Pubblicazione: (2021)

From Gradient Clipping to Normalization for Heavy Tailed SGD
di: Hübler, Florian, et al.
Pubblicazione: (2024)

Multiplayer Federated Learning: Reaching Equilibrium with Less Communication
di: Yoon, TaeHo, et al.
Pubblicazione: (2025)

Stochastic Gradient Methods with Preconditioned Updates
di: Sadiev, Abdurakhmon, et al.
Pubblicazione: (2022)

Adaptive Batch Size Schedules for Distributed Training of Language Models with Data and Model Parallelism
di: Lau, Tim Tsz-Kit, et al.
Pubblicazione: (2024)

Regularized Gradient Clipping Provably Trains Wide and Deep Neural Networks
di: Tucat, Matteo, et al.
Pubblicazione: (2024)

Pessimism Meets Risk: Risk-Sensitive Offline Reinforcement Learning
di: Zhang, Dake, et al.
Pubblicazione: (2024)

Muon Optimizes Under Spectral Norm Constraints
di: Chen, Lizhang, et al.
Pubblicazione: (2025)

Trust-Region Sequential Quadratic Programming for Stochastic Optimization with Random Models
di: Fang, Yuchen, et al.
Pubblicazione: (2024)

Clipped Gradient Methods for Nonsmooth Convex Optimization under Heavy-Tailed Noise: A Refined Analysis
di: Liu, Zijian
Pubblicazione: (2025)

Small Gradient Norm Regret for Online Convex Optimization
di: Gao, Wenzhi, et al.
Pubblicazione: (2026)

Nonconvex Stochastic Optimization under Heavy-Tailed Noises: Optimal Convergence without Gradient Clipping
di: Liu, Zijian, et al.
Pubblicazione: (2024)

Differentially Private Clipped-SGD: High-Probability Convergence with Arbitrary Clipping Level
di: Khah, Saleh Vatan, et al.
Pubblicazione: (2025)

Revisiting Gradient Normalization and Clipping for Nonconvex SGD under Heavy-Tailed Noise: Necessity, Sufficiency, and Acceleration
di: Sun, Tao, et al.
Pubblicazione: (2024)

Federated Learning Can Find Friends That Are Advantageous
di: Tupitsa, Nazarii, et al.
Pubblicazione: (2024)

AdaBatchGrad: Combining Adaptive Batch Size and Adaptive Step Size
di: Ostroukhov, Petr, et al.
Pubblicazione: (2024)

Simple Stepsize for Quasi-Newton Methods with Global Convergence Guarantees
di: Agafonov, Artem, et al.
Pubblicazione: (2025)

Convergence of Alternating Gradient Descent for Matrix Factorization
di: Ward, Rachel, et al.
Pubblicazione: (2023)

SANIA: Polyak-type Optimization Framework Leads to Scale Invariant Stochastic Algorithms
di: Abdukhakimov, Farshed, et al.
Pubblicazione: (2023)

LoFT: Low-Rank Adaptation That Behaves Like Full Fine-Tuning
di: Tastan, Nurbek, et al.
Pubblicazione: (2025)

Non-Asymptotic Global Convergence of PPO-Clip
di: Liu, Yin, et al.
Pubblicazione: (2025)

Bayesian Optimization with Structured Measurements: A Vector-Valued RKHS Framework
di: Wang, Wenbin, et al.
Pubblicazione: (2026)

Robust and Fast Training via Per-Sample Clipping
di: Nobile, Davide, et al.
Pubblicazione: (2026)

Exploring New Frontiers in Vertical Federated Learning: the Role of Saddle Point Reformulation
di: Beznosikov, Aleksandr, et al.
Pubblicazione: (2026)

Preconditioned Gradient Descent for Over-Parameterized Nonconvex Matrix Factorization
di: Zhang, Gavin, et al.
Pubblicazione: (2025)

Convergence of Gradient Descent with Small Initialization for Unregularized Matrix Completion
di: Ma, Jianhao, et al.
Pubblicazione: (2024)

Gradient-Free Approaches is a Key to an Efficient Interaction with Markovian Stochasticity
di: Prokhorov, Boris, et al.
Pubblicazione: (2026)