Image de couverture de livre

Non-Euclidean Gradient Descent Operates at the Edge of Stability

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Islamov, Rustem, Crawshaw, Michael, Cohen, Jeremy, Gower, Robert
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning Optimization and Control
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Safe-EF: Error Feedback for Nonsmooth Constrained Optimization
par: Islamov, Rustem, et autres
Publié: (2025)

Enhancing Optimizer Stability: Momentum Adaptation of The NGN Step-size
par: Islamov, Rustem, et autres
Publié: (2025)

Complexity Lower Bounds of Adaptive Gradient Algorithms for Non-convex Stochastic Optimization under Relaxed Smoothness
par: Crawshaw, Michael, et autres
Publié: (2025)

Why Do We Need Warm-up? A Theoretical Perspective
par: Alimisis, Foivos, et autres
Publié: (2025)

Muon Does Not Converge on Convex Lipschitz Functions
par: Parshakova, Tetiana, et autres
Publié: (2026)

On the Role of Batch Size in Stochastic Conditional Gradient Methods
par: Islamov, Rustem, et autres
Publié: (2026)

Loss Landscape Characterization of Neural Networks without Over-Parametrization
par: Islamov, Rustem, et autres
Publié: (2024)

An Exploration of Non-Euclidean Gradient Descent: Muon and its Many Variants
par: Crawshaw, Michael, et autres
Publié: (2025)

Towards Faster Decentralized Stochastic Optimization with Communication Compression
par: Islamov, Rustem, et autres
Publié: (2024)

Double Momentum and Error Feedback for Clipping with Fast Rates and Differential Privacy
par: Islamov, Rustem, et autres
Publié: (2025)

Convergence Rates for Gradient Descent on the Edge of Stability in Overparametrised Least Squares
par: MacDonald, Lachlan Ewen, et autres
Publié: (2025)

Implicit Bias of Gradient Descent for Non-Homogeneous Deep Networks
par: Cai, Yuhang, et autres
Publié: (2025)

Rod Flow: A Continuous-Time Model for Gradient Descent at the Edge of Stability
par: Regis, Eric, et autres
Publié: (2026)

Byzantine-Robust and Differentially Private Federated Optimization under Weaker Assumptions
par: Islamov, Rustem, et autres
Publié: (2026)

Directional Smoothness and Gradient Methods: Convergence and Adaptivity
par: Mishkin, Aaron, et autres
Publié: (2024)

Stochastic Adaptive Gradient Descent Without Descent
par: Aujol, Jean-François, et autres
Publié: (2025)

Corner Gradient Descent
par: Yarotsky, Dmitry
Publié: (2025)

Adaptive Conditional Gradient Descent
par: Khademi, Abbas, et autres
Publié: (2025)

$k$-SVD with Gradient Descent
par: Jedra, Yassir, et autres
Publié: (2025)

On Gradient Descent Ascent for Nonconvex-Concave Minimax Problems
par: Lin, Tianyi, et autres
Publié: (2019)

Large Stepsize Gradient Descent for Non-Homogeneous Two-Layer Networks: Margin Improvement and Fast Optimization
par: Cai, Yuhang, et autres
Publié: (2024)

Gradient Descent on Logistic Regression with Non-Separable Data and Large Step Sizes
par: Meng, Si Yi, et autres
Publié: (2024)

Non-Singularity of the Gradient Descent map for Neural Networks with Piecewise Analytic Activations
par: Crăciun, Alexandru, et autres
Publié: (2025)

Understanding Gradient Orthogonalization for Deep Learning via Non-Euclidean Trust-Region Optimization
par: Kovalev, Dmitry
Publié: (2025)

Anytime Acceleration of Gradient Descent
par: Zhang, Zihan, et autres
Publié: (2024)

Controlling the Flow: Stability and Convergence for Stochastic Gradient Descent with Decaying Regularization
par: Kassing, Sebastian, et autres
Publié: (2025)

More Optimal Fractional-Order Stochastic Gradient Descent for Non-Convex Optimization Problems
par: Partohaghighi, Mohammad, et autres
Publié: (2025)

Stochastic Gradient Descent with Adaptive Data
par: Che, Ethan, et autres
Publié: (2024)

Stochastic Gradient Descent with Strategic Querying
par: Jiang, Nanfei, et autres
Publié: (2025)

Unraveling the Gradient Descent Dynamics of Transformers
par: Song, Bingqing, et autres
Publié: (2024)

Algorithmic Stability of Stochastic Gradient Descent with Momentum under Heavy-Tailed Noise
par: Dang, Thanh, et autres
Publié: (2025)

Two-Timescale Gradient Descent Ascent Algorithms for Nonconvex Minimax Optimization
par: Lin, Tianyi, et autres
Publié: (2024)

Step-Size Stability in Stochastic Optimization: A Theoretical Perspective
par: Schaipp, Fabian, et autres
Publié: (2026)

Level Set Teleportation: An Optimization Perspective
par: Mishkin, Aaron, et autres
Publié: (2024)

Learning Provably Improves the Convergence of Gradient Descent
par: Song, Qingyu, et autres
Publié: (2025)

Mirror and Preconditioned Gradient Descent in Wasserstein Space
par: Bonet, Clément, et autres
Publié: (2024)

Enhancing Fractional Gradient Descent with Learned Optimizers
par: Sobotka, Jan, et autres
Publié: (2025)

Derivatives of Stochastic Gradient Descent in parametric optimization
par: Iutzeler, Franck, et autres
Publié: (2024)

Convergence of Alternating Gradient Descent for Matrix Factorization
par: Ward, Rachel, et autres
Publié: (2023)

On Penalty-based Bilevel Gradient Descent Method
par: Shen, Han, et autres
Publié: (2023)