Imagen de Portada

Does SGD Seek Flatness or Sharpness? An Exactly Solvable Model

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Xu, Yizhou, Beneventano, Pierfrancesco, Chuang, Isaac, Ziyin, Liu
Formato:	Preprint
Publicado:	2026
Materias:	Machine Learning Optimization and Control
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

On the Trajectories of SGD Without Replacement
por: Beneventano, Pierfrancesco
Publicado: (2023)

Edge of Stochastic Stability: Revisiting the Edge of Stability for SGD
por: Andreyev, Arseniy, et al.
Publicado: (2024)

How Neural Networks Learn the Support is an Implicit Regularization Effect of SGD
por: Beneventano, Pierfrancesco, et al.
Publicado: (2024)

Gradient Descent Converges Linearly to Flatter Minima than Gradient Flow in Shallow Linear Networks
por: Beneventano, Pierfrancesco, et al.
Publicado: (2025)

Does Weight Decay Enhance Training Stability?
por: Saether, Marius, et al.
Publicado: (2026)

Too Sharp, Too Sure: When Calibration Follows Curvature
por: Morosini, Alessandro, et al.
Publicado: (2026)

Momentum Further Constrains Sharpness at the Edge of Stochastic Stability
por: Andreyev, Arseniy, et al.
Publicado: (2026)

Do Deep Networks Forget Initialization? A Forgetting-Time View of Practical Inductive Bias
por: Das, Mohua, et al.
Publicado: (2026)

Does SGD really happen in tiny subspaces?
por: Song, Minhak, et al.
Publicado: (2024)

ROOT-SGD: Sharp Nonasymptotics and Near-Optimal Asymptotics in a Single Algorithm
por: Li, Chris Junchi, et al.
Publicado: (2020)

SGD at the Edge of Stability: The Stochastic Sharpness Gap
por: Liao, Fangshuo, et al.
Publicado: (2026)

Sharp High-Probability Rates for Nonlinear SGD under Heavy-Tailed Noise via Symmetrization
por: Armacki, Aleksandar, et al.
Publicado: (2025)

Scaling Laws of SignSGD in Linear Regression: When Does It Outperform SGD?
por: Kim, Jihwan, et al.
Publicado: (2026)

Schrödinger Bridge with Quadratic State Cost is Exactly Solvable
por: Teter, Alexis M. H., et al.
Publicado: (2024)

Does Worst-Performing Agent Lead the Pack? Analyzing Agent Dynamics in Unified Distributed SGD
por: Hu, Jie, et al.
Publicado: (2024)

Weyl Calculus and Exactly Solvable Schrödinger Bridges with Quadratic State Cost
por: Teter, Alexis M. H., et al.
Publicado: (2024)

Faster Convergence of Local SGD for Over-Parameterized Models
por: Qin, Tiancheng, et al.
Publicado: (2022)

SLowcal-SGD: Slow Query Points Improve Local-SGD for Stochastic Convex Optimization
por: Dahan, Tehila, et al.
Publicado: (2023)

Making SGD Parameter-Free
por: Carmon, Yair, et al.
Publicado: (2022)

The Optimality of (Accelerated) SGD for High-Dimensional Quadratic Optimization
por: Zhang, Haihan, et al.
Publicado: (2024)

A Hessian-Aware Stochastic Differential Equation for Modelling SGD
por: Li, Xiang, et al.
Publicado: (2024)

From PowerSGD to PowerSGD+: Low-Rank Gradient Compression for Distributed Optimization with Convergence Guarantees
por: Xie, Shengping, et al.
Publicado: (2025)

Shadowheart SGD: Distributed Asynchronous SGD with Optimal Time Complexity Under Arbitrary Computation and Communication Heterogeneity
por: Tyurin, Alexander, et al.
Publicado: (2024)

StoSignSGD: Unbiased Structural Stochasticity Fixes SignSGD for Training Large Language Models
por: Yu, Dingzhi, et al.
Publicado: (2026)

Dimension-adapted Momentum Outscales SGD
por: Ferbach, Damien, et al.
Publicado: (2025)

Heavy-Tail Phenomenon in Decentralized SGD
por: Gurbuzbalaban, Mert, et al.
Publicado: (2022)

Demystifying SGD with Doubly Stochastic Gradients
por: Kim, Kyurae, et al.
Publicado: (2024)

Parameter Symmetry and Noise Equilibrium of Stochastic Gradient Descent
por: Ziyin, Liu, et al.
Publicado: (2024)

The Rich and the Simple: On the Implicit Bias of Adam and SGD
por: Vasudeva, Bhavya, et al.
Publicado: (2025)

Sign-SGD via Parameter-Free Optimization
por: Medyakov, Daniil, et al.
Publicado: (2025)

Can SGD Handle Heavy-Tailed Noise?
por: Fatkhullin, Ilyas, et al.
Publicado: (2025)

SGD with memory: fundamental properties and stochastic acceleration
por: Yarotsky, Dmitry, et al.
Publicado: (2024)

Diagonalisation SGD: Fast & Convergent SGD for Non-Differentiable Models via Reparameterisation and Smoothing
por: Wagner, Dominik, et al.
Publicado: (2024)

Byzantine-Robust Distributed SGD: A Unified Analysis and Tight Error Bounds
por: Ruan, Boyuan, et al.
Publicado: (2026)

Catapults in SGD: spikes in the training loss and their impact on generalization through feature learning
por: Zhu, Libin, et al.
Publicado: (2023)

Phases of Muon: When Muon Eclipses SignSGD
por: Paquette, Elliot, et al.
Publicado: (2026)

Optimal Projection-Free Adaptive SGD for Matrix Optimization
por: Kovalev, Dmitry
Publicado: (2026)

On the Provable Suboptimality of Momentum SGD in Nonstationary Stochastic Optimization
por: Sahu, Sharan, et al.
Publicado: (2026)

Accelerating Single-Pass SGD for Generalized Linear Prediction
por: Chen, Qian, et al.
Publicado: (2026)

From Gradient Clipping to Normalization for Heavy Tailed SGD
por: Hübler, Florian, et al.
Publicado: (2024)