Image de couverture de livre

On the Trajectories of SGD Without Replacement

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteur principal:	Beneventano, Pierfrancesco
Format:	Preprint
Publié:	2023
Sujets:	Machine Learning Optimization and Control
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Edge of Stochastic Stability: Revisiting the Edge of Stability for SGD
par: Andreyev, Arseniy, et autres
Publié: (2024)

How Neural Networks Learn the Support is an Implicit Regularization Effect of SGD
par: Beneventano, Pierfrancesco, et autres
Publié: (2024)

Does SGD Seek Flatness or Sharpness? An Exactly Solvable Model
par: Xu, Yizhou, et autres
Publié: (2026)

Gradient Descent Converges Linearly to Flatter Minima than Gradient Flow in Shallow Linear Networks
par: Beneventano, Pierfrancesco, et autres
Publié: (2025)

Does Weight Decay Enhance Training Stability?
par: Saether, Marius, et autres
Publié: (2026)

Too Sharp, Too Sure: When Calibration Follows Curvature
par: Morosini, Alessandro, et autres
Publié: (2026)

Do Deep Networks Forget Initialization? A Forgetting-Time View of Practical Inductive Bias
par: Das, Mohua, et autres
Publié: (2026)

Momentum Further Constrains Sharpness at the Edge of Stochastic Stability
par: Andreyev, Arseniy, et autres
Publié: (2026)

Provably Faster Algorithms for Bilevel Optimization via Without-Replacement Sampling
par: Li, Junyi, et autres
Publié: (2024)

SLowcal-SGD: Slow Query Points Improve Local-SGD for Stochastic Convex Optimization
par: Dahan, Tehila, et autres
Publié: (2023)

Making SGD Parameter-Free
par: Carmon, Yair, et autres
Publié: (2022)

From PowerSGD to PowerSGD+: Low-Rank Gradient Compression for Distributed Optimization with Convergence Guarantees
par: Xie, Shengping, et autres
Publié: (2025)

Shadowheart SGD: Distributed Asynchronous SGD with Optimal Time Complexity Under Arbitrary Computation and Communication Heterogeneity
par: Tyurin, Alexander, et autres
Publié: (2024)

Dimension-adapted Momentum Outscales SGD
par: Ferbach, Damien, et autres
Publié: (2025)

Heavy-Tail Phenomenon in Decentralized SGD
par: Gurbuzbalaban, Mert, et autres
Publié: (2022)

Demystifying SGD with Doubly Stochastic Gradients
par: Kim, Kyurae, et autres
Publié: (2024)

The Rich and the Simple: On the Implicit Bias of Adam and SGD
par: Vasudeva, Bhavya, et autres
Publié: (2025)

Sign-SGD via Parameter-Free Optimization
par: Medyakov, Daniil, et autres
Publié: (2025)

Can SGD Handle Heavy-Tailed Noise?
par: Fatkhullin, Ilyas, et autres
Publié: (2025)

SGD with memory: fundamental properties and stochastic acceleration
par: Yarotsky, Dmitry, et autres
Publié: (2024)

Does SGD really happen in tiny subspaces?
par: Song, Minhak, et autres
Publié: (2024)

The Marginal Value of Momentum for Small Learning Rate SGD
par: Wang, Runzhe, et autres
Publié: (2023)

From Gradient Clipping to Normalization for Heavy Tailed SGD
par: Hübler, Florian, et autres
Publié: (2024)

The Optimality of (Accelerated) SGD for High-Dimensional Quadratic Optimization
par: Zhang, Haihan, et autres
Publié: (2024)

Dual-Delayed Asynchronous SGD for Arbitrarily Heterogeneous Data
par: Wang, Xiaolu, et autres
Publié: (2024)

Faster Convergence of Local SGD for Over-Parameterized Models
par: Qin, Tiancheng, et autres
Publié: (2022)

Phases of Muon: When Muon Eclipses SignSGD
par: Paquette, Elliot, et autres
Publié: (2026)

SGD with Partial Hessian for Deep Neural Networks Optimization
par: Sun, Ying, et autres
Publié: (2024)

Optimal Projection-Free Adaptive SGD for Matrix Optimization
par: Kovalev, Dmitry
Publié: (2026)

SGD with Adaptive Preconditioning: Unified Analysis and Momentum Acceleration
par: Kovalev, Dmitry
Publié: (2025)

Global Convergence of SGD On Two Layer Neural Nets
par: Gopalani, Pulkit, et autres
Publié: (2022)

On the Provable Suboptimality of Momentum SGD in Nonstationary Stochastic Optimization
par: Sahu, Sharan, et autres
Publié: (2026)

Accelerating Single-Pass SGD for Generalized Linear Prediction
par: Chen, Qian, et autres
Publié: (2026)

A Simplified Analysis of SGD for Linear Regression with Weight Averaging
par: Meterez, Alexandru, et autres
Publié: (2025)

Fast Last-Iterate Convergence of SGD in the Smooth Interpolation Regime
par: Attia, Amit, et autres
Publié: (2025)

A Hessian-Aware Stochastic Differential Equation for Modelling SGD
par: Li, Xiang, et autres
Publié: (2024)

Global Convergence of SGD For Logistic Loss on Two Layer Neural Nets
par: Gopalani, Pulkit, et autres
Publié: (2023)

Understanding Outer Optimizers in Local SGD: Learning Rates, Momentum, and Acceleration
par: Khaled, Ahmed, et autres
Publié: (2025)

SketchySGD: Reliable Stochastic Optimization via Randomized Curvature Estimates
par: Frangella, Zachary, et autres
Publié: (2022)

A Precise Characterization of SGD Stability Using Loss Surface Geometry
par: Dexter, Gregory, et autres
Publié: (2024)