Imagen de Portada

$μ^2$-SGD: Stable Stochastic Optimization via a Double Momentum Mechanism

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Dahan, Tehila, Levy, Kfir Y.
Formato:	Preprint
Publicado:	2023
Materias:	Machine Learning Optimization and Control
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

SLowcal-SGD: Slow Query Points Improve Local-SGD for Stochastic Convex Optimization
por: Dahan, Tehila, et al.
Publicado: (2023)

Bringing Order to Asynchronous SGD: Towards Optimality under Data-Dependent Delays with Momentum
por: Dahan, Tehila, et al.
Publicado: (2026)

On the Provable Suboptimality of Momentum SGD in Nonstationary Stochastic Optimization
por: Sahu, Sharan, et al.
Publicado: (2026)

Weight for Robustness: A Comprehensive Approach towards Optimal Fault-Tolerant Asynchronous ML
por: Dahan, Tehila, et al.
Publicado: (2025)

Fault Tolerant ML: Efficient Meta-Aggregation and Synchronous Training
por: Dahan, Tehila, et al.
Publicado: (2024)

Dimension-adapted Momentum Outscales SGD
por: Ferbach, Damien, et al.
Publicado: (2025)

Understanding Outer Optimizers in Local SGD: Learning Rates, Momentum, and Acceleration
por: Khaled, Ahmed, et al.
Publicado: (2025)

SketchySGD: Reliable Stochastic Optimization via Randomized Curvature Estimates
por: Frangella, Zachary, et al.
Publicado: (2022)

Leveraging Coordinate Momentum in SignSGD and Muon: Memory-Optimized Zero-Order
por: Petrov, Egor, et al.
Publicado: (2025)

The Marginal Value of Momentum for Small Learning Rate SGD
por: Wang, Runzhe, et al.
Publicado: (2023)

SGD with Adaptive Preconditioning: Unified Analysis and Momentum Acceleration
por: Kovalev, Dmitry
Publicado: (2025)

Stochastic Compositional Optimization via Hybrid Momentum Frank--Wolfe
por: Chayti, El Mahdi
Publicado: (2026)

Stochastic Difference-of-Convex Optimization with Momentum
por: Chayti, El Mahdi, et al.
Publicado: (2025)

Gradient-Variation Online Adaptivity for Accelerated Optimization with Hölder Smoothness
por: Zhao, Yuheng, et al.
Publicado: (2025)

Demystifying SGD with Doubly Stochastic Gradients
por: Kim, Kyurae, et al.
Publicado: (2024)

Sign-SGD via Parameter-Free Optimization
por: Medyakov, Daniil, et al.
Publicado: (2025)

Enhancing Stochastic Optimization for Statistical Efficiency Using ROOT-SGD with Diminishing Stepsize
por: Li, Chris Junchi
Publicado: (2024)

Non-convex Stochastic Composite Optimization with Polyak Momentum
por: Gao, Yuan, et al.
Publicado: (2024)

Compressed Decentralized Momentum Stochastic Gradient Methods for Nonconvex Optimization
por: Liu, Wei, et al.
Publicado: (2025)

Edge of Stochastic Stability: Revisiting the Edge of Stability for SGD
por: Andreyev, Arseniy, et al.
Publicado: (2024)

Accelerated Stochastic Min-Max Optimization Based on Bias-corrected Momentum
por: Cai, Haoyuan, et al.
Publicado: (2024)

A Hessian-Aware Stochastic Differential Equation for Modelling SGD
por: Li, Xiang, et al.
Publicado: (2024)

Provable Complexity Improvement of AdaGrad over SGD: Upper and Lower Bounds in Stochastic Non-Convex Optimization
por: Jiang, Ruichen, et al.
Publicado: (2024)

Improving Stochastic Cubic Newton with Momentum
por: Chayti, El Mahdi, et al.
Publicado: (2024)

Stability and Generalization for Stochastic Recursive Momentum-based Algorithms for (Strongly-)Convex One to $K$-Level Stochastic Optimizations
por: Pan, Xiaokang, et al.
Publicado: (2024)

Momentum Does Not Reduce Stochastic Noise in Stochastic Gradient Descent
por: Sato, Naoki, et al.
Publicado: (2024)

From PowerSGD to PowerSGD+: Low-Rank Gradient Compression for Distributed Optimization with Convergence Guarantees
por: Xie, Shengping, et al.
Publicado: (2025)

The Optimality of (Accelerated) SGD for High-Dimensional Quadratic Optimization
por: Zhang, Haihan, et al.
Publicado: (2024)

SGD with Partial Hessian for Deep Neural Networks Optimization
por: Sun, Ying, et al.
Publicado: (2024)

Optimal Projection-Free Adaptive SGD for Matrix Optimization
por: Kovalev, Dmitry
Publicado: (2026)

Adaptive Optimization via Momentum on Variance-Normalized Gradients
por: Patitucci, Francisco, et al.
Publicado: (2026)

Convergence of Clipped SGD on Convex $(L_0,L_1)$-Smooth Functions
por: Gaash, Ofir, et al.
Publicado: (2025)

Double Momentum Method for Lower-Level Constrained Bilevel Optimization
por: Shi, Wanli, et al.
Publicado: (2024)

SGD at the Edge of Stability: The Stochastic Sharpness Gap
por: Liao, Fangshuo, et al.
Publicado: (2026)

(Accelerated) Noise-adaptive Stochastic Heavy-Ball Momentum
por: Dang, Anh, et al.
Publicado: (2024)

Dynamics of Stochastic Momentum with Sparse Updates in High Dimensions
por: Everett, Katie, et al.
Publicado: (2026)

AutoSGD: Automatic Learning Rate Selection for Stochastic Gradient Descent
por: Surjanovic, Nikola, et al.
Publicado: (2025)

StoSignSGD: Unbiased Structural Stochasticity Fixes SignSGD for Training Large Language Models
por: Yu, Dingzhi, et al.
Publicado: (2026)

Double Momentum and Error Feedback for Clipping with Fast Rates and Differential Privacy
por: Islamov, Rustem, et al.
Publicado: (2025)

Non-Euclidean SGD for Structured Optimization: Unified Analysis and Improved Rates
por: Kovalev, Dmitry, et al.
Publicado: (2025)