Copertina

Perfect Parallelization in Mini-Batch SGD with Classical Momentum Acceleration

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Garg, Sachin, Dereziński, Michał
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Second-order Information Promotes Mini-Batch Robustness in Variance-Reduced Gradients
di: Garg, Sachin, et al.
Pubblicazione: (2024)

Stochastic Variance-Reduced Newton: Accelerating Finite-Sum Minimization with Large Batches
di: Dereziński, Michał
Pubblicazione: (2022)

Faster Low-Rank Approximation and Kernel Ridge Regression via the Block-Nyström Method
di: Garg, Sachin, et al.
Pubblicazione: (2025)

Distributed Least Squares in Small Space via Sketching and Bias Reduction
di: Garg, Sachin, et al.
Pubblicazione: (2024)

Last-Iterate Convergence of Randomized Kaczmarz and SGD with Greedy Step Size
di: Dereziński, Michał, et al.
Pubblicazione: (2026)

Debiasing Random Oblique Projections for Subsampled OLS and Fast CUR in High Dimensions
di: Niu, Chengmei, et al.
Pubblicazione: (2026)

Accelerating Power Method with Fast Sketching for Stronger Low-Rank Approximation
di: Chenakkod, Shabarish, et al.
Pubblicazione: (2026)

Turbocharging Gaussian Process Inference with Approximate Sketch-and-Project
di: Rathore, Pratik, et al.
Pubblicazione: (2025)

SGD with Adaptive Preconditioning: Unified Analysis and Momentum Acceleration
di: Kovalev, Dmitry
Pubblicazione: (2025)

From One-Pass SGD to Data Reuse: Mini-Batch Scaling Laws in Sketched Linear Regression
di: Chen, Ziyan, et al.
Pubblicazione: (2026)

Understanding Outer Optimizers in Local SGD: Learning Rates, Momentum, and Acceleration
di: Khaled, Ahmed, et al.
Pubblicazione: (2025)

Dimension-adapted Momentum Outscales SGD
di: Ferbach, Damien, et al.
Pubblicazione: (2025)

Ordered Momentum for Asynchronous SGD
di: Shi, Chang-Wei, et al.
Pubblicazione: (2024)

Recent and Upcoming Developments in Randomized Numerical Linear Algebra for Machine Learning
di: Dereziński, Michał, et al.
Pubblicazione: (2024)

Approaching Optimality for Solving Dense Linear Systems with Low-Rank Structure
di: Dereziński, Michał, et al.
Pubblicazione: (2025)

Solving Dense Linear Systems Faster Than via Preconditioning
di: Dereziński, Michał, et al.
Pubblicazione: (2023)

Stochastic Newton Proximal Extragradient Method
di: Jiang, Ruichen, et al.
Pubblicazione: (2024)

On the Provable Suboptimality of Momentum SGD in Nonstationary Stochastic Optimization
di: Sahu, Sharan, et al.
Pubblicazione: (2026)

The Marginal Value of Momentum for Small Learning Rate SGD
di: Wang, Runzhe, et al.
Pubblicazione: (2023)

Signal Processing Meets SGD: From Momentum to Filter
di: Yao, Zhipeng, et al.
Pubblicazione: (2023)

Debiasing Mini-Batch Quadratics for Applications in Deep Learning
di: Tatzel, Lukas, et al.
Pubblicazione: (2024)

High-dimensional limit theorems for SGD: Momentum and Adaptive Step-sizes
di: Jagannath, Aukosh, et al.
Pubblicazione: (2025)

HERTA: A High-Efficiency and Rigorous Training Algorithm for Unfolded Graph Neural Networks
di: Yang, Yongyi, et al.
Pubblicazione: (2024)

Enhancing SignSGD: Small-Batch Convergence Analysis and a Hybrid Switching Strategy
di: Chen, Haoran, et al.
Pubblicazione: (2026)

Mini-Batch Kernel $k$-means
di: Jourdan, Ben, et al.
Pubblicazione: (2024)

Mini-Batch Class Composition Bias in Link Prediction
di: Maguire, Kieran, et al.
Pubblicazione: (2026)

Stochastic Normalized Gradient Descent with Momentum for Large-Batch Training
di: Zhao, Shen-Yi, et al.
Pubblicazione: (2020)

Bringing Order to Asynchronous SGD: Towards Optimality under Data-Dependent Delays with Momentum
di: Dahan, Tehila, et al.
Pubblicazione: (2026)

Can Microcanonical Langevin Dynamics Leverage Mini-Batch Gradient Noise?
di: Sommer, Emanuel, et al.
Pubblicazione: (2026)

SGD for Variational Inference: Tackling Unbounded Variance via Preconditioning and Dynamic Batching
di: Labarrière, Hippolyte, et al.
Pubblicazione: (2026)

Optimal Oblivious Subspace Embeddings with Near-optimal Sparsity
di: Chenakkod, Shabarish, et al.
Pubblicazione: (2024)

Optimal Subspace Embeddings: Resolving Nelson-Nguyen Conjecture Up to Sub-Polylogarithmic Factors
di: Chenakkod, Shabarish, et al.
Pubblicazione: (2025)

Pseudo-Asynchronous Local SGD: Robust and Efficient Data-Parallel Training
di: Naganuma, Hiroki, et al.
Pubblicazione: (2025)

Increasing Batch Size Improves Convergence of Stochastic Gradient Descent with Momentum
di: Kamo, Keisuke, et al.
Pubblicazione: (2025)

Towards Universal Convergence of Backward Error in Linear System Solvers
di: Dereziński, Michał, et al.
Pubblicazione: (2026)

Faster Linear Systems and Matrix Norm Approximation via Multi-level Sketched Preconditioning
di: Dereziński, Michał, et al.
Pubblicazione: (2024)

Leveraging Coordinate Momentum in SignSGD and Muon: Memory-Optimized Zero-Order
di: Petrov, Egor, et al.
Pubblicazione: (2025)

Full-Graph vs. Mini-Batch Training: Comprehensive Analysis from a Batch Size and Fan-Out Size Perspective
di: Liu, Mengfan, et al.
Pubblicazione: (2026)

Optimal Growth Schedules for Batch Size and Learning Rate in SGD that Reduce SFO Complexity
di: Umeda, Hikaru, et al.
Pubblicazione: (2025)

Hierarchical Rectified Flow Matching with Mini-Batch Couplings
di: Zhang, Yichi, et al.
Pubblicazione: (2025)