Image de couverture de livre

Dual-Delayed Asynchronous SGD for Arbitrarily Heterogeneous Data

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Wang, Xiaolu, Sun, Yuchang, Wai, Hoi-To, Zhang, Jun
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning Optimization and Control
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Asynchronous and Stochastic Distributed Resource Allocation
par: Li, Qiang, et autres
Publié: (2025)

Shadowheart SGD: Distributed Asynchronous SGD with Optimal Time Complexity Under Arbitrary Computation and Communication Heterogeneity
par: Tyurin, Alexander, et autres
Publié: (2024)

Clipped SGD Algorithms for Performative Prediction: Tight Bounds for Clipping Bias and Remedies
par: Li, Qiang, et autres
Publié: (2024)

First Provably Optimal Asynchronous SGD for Homogeneous and Heterogeneous Data
par: Maranjyan, Artavazd
Publié: (2026)

Rescaled Asynchronous SGD: Optimal Distributed Optimization under Data and System Heterogeneity
par: Mahran, Ammar, et autres
Publié: (2026)

Ringleader ASGD: The First Asynchronous SGD with Optimal Time Complexity under Data Heterogeneity
par: Maranjyan, Artavazd, et autres
Publié: (2025)

Asynchronous Decentralized SGD under Non-Convexity: A Block-Coordinate Descent Framework
par: Zhou, Yijie, et autres
Publié: (2025)

Asynchronous Distributed Optimization with Delay-free Parameters
par: Wu, Xuyang, et autres
Publié: (2023)

Stochastic Gradient Descent with Strategic Querying
par: Jiang, Nanfei, et autres
Publié: (2025)

Birch SGD: A Tree Graph Framework for Local and Asynchronous SGD Methods
par: Tyurin, Alexander, et autres
Publié: (2025)

SGD with Partial Hessian for Deep Neural Networks Optimization
par: Sun, Ying, et autres
Publié: (2024)

Faster Stochastic Optimization with Arbitrary Delays via Asynchronous Mini-Batching
par: Attia, Amit, et autres
Publié: (2024)

Revisiting Multi-Agent Asynchronous Online Optimization with Delays: the Strongly Convex Case
par: Bao, Lingchan, et autres
Publié: (2025)

Empirical Risk Minimization with Shuffled SGD: A Primal-Dual Perspective and Improved Bounds
par: Cai, Xufeng, et autres
Publié: (2023)

Ringmaster ASGD: The First Asynchronous SGD with Optimal Time Complexity
par: Maranjyan, Artavazd, et autres
Publié: (2025)

SLowcal-SGD: Slow Query Points Improve Local-SGD for Stochastic Convex Optimization
par: Dahan, Tehila, et autres
Publié: (2023)

Making SGD Parameter-Free
par: Carmon, Yair, et autres
Publié: (2022)

On the Trajectories of SGD Without Replacement
par: Beneventano, Pierfrancesco
Publié: (2023)

The Marginal Value of Momentum for Small Learning Rate SGD
par: Wang, Runzhe, et autres
Publié: (2023)

LOSCAR-SGD: Local SGD with Communication-Computation Overlap and Delay-Corrected Sparse Model Averaging
par: Maziane, Yassine, et autres
Publié: (2026)

The Optimality of (Accelerated) SGD for High-Dimensional Quadratic Optimization
par: Zhang, Haihan, et autres
Publié: (2024)

From PowerSGD to PowerSGD+: Low-Rank Gradient Compression for Distributed Optimization with Convergence Guarantees
par: Xie, Shengping, et autres
Publié: (2025)

Demystifying SGD with Doubly Stochastic Gradients
par: Kim, Kyurae, et autres
Publié: (2024)

Dimension-adapted Momentum Outscales SGD
par: Ferbach, Damien, et autres
Publié: (2025)

Heavy-Tail Phenomenon in Decentralized SGD
par: Gurbuzbalaban, Mert, et autres
Publié: (2022)

Phases of Muon: When Muon Eclipses SignSGD
par: Paquette, Elliot, et autres
Publié: (2026)

Shuffling the Data, Stretching the Step-size: Sharper Bias in constant step-size SGD
par: Emmanouilidis, Konstantinos, et autres
Publié: (2026)

A Hessian-Aware Stochastic Differential Equation for Modelling SGD
par: Li, Xiang, et autres
Publié: (2024)

SGD with memory: fundamental properties and stochastic acceleration
par: Yarotsky, Dmitry, et autres
Publié: (2024)

Does SGD really happen in tiny subspaces?
par: Song, Minhak, et autres
Publié: (2024)

The Rich and the Simple: On the Implicit Bias of Adam and SGD
par: Vasudeva, Bhavya, et autres
Publié: (2025)

Sign-SGD via Parameter-Free Optimization
par: Medyakov, Daniil, et autres
Publié: (2025)

Can SGD Handle Heavy-Tailed Noise?
par: Fatkhullin, Ilyas, et autres
Publié: (2025)

MindFlayer SGD: Efficient Parallel SGD in the Presence of Heterogeneous and Random Worker Compute Times
par: Maranjyan, Artavazd, et autres
Publié: (2024)

Revisiting Gradient Normalization and Clipping for Nonconvex SGD under Heavy-Tailed Noise: Necessity, Sufficiency, and Acceleration
par: Sun, Tao, et autres
Publié: (2024)

A Comprehensive Framework for Analyzing the Convergence of Adam: Bridging the Gap with SGD
par: Jin, Ruinan, et autres
Publié: (2024)

EMA-Nesterov: Stabilizing Nesterov's Lookahead for Accelerated Deep Learning Optimization
par: Yau, Chung-Yiu, et autres
Publié: (2026)

Federated Majorize-Minimization: Beyond Parameter Aggregation
par: Dieuleveut, Aymeric, et autres
Publié: (2025)

Byzantine-Robust Distributed SGD: A Unified Analysis and Tight Error Bounds
par: Ruan, Boyuan, et autres
Publié: (2026)

Edge of Stochastic Stability: Revisiting the Edge of Stability for SGD
par: Andreyev, Arseniy, et autres
Publié: (2024)