Copertina

Shadowheart SGD: Distributed Asynchronous SGD with Optimal Time Complexity Under Arbitrary Computation and Communication Heterogeneity

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Tyurin, Alexander, Pozzi, Marta, Ilin, Ivan, Richtárik, Peter
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Optimization and Control Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Ringmaster ASGD: The First Asynchronous SGD with Optimal Time Complexity
di: Maranjyan, Artavazd, et al.
Pubblicazione: (2025)

Freya PAGE: First Optimal Time Complexity for Large-Scale Nonconvex Finite-Sum Optimization with Heterogeneous Asynchronous Computations
di: Tyurin, Alexander, et al.
Pubblicazione: (2024)

Ringleader ASGD: The First Asynchronous SGD with Optimal Time Complexity under Data Heterogeneity
di: Maranjyan, Artavazd, et al.
Pubblicazione: (2025)

Rescaled Asynchronous SGD: Optimal Distributed Optimization under Data and System Heterogeneity
di: Mahran, Ammar, et al.
Pubblicazione: (2026)

Birch SGD: A Tree Graph Framework for Local and Asynchronous SGD Methods
di: Tyurin, Alexander, et al.
Pubblicazione: (2025)

Local SGD and Federated Averaging Through the Lens of Time Complexity
di: Fradin, Adrien, et al.
Pubblicazione: (2025)

A Computation and Communication Efficient Method for Distributed Nonconvex Problems in the Partial Participation Setting
di: Tyurin, Alexander, et al.
Pubblicazione: (2022)

On the Optimal Time Complexities in Decentralized Stochastic Asynchronous Optimization
di: Tyurin, Alexander, et al.
Pubblicazione: (2024)

MindFlayer SGD: Efficient Parallel SGD in the Presence of Heterogeneous and Random Worker Compute Times
di: Maranjyan, Artavazd, et al.
Pubblicazione: (2024)

Improving the Worst-Case Bidirectional Communication Complexity for Nonconvex Distributed Optimization under Function Similarity
di: Gruntkowska, Kaja, et al.
Pubblicazione: (2024)

LOSCAR-SGD: Local SGD with Communication-Computation Overlap and Delay-Corrected Sparse Model Averaging
di: Maziane, Yassine, et al.
Pubblicazione: (2026)

On the Convergence of DP-SGD with Adaptive Clipping
di: Shulgin, Egor, et al.
Pubblicazione: (2024)

Dual-Delayed Asynchronous SGD for Arbitrarily Heterogeneous Data
di: Wang, Xiaolu, et al.
Pubblicazione: (2024)

First Provably Optimal Asynchronous SGD for Homogeneous and Heterogeneous Data
di: Maranjyan, Artavazd
Pubblicazione: (2026)

Do We Need Asynchronous SGD? On the Near-Optimality of Synchronous Methods
di: Begunov, Grigory, et al.
Pubblicazione: (2026)

Tighter Performance Theory of FedExProx
di: Anyszka, Wojciech, et al.
Pubblicazione: (2024)

Ringmaster LMO: Asynchronous Linear Minimization Oracle Momentum Method
di: Sadiev, Abdurakhmon, et al.
Pubblicazione: (2026)

Bias-Optimal Bounds for SGD: A Computer-Aided Lyapunov Analysis
di: Cortild, Daniel, et al.
Pubblicazione: (2025)

From PowerSGD to PowerSGD+: Low-Rank Gradient Compression for Distributed Optimization with Convergence Guarantees
di: Xie, Shengping, et al.
Pubblicazione: (2025)

The Limits and Potentials of Local SGD for Distributed Heterogeneous Learning with Intermittent Communication
di: Patel, Kumar Kshitij, et al.
Pubblicazione: (2024)

Differentially Private Clipped-SGD: High-Probability Convergence with Arbitrary Clipping Level
di: Khah, Saleh Vatan, et al.
Pubblicazione: (2025)

Near-Optimal Convergence of Accelerated Gradient Methods under Generalized and $(L_0, L_1)$-Smoothness
di: Tyurin, Alexander
Pubblicazione: (2025)

The Optimality of (Accelerated) SGD for High-Dimensional Quadratic Optimization
di: Zhang, Haihan, et al.
Pubblicazione: (2024)

Optimal Projection-Free Adaptive SGD for Matrix Optimization
di: Kovalev, Dmitry
Pubblicazione: (2026)

Optimal Growth Schedules for Batch Size and Learning Rate in SGD that Reduce SFO Complexity
di: Umeda, Hikaru, et al.
Pubblicazione: (2025)

Tight Time Complexities in Parallel Stochastic Optimization with Arbitrary Computation Dynamics
di: Tyurin, Alexander
Pubblicazione: (2024)

Asynchronous Decentralized SGD under Non-Convexity: A Block-Coordinate Descent Framework
di: Zhou, Yijie, et al.
Pubblicazione: (2025)

SLowcal-SGD: Slow Query Points Improve Local-SGD for Stochastic Convex Optimization
di: Dahan, Tehila, et al.
Pubblicazione: (2023)

Making SGD Parameter-Free
di: Carmon, Yair, et al.
Pubblicazione: (2022)

On the Trajectories of SGD Without Replacement
di: Beneventano, Pierfrancesco
Pubblicazione: (2023)

Asynchronous Policy Gradient Aggregation for Efficient Distributed Reinforcement Learning
di: Tyurin, Alexander, et al.
Pubblicazione: (2025)

Demystifying SGD with Doubly Stochastic Gradients
di: Kim, Kyurae, et al.
Pubblicazione: (2024)

Dimension-adapted Momentum Outscales SGD
di: Ferbach, Damien, et al.
Pubblicazione: (2025)

Heavy-Tail Phenomenon in Decentralized SGD
di: Gurbuzbalaban, Mert, et al.
Pubblicazione: (2022)

ROOT-SGD: Sharp Nonasymptotics and Near-Optimal Asymptotics in a Single Algorithm
di: Li, Chris Junchi, et al.
Pubblicazione: (2020)

Scaling Laws of SignSGD in Linear Regression: When Does It Outperform SGD?
di: Kim, Jihwan, et al.
Pubblicazione: (2026)

Lower Bounds and Proximally Anchored SGD for Non-Convex Minimization Under Unbounded Variance
di: Fazla, Arda, et al.
Pubblicazione: (2026)

Byzantine-Robust Distributed SGD: A Unified Analysis and Tight Error Bounds
di: Ruan, Boyuan, et al.
Pubblicazione: (2026)

SGD with memory: fundamental properties and stochastic acceleration
di: Yarotsky, Dmitry, et al.
Pubblicazione: (2024)

Does SGD really happen in tiny subspaces?
di: Song, Minhak, et al.
Pubblicazione: (2024)