Cover Image

Ringleader ASGD: The First Asynchronous SGD with Optimal Time Complexity under Data Heterogeneity

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Maranjyan, Artavazd, Richtárik, Peter
Format:	Preprint
Published:	2025
Subjects:	Optimization and Control Distributed, Parallel, and Cluster Computing Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Ringmaster ASGD: The First Asynchronous SGD with Optimal Time Complexity
by: Maranjyan, Artavazd, et al.
Published: (2025)

Rescaled Asynchronous SGD: Optimal Distributed Optimization under Data and System Heterogeneity
by: Mahran, Ammar, et al.
Published: (2026)

First Provably Optimal Asynchronous SGD for Homogeneous and Heterogeneous Data
by: Maranjyan, Artavazd
Published: (2026)

MindFlayer SGD: Efficient Parallel SGD in the Presence of Heterogeneous and Random Worker Compute Times
by: Maranjyan, Artavazd, et al.
Published: (2024)

LOSCAR-SGD: Local SGD with Communication-Computation Overlap and Delay-Corrected Sparse Model Averaging
by: Maziane, Yassine, et al.
Published: (2026)

Rennala MVR: Improved Time Complexity for Parallel Stochastic Optimization via Momentum-Based Variance Reduction
by: Tovmasyan, Zhirayr, et al.
Published: (2026)

GradSkip: Communication-Accelerated Local Gradient Methods with Better Computational Complexity
by: Maranjyan, Artavazd, et al.
Published: (2022)

Ringmaster LMO: Asynchronous Linear Minimization Oracle Momentum Method
by: Sadiev, Abdurakhmon, et al.
Published: (2026)

LoCoDL: Communication-Efficient Distributed Learning with Local Training and Compression
by: Condat, Laurent, et al.
Published: (2024)

ATA: Adaptive Task Allocation for Efficient Resource Management in Distributed Machine Learning
by: Maranjyan, Artavazd, et al.
Published: (2025)

Birch SGD: A Tree Graph Framework for Local and Asynchronous SGD Methods
by: Tyurin, Alexander, et al.
Published: (2025)

Towards a Better Theoretical Understanding of Independent Subnetwork Training
by: Shulgin, Egor, et al.
Published: (2023)

The Limits and Potentials of Local SGD for Distributed Heterogeneous Learning with Intermittent Communication
by: Patel, Kumar Kshitij, et al.
Published: (2024)

On Biased Compression for Distributed Learning
by: Beznosikov, Aleksandr, et al.
Published: (2020)

Revisiting LocalSGD and SCAFFOLD: Improved Rates and Missing Analysis
by: Luo, Ruichen, et al.
Published: (2025)

Smoothed Normalization for Efficient Distributed Private Optimization
by: Shulgin, Egor, et al.
Published: (2025)

Asynchronous Policy Gradient Aggregation for Efficient Distributed Reinforcement Learning
by: Tyurin, Alexander, et al.
Published: (2025)

MAST: Model-Agnostic Sparsified Training
by: Demidovich, Yury, et al.
Published: (2023)

Byzantine Robustness and Partial Participation Can Be Achieved at Once: Just Clip Gradient Differences
by: Malinovsky, Grigory, et al.
Published: (2023)

Correlated Quantization for Faster Nonconvex Distributed Optimization
by: Panferov, Andrei, et al.
Published: (2024)

FedCanon: Non-Convex Composite Federated Learning with Efficient Proximal Operation on Heterogeneous Data
by: Zhou, Yuan, et al.
Published: (2025)

Accelerated Methods with Compressed Communications for Distributed Optimization Problems under Data Similarity
by: Bylinkin, Dmitry, et al.
Published: (2024)

Efficient Federated Learning against Heterogeneous and Non-stationary Client Unavailability
by: Xiang, Ming, et al.
Published: (2024)

FIARSE: Model-Heterogeneous Federated Learning via Importance-Aware Submodel Extraction
by: Wu, Feijie, et al.
Published: (2024)

Convergence Analysis of Decentralized ASGD
by: Tosi, Mauro DL, et al.
Published: (2023)

Achieving Near-Optimal Convergence for Distributed Minimax Optimization with Adaptive Stepsizes
by: Huang, Yan, et al.
Published: (2024)

Distributed Saddle-Point Problems: Lower Bounds, Near-Optimal and Robust Algorithms
by: Beznosikov, Aleksandr, et al.
Published: (2020)

Demystifying Why Local Aggregation Helps: Convergence Analysis of Hierarchical SGD
by: Wang, Jiayi, et al.
Published: (2020)

A Privacy Preserving Randomized Gossip Algorithm via Controlled Noise Insertion
by: Hanzely, Filip, et al.
Published: (2019)

Optimizing Stochastic Gradient Push under Broadcast Communications
by: Nguyen, Tuan, et al.
Published: (2026)

Do We Need Asynchronous SGD? On the Near-Optimality of Synchronous Methods
by: Begunov, Grigory, et al.
Published: (2026)

Continuous-Time Analysis of Federated Averaging
by: Overman, Tom, et al.
Published: (2025)

Online Distributed Learning with Quantized Finite-Time Coordination
by: Bastianello, Nicola, et al.
Published: (2023)

FADAS: Towards Federated Adaptive Asynchronous Optimization
by: Wang, Yujia, et al.
Published: (2024)

Time-varying Mixing Matrix Design for Energy-efficient Decentralized Federated Learning
by: Zhang, Xusheng, et al.
Published: (2025)

A Hybrid Stochastic Gradient Tracking Method for Distributed Online Optimization Over Time-Varying Directed Networks
by: Shi, Xinli, et al.
Published: (2025)

High-Performance Hybrid Algorithm for Minimum Sum-of-Squares Clustering of Infinitely Tall Data
by: Mussabayev, Ravil, et al.
Published: (2023)

A New Theoretical Perspective on Data Heterogeneity in Federated Optimization
by: Wang, Jiayi, et al.
Published: (2024)

Ordered Momentum for Asynchronous SGD
by: Shi, Chang-Wei, et al.
Published: (2024)

Large-Scale LLM Inference with Heterogeneous Workloads: Prefill-Decode Contention and Asymptotically Optimal Control
by: Lin, Ruihan, et al.
Published: (2026)