Cover Image

Demystifying Why Local Aggregation Helps: Convergence Analysis of Hierarchical SGD

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wang, Jiayi, Wang, Shiqiang, Chen, Rong-Rong, Ji, Mingyue
Format:	Preprint
Published:	2020
Subjects:	Machine Learning Distributed, Parallel, and Cluster Computing Information Theory Optimization and Control
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

A New Theoretical Perspective on Data Heterogeneity in Federated Optimization
by: Wang, Jiayi, et al.
Published: (2024)

A Unified Analysis of Federated Learning with Arbitrary Client Participation
by: Wang, Shiqiang, et al.
Published: (2022)

A Lightweight Method for Tackling Unknown Participation Statistics in Federated Averaging
by: Wang, Shiqiang, et al.
Published: (2023)

Uncoded Storage Coded Transmission Elastic Computing with Straggler Tolerance in Heterogeneous Systems
by: Zhong, Xi, et al.
Published: (2024)

Revisiting LocalSGD and SCAFFOLD: Improved Rates and Missing Analysis
by: Luo, Ruichen, et al.
Published: (2025)

Birch SGD: A Tree Graph Framework for Local and Asynchronous SGD Methods
by: Tyurin, Alexander, et al.
Published: (2025)

LOSCAR-SGD: Local SGD with Communication-Computation Overlap and Delay-Corrected Sparse Model Averaging
by: Maziane, Yassine, et al.
Published: (2026)

The Limits and Potentials of Local SGD for Distributed Heterogeneous Learning with Intermittent Communication
by: Patel, Kumar Kshitij, et al.
Published: (2024)

MindFlayer SGD: Efficient Parallel SGD in the Presence of Heterogeneous and Random Worker Compute Times
by: Maranjyan, Artavazd, et al.
Published: (2024)

Ringmaster ASGD: The First Asynchronous SGD with Optimal Time Complexity
by: Maranjyan, Artavazd, et al.
Published: (2025)

First Provably Optimal Asynchronous SGD for Homogeneous and Heterogeneous Data
by: Maranjyan, Artavazd
Published: (2026)

Fundamental Limits of Coded Polynomial Aggregation
by: Zhong, Xi, et al.
Published: (2026)

Rescaled Asynchronous SGD: Optimal Distributed Optimization under Data and System Heterogeneity
by: Mahran, Ammar, et al.
Published: (2026)

Ringleader ASGD: The First Asynchronous SGD with Optimal Time Complexity under Data Heterogeneity
by: Maranjyan, Artavazd, et al.
Published: (2025)

FADAS: Towards Federated Adaptive Asynchronous Optimization
by: Wang, Yujia, et al.
Published: (2024)

A Survey of Optimization Methods for Training DL Models: Theoretical Perspective on Convergence and Generalization
by: Wang, Jing, et al.
Published: (2025)

Towards Dynamic Resource Allocation and Client Scheduling in Hierarchical Federated Learning: A Two-Phase Deep Reinforcement Learning Approach
by: Chen, Xiaojing, et al.
Published: (2024)

Hierarchical Federated Learning with SignSGD: A Highly Communication-Efficient Approach
by: Kazemi, Amirreza, et al.
Published: (2026)

Asynchronous Policy Gradient Aggregation for Efficient Distributed Reinforcement Learning
by: Tyurin, Alexander, et al.
Published: (2025)

Convergence of Sign-based Random Reshuffling Algorithms for Nonconvex Optimization
by: Qin, Zhen, et al.
Published: (2023)

Achieving Near-Optimal Convergence for Distributed Minimax Optimization with Adaptive Stepsizes
by: Huang, Yan, et al.
Published: (2024)

Vertical Federated Learning with Missing Features During Training and Inference
by: Valdeira, Pedro, et al.
Published: (2024)

Local Methods with Adaptivity via Scaling
by: Chezhegov, Savelii, et al.
Published: (2024)

On Principled Local Optimization Methods for Federated Learning
by: Yuan, Honglin
Published: (2024)

Decentralized Nonconvex Optimization under Heavy-Tailed Noise: Normalization and Optimal Convergence
by: Yu, Shuhua, et al.
Published: (2025)

Accelerating Distributed Optimization: A Primal-Dual Perspective on Local Steps
by: Yang, Junchi, et al.
Published: (2024)

GradSkip: Communication-Accelerated Local Gradient Methods with Better Computational Complexity
by: Maranjyan, Artavazd, et al.
Published: (2022)

Communication-Efficient Federated Bilevel Optimization with Local and Global Lower Level Problems
by: Li, Junyi, et al.
Published: (2023)

LoCoDL: Communication-Efficient Distributed Learning with Local Training and Compression
by: Condat, Laurent, et al.
Published: (2024)

Resource Allocation for Stable LLM Training in Mobile Edge Computing
by: Liu, Chang, et al.
Published: (2024)

Continuous-Time Analysis of Federated Averaging
by: Overman, Tom, et al.
Published: (2025)

Communication-Efficient Federated Optimization over Semi-Decentralized Networks
by: Wang, He, et al.
Published: (2023)

FIARSE: Model-Heterogeneous Federated Learning via Importance-Aware Submodel Extraction
by: Wu, Feijie, et al.
Published: (2024)

A Double Tracking Method for Optimization with Decentralized Generalized Orthogonality Constraints
by: Wang, Lei, et al.
Published: (2024)

Provable Model-Parallel Distributed Principal Component Analysis with Parallel Deflation
by: Liao, Fangshuo, et al.
Published: (2025)

LoDAdaC: a unified local training-based decentralized framework with adaptive gradients and compressed communication
by: Liu, Wei, et al.
Published: (2026)

Distributed Stochastic Momentum Tracking with Local Updates: Achieving Optimal Communication and Iteration Complexities
by: Huang, Kun, et al.
Published: (2025)

S$^3$LDBO: A Snapshot Single-Loop Algorithm for Decentralized Bilevel Optimization
by: Yin, Chao, et al.
Published: (2026)

Decentralized Directed Collaboration for Personalized Federated Learning
by: Liu, Yingqi, et al.
Published: (2024)

Tailoring Gradient Methods for Differentially-Private Distributed Optimization
by: Wang, Yongqiang, et al.
Published: (2022)