Cover Image

Towards a Better Theoretical Understanding of Independent Subnetwork Training

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Shulgin, Egor, Richtárik, Peter
Format:	Preprint
Published:	2023
Subjects:	Machine Learning Distributed, Parallel, and Cluster Computing Optimization and Control
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

MAST: Model-Agnostic Sparsified Training
by: Demidovich, Yury, et al.
Published: (2023)

Smoothed Normalization for Efficient Distributed Private Optimization
by: Shulgin, Egor, et al.
Published: (2025)

GradSkip: Communication-Accelerated Local Gradient Methods with Better Computational Complexity
by: Maranjyan, Artavazd, et al.
Published: (2022)

LoCoDL: Communication-Efficient Distributed Learning with Local Training and Compression
by: Condat, Laurent, et al.
Published: (2024)

Ringleader ASGD: The First Asynchronous SGD with Optimal Time Complexity under Data Heterogeneity
by: Maranjyan, Artavazd, et al.
Published: (2025)

Ringmaster ASGD: The First Asynchronous SGD with Optimal Time Complexity
by: Maranjyan, Artavazd, et al.
Published: (2025)

Rescaled Asynchronous SGD: Optimal Distributed Optimization under Data and System Heterogeneity
by: Mahran, Ammar, et al.
Published: (2026)

Rennala MVR: Improved Time Complexity for Parallel Stochastic Optimization via Momentum-Based Variance Reduction
by: Tovmasyan, Zhirayr, et al.
Published: (2026)

MindFlayer SGD: Efficient Parallel SGD in the Presence of Heterogeneous and Random Worker Compute Times
by: Maranjyan, Artavazd, et al.
Published: (2024)

Ringmaster LMO: Asynchronous Linear Minimization Oracle Momentum Method
by: Sadiev, Abdurakhmon, et al.
Published: (2026)

LOSCAR-SGD: Local SGD with Communication-Computation Overlap and Delay-Corrected Sparse Model Averaging
by: Maziane, Yassine, et al.
Published: (2026)

On Biased Compression for Distributed Learning
by: Beznosikov, Aleksandr, et al.
Published: (2020)

ATA: Adaptive Task Allocation for Efficient Resource Management in Distributed Machine Learning
by: Maranjyan, Artavazd, et al.
Published: (2025)

A Survey of Optimization Methods for Training DL Models: Theoretical Perspective on Convergence and Generalization
by: Wang, Jing, et al.
Published: (2025)

Byzantine Robustness and Partial Participation Can Be Achieved at Once: Just Clip Gradient Differences
by: Malinovsky, Grigory, et al.
Published: (2023)

Correlated Quantization for Faster Nonconvex Distributed Optimization
by: Panferov, Andrei, et al.
Published: (2024)

Activations and Gradients Compression for Model-Parallel Training
by: Rudakov, Mikhail, et al.
Published: (2024)

A Privacy Preserving Randomized Gossip Algorithm via Controlled Noise Insertion
by: Hanzely, Filip, et al.
Published: (2019)

GRAWA: Gradient-based Weighted Averaging for Distributed Training of Deep Learning Models
by: Dimlioglu, Tolga, et al.
Published: (2024)

Towards Dynamic Resource Allocation and Client Scheduling in Hierarchical Federated Learning: A Two-Phase Deep Reinforcement Learning Approach
by: Chen, Xiaojing, et al.
Published: (2024)

Proving the Limited Scalability of Centralized Distributed Optimization via a New Lower Bound Construction
by: Tyurin, Alexander
Published: (2025)

LoDAdaC: a unified local training-based decentralized framework with adaptive gradients and compressed communication
by: Liu, Wei, et al.
Published: (2026)

Stochastic Controlled Averaging for Federated Learning with Communication Compression
by: Huang, Xinmeng, et al.
Published: (2023)

High-Performance Hybrid Algorithm for Minimum Sum-of-Squares Clustering of Infinitely Tall Data
by: Mussabayev, Ravil, et al.
Published: (2023)

Convergence of Sign-based Random Reshuffling Algorithms for Nonconvex Optimization
by: Qin, Zhen, et al.
Published: (2023)

Communication-Efficient Federated Bilevel Optimization with Local and Global Lower Level Problems
by: Li, Junyi, et al.
Published: (2023)

Dynamic Regularized Sharpness Aware Minimization in Federated Learning: Approaching Global Consistency and Smooth Landscape
by: Sun, Yan, et al.
Published: (2023)

Online Distributed Learning with Quantized Finite-Time Coordination
by: Bastianello, Nicola, et al.
Published: (2023)

AGD: an Auto-switchable Optimizer using Stepwise Gradient Difference for Preconditioning Matrix
by: Yue, Yun, et al.
Published: (2023)

Lower Bounds and Accelerated Algorithms in Distributed Stochastic Optimization with Communication Compression
by: He, Yutong, et al.
Published: (2023)

Unbiased Compression Saves Communication in Distributed Optimization: When and How Much?
by: He, Yutong, et al.
Published: (2023)

Communication-Efficient Federated Optimization over Semi-Decentralized Networks
by: Wang, He, et al.
Published: (2023)

A Single-Loop Algorithm for Decentralized Bilevel Optimization
by: Dong, Youran, et al.
Published: (2023)

UniAP: Unifying Inter- and Intra-Layer Automatic Parallelism by Mixed Integer Quadratic Programming
by: Lin, Hao, et al.
Published: (2023)

Adaptive Federated Learning with Auto-Tuned Clients
by: Kim, Junhyung Lyle, et al.
Published: (2023)

FedCanon: Non-Convex Composite Federated Learning with Efficient Proximal Operation on Heterogeneous Data
by: Zhou, Yuan, et al.
Published: (2025)

On Principled Local Optimization Methods for Federated Learning
by: Yuan, Honglin
Published: (2024)

Decentralized Directed Collaboration for Personalized Federated Learning
by: Liu, Yingqi, et al.
Published: (2024)

Time-varying Mixing Matrix Design for Energy-efficient Decentralized Federated Learning
by: Zhang, Xusheng, et al.
Published: (2025)

Continuous-Time Analysis of Federated Averaging
by: Overman, Tom, et al.
Published: (2025)