Cover Image

Communication-Efficient Distributed Learning via Sparse and Adaptive Stochastic Gradient

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Deng, Xiaoge, Li, Dongsheng, Sun, Tao, Lu, Xicheng
Format:	Preprint
Published:	2021
Subjects:	Distributed, Parallel, and Cluster Computing
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Accelerating Federated Learning by Selecting Beneficial Herd of Local Gradients
by: Luo, Ping, et al.
Published: (2024)

Oases: Efficient Large-Scale Model Training on Commodity Servers via Overlapped and Automated Tensor Model Parallelism
by: Li, Shengwei, et al.
Published: (2023)

PacTrain: Pruning and Adaptive Sparse Gradient Compression for Efficient Collective Communication in Distributed Deep Learning
by: Wang, Yisu, et al.
Published: (2025)

Toward Efficient SpMV in Sparse LLMs via Block Extraction and Compressed Storage
by: Lin, Junqing, et al.
Published: (2025)

SparDL: Distributed Deep Learning Training with Efficient Sparse Communication
by: Zhao, Minjun, et al.
Published: (2023)

Local Gradient Regulation Stabilizes Federated Learning under Client Heterogeneity
by: Luo, Ping, et al.
Published: (2026)

Towards Communication-efficient Federated Learning via Sparse and Aligned Adaptive Optimization
by: Deng, Xiumei, et al.
Published: (2024)

Energy-Efficient Wireless Federated Learning via Doubly Adaptive Quantization
by: Han, Xuefeng, et al.
Published: (2024)

Communication-Efficient Sparsely-Activated Model Training via Sequence Migration and Token Condensation
by: Chen, Fahao, et al.
Published: (2024)

Biased Compression in Gradient Coding for Distributed Learning
by: Li, Chengxi, et al.
Published: (2026)

Hecate: Unlocking Efficient Sparse Model Training via Fully Sharded Sparse Data Parallelism
by: Qing, Yuhao, et al.
Published: (2025)

NetSenseML: Network-Adaptive Compression for Efficient Distributed Machine Learning
by: Wang, Yisu, et al.
Published: (2025)

Communication-and-Computation Efficient Split Federated Learning: Gradient Aggregation and Resource Management
by: Liang, Yipeng, et al.
Published: (2025)

GreenDyGNN: Runtime-Adaptive Energy-Efficient Communication for Distributed GNN Training
by: Niam, Arefin, et al.
Published: (2026)

FedCod: An Efficient Communication Protocol for Cross-Silo Federated Learning with Coding
by: Yan, Peishen, et al.
Published: (2024)

ParamSpMM: Adaptive and Efficient Sparse Matrix-Matrix Multiplication on GPUs for GNNs
by: Zhang, Lixing, et al.
Published: (2026)

AB-Sparse: Sparse Attention with Adaptive Block Size for Accurate and Efficient Long-Context Inference
by: Liu, Di, et al.
Published: (2026)

CondenseGraph: Communication-Efficient Distributed GNN Training via On-the-Fly Graph Condensation
by: Zhang, Zizhao, et al.
Published: (2026)

High-Dimensional Distributed Sparse Classification with Scalable Communication-Efficient Global Updates
by: Lu, Fred, et al.
Published: (2024)

Distributed Deep Learning using Stochastic Gradient Staleness
by: Pham, Viet Hoang, et al.
Published: (2025)

ACE-Sync: An Adaptive Cloud-Edge Synchronization Framework for Communication-Efficient Large-Scale Distributed Model Training
by: Yang, Yi, et al.
Published: (2025)

Efficient Multi-Worker Selection based Distributed Swarm Learning via Analog Aggregation
by: Yao, Zhuoyu, et al.
Published: (2025)

Byzantine-Robust and Communication-Efficient Distributed Training: Compressive and Cyclic Gradient Coding
by: Li, Chengxi, et al.
Published: (2026)

SPPO:Efficient Long-sequence LLM Training via Adaptive Sequence Pipeline Parallel Offloading
by: Chen, Qiaoling, et al.
Published: (2025)

ADP-VRSGP: Decentralized Learning with Adaptive Differential Privacy via Variance-Reduced Stochastic Gradient Push
by: Wu, Xiaoming, et al.
Published: (2025)

Communication-Efficient Distributed Learning with Local Immediate Error Compensation
by: Cheng, Yifei, et al.
Published: (2024)

Clock Distribution with Gradient TRIX
by: Lenzen, Christoph, et al.
Published: (2023)

Heterogeneity-Aware Memory Efficient Federated Learning via Progressive Layer Freezing
by: Yebo, Wu, et al.
Published: (2024)

CO2: Efficient Distributed Training with Full Communication-Computation Overlap
by: Sun, Weigao, et al.
Published: (2024)

Byzantine-Robust and Communication-Efficient Distributed Learning via Compressed Momentum Filtering
by: Liu, Changxin, et al.
Published: (2024)

Efficient Distributed MLLM Training with Cornstarch
by: Jang, Insu, et al.
Published: (2025)

Chameleon: Adaptive Fault Tolerance for Distributed Training via Real-time Policy Selection
by: Zhou, Yuhang, et al.
Published: (2025)

FUSCO: High-Performance Distributed Data Shuffling via Transformation-Communication Fusion
by: Zhu, Zhuoran, et al.
Published: (2025)

Trustworthiness of Stochastic Gradient Descent in Distributed Learning
by: Li, Hongyang, et al.
Published: (2024)

Distributed-Memory Parallel Algorithms for Sparse Matrix and Sparse Tall-and-Skinny Matrix Multiplication
by: Ranawaka, Isuru, et al.
Published: (2024)

SHIRO: Near-Optimal Communication Strategies for Distributed Sparse Matrix Multiplication
by: Zhuang, Chen, et al.
Published: (2025)

cuFastTuckerPlus: A Stochastic Parallel Sparse FastTucker Decomposition Using GPU Tensor Cores
by: Li, Zixuan, et al.
Published: (2024)

High-Dimensional Sparse Data Low-rank Representation via Accelerated Asynchronous Parallel Stochastic Gradient Descent
by: Hu, Qicong, et al.
Published: (2024)

Optimizing the Optimal Weighted Average: Efficient Distributed Sparse Classification
by: Lu, Fred, et al.
Published: (2024)

Selection of Supervised Learning-based Sparse Matrix Reordering Algorithms
by: Tang, Tao, et al.
Published: (2025)