Cover Image

Preserving Near-Optimal Gradient Sparsification Cost for Scalable Distributed Deep Learning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yoon, Daegun, Oh, Sangyoon
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Distributed, Parallel, and Cluster Computing
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

MiCRO: Near-Zero Cost Gradient Sparsification for Scaling and Accelerating Distributed DNN Training
by: Yoon, Daegun, et al.
Published: (2023)

Mask-Encoded Sparsification: Mitigating Biased Gradients in Communication-Efficient Split Learning
by: Zhou, Wenxuan, et al.
Published: (2024)

Distributed Deep Learning using Stochastic Gradient Staleness
by: Pham, Viet Hoang, et al.
Published: (2025)

Low-Cost Privacy-Preserving Decentralized Learning
by: Biswas, Sayan, et al.
Published: (2024)

Resource-Aware Aggregation and Sparsification in Heterogeneous Ensemble Federated Learning
by: Ryum, Keumseo, et al.
Published: (2025)

Reducing Communication for Split Learning by Randomized Top-k Sparsification
by: Zheng, Fei, et al.
Published: (2023)

Near-Optimal Resilient Aggregation Rules for Distributed Learning Using 1-Center and 1-Mean Clustering with Outliers
by: Yi, Yuhao, et al.
Published: (2023)

Near-Optimal Privacy-Preserving Learning for Max-Min Fair Multi-Agent Bandits
by: Leshem, Amir
Published: (2023)

Traversal Learning: A Lossless And Efficient Distributed Learning Framework
by: Batbaatar, Erdenebileg, et al.
Published: (2025)

Improved Quantization Strategies for Managing Heavy-tailed Gradients in Distributed Learning
by: Yan, Guangfeng, et al.
Published: (2024)

Distributed Learning based on 1-Bit Gradient Coding in the Presence of Stragglers
by: Li, Chengxi, et al.
Published: (2024)

Going Forward-Forward in Distributed Deep Learning
by: Aktemur, Ege, et al.
Published: (2024)

Grappa: Gradient-Only Communication for Scalable Graph Neural Network Training
by: Xu, Chongyang, et al.
Published: (2026)

Efficient Asynchronous Federated Learning with Sparsification and Quantization
by: Jia, Juncheng, et al.
Published: (2023)

Scalable and Cost-Efficient ML Inference: Parallel Batch Processing with Serverless Functions
by: Barrak, Amine, et al.
Published: (2025)

ECHO-2: A Large-Scale Distributed Rollout Framework for Cost-Efficient Reinforcement Learning
by: Song, Jingwei, et al.
Published: (2026)

Energy-efficient Decentralized Learning via Graph Sparsification
by: Zhang, Xusheng, et al.
Published: (2024)

Communication-Efficient Distributed Deep Learning via Federated Dynamic Averaging
by: Theologitis, Michail, et al.
Published: (2024)

NEST: Network- and Memory-Aware Device Placement For Distributed Deep Learning
by: Wang, Irene, et al.
Published: (2026)

SparDL: Distributed Deep Learning Training with Efficient Sparse Communication
by: Zhao, Minjun, et al.
Published: (2023)

Effectiveness of Distributed Gradient Descent with Local Steps for Overparameterized Models
by: Zhu, Heng, et al.
Published: (2024)

Communication-Efficient Distributed Training for Collaborative Flat Optima Recovery in Deep Learning
by: Dimlioglu, Tolga, et al.
Published: (2025)

Prediction-Assisted Online Distributed Deep Learning Workload Scheduling in GPU Clusters
by: Luo, Ziyue, et al.
Published: (2025)

Scalable High-Dimensional Multivariate Linear Regression for Feature-Distributed Data
by: Huang, Shuo-Chieh, et al.
Published: (2023)

GRAWA: Gradient-based Weighted Averaging for Distributed Training of Deep Learning Models
by: Dimlioglu, Tolga, et al.
Published: (2024)

A Dynamic Weighting Strategy to Mitigate Worker Node Failure in Distributed Deep Learning
by: Xu, Yuesheng, et al.
Published: (2024)

Time-Series Learning for Proactive Fault Prediction in Distributed Systems with Deep Neural Structures
by: Wang, Yang, et al.
Published: (2025)

High-Dimensional Distributed Sparse Classification with Scalable Communication-Efficient Global Updates
by: Lu, Fred, et al.
Published: (2024)

Achieving Near-Optimal Convergence for Distributed Minimax Optimization with Adaptive Stepsizes
by: Huang, Yan, et al.
Published: (2024)

Optimizing the Optimal Weighted Average: Efficient Distributed Sparse Classification
by: Lu, Fred, et al.
Published: (2024)

Blockchain-based Framework for Scalable and Incentivized Federated Learning
by: Wu, Bijun, et al.
Published: (2025)

Distributed Stochastic Gradient Descent with Staleness: A Stochastic Delay Differential Equation Based Framework
by: Yu, Siyuan, et al.
Published: (2024)

Robust Decentralized Learning with Local Updates and Gradient Tracking
by: Ghiasvand, Sajjad, et al.
Published: (2024)

Buffer-based Gradient Projection for Continual Federated Learning
by: Dai, Shenghong, et al.
Published: (2024)

Distributed Saddle-Point Problems: Lower Bounds, Near-Optimal and Robust Algorithms
by: Beznosikov, Aleksandr, et al.
Published: (2020)

FedAgg: Adaptive Federated Learning with Aggregated Gradients
by: Yuan, Wenhao, et al.
Published: (2023)

Optimizing Federated Learning for Scalable Power-demand Forecasting in Microgrids
by: Banerjee, Roopkatha, et al.
Published: (2025)

First-Order Softmax Weighted Switching Gradient Method for Distributed Stochastic Minimax Optimization with Stochastic Constraints
by: Luo, Zhankun, et al.
Published: (2026)

On the Convergence of Continual Federated Learning Using Incrementally Aggregated Gradients
by: Keshri, Satish Kumar, et al.
Published: (2024)

Accelerating Federated Learning by Selecting Beneficial Herd of Local Gradients
by: Luo, Ping, et al.
Published: (2024)