Cover Image

Global Momentum Compression for Sparse Communication in Distributed Learning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Shi, Chang-Wei, Zhao, Shen-Yi, Xie, Yin-Peng, Gao, Hao, Li, Wu-Jun
Format:	Preprint
Published:	2019
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Stochastic Normalized Gradient Descent with Momentum for Large-Batch Training
by: Zhao, Shen-Yi, et al.
Published: (2020)

Ordered Local Momentum for Asynchronous Distributed Learning under Arbitrary Delays
by: Shi, Chang-Wei, et al.
Published: (2026)

Accelerating Byzantine-Robust Distributed Learning with Compressed Communication via Double Momentum and Variance Reduction
by: Li, Yanghao, et al.
Published: (2026)

Ordered Momentum for Asynchronous SGD
by: Shi, Chang-Wei, et al.
Published: (2024)

Byzantine-Robust and Communication-Efficient Distributed Learning via Compressed Momentum Filtering
by: Liu, Changxin, et al.
Published: (2024)

SparDL: Distributed Deep Learning Training with Efficient Sparse Communication
by: Zhao, Minjun, et al.
Published: (2023)

Globally Interpretable Graph Learning via Distribution Matching
by: Nian, Yi, et al.
Published: (2023)

Strategies for Improving Communication Efficiency in Distributed and Federated Learning: Compression, Local Training, and Personalization
by: Yi, Kai
Published: (2025)

Compressed Decentralized Momentum Stochastic Gradient Methods for Nonconvex Optimization
by: Liu, Wei, et al.
Published: (2025)

Double Momentum Method for Lower-Level Constrained Bilevel Optimization
by: Shi, Wanli, et al.
Published: (2024)

Dynamic Momentum Recalibration in Online Gradient Learning
by: Yao, Zhipeng, et al.
Published: (2026)

MLorc: Momentum Low-rank Compression for Memory Efficient Large Language Model Adaptation
by: Shen, Wei, et al.
Published: (2025)

Towards Communication-efficient Federated Learning via Sparse and Aligned Adaptive Optimization
by: Deng, Xiumei, et al.
Published: (2024)

Distributed Sign Momentum with Local Steps for Training Transformers
by: Yu, Shuhua, et al.
Published: (2024)

Retraining-Free Merging of Sparse MoE via Hierarchical Clustering
by: Chen, I-Chun, et al.
Published: (2024)

MLFEF: Machine Learning Fusion Model with Empirical Formula to Explore the Momentum in Competitive Sports
by: Peng, Ruixin, et al.
Published: (2024)

Nonconvex Approach for Sparse and Low-Rank Constrained Models with Dual Momentum
by: Wu, Cho-Ying, et al.
Published: (2019)

Distillation-Guided Structural Transfer for Continual Learning Beyond Sparse Distributed Memory
by: Xue, Huiyan, et al.
Published: (2025)

Distributed Online Convex Optimization with Compressed Communication: Optimal Regret and Applications
by: Yang, Sifan, et al.
Published: (2026)

High-Dimensional Distributed Sparse Classification with Scalable Communication-Efficient Global Updates
by: Lu, Fred, et al.
Published: (2024)

Semi-Supervised Multi-Label Feature Selection with Consistent Sparse Graph Learning
by: Zhong, Yan, et al.
Published: (2025)

ExCP: Extreme LLM Checkpoint Compression via Weight-Momentum Joint Shrinking
by: Li, Wenshuo, et al.
Published: (2024)

Attentional Graph Meta-Learning for Indoor Localization Using Extremely Sparse Fingerprints
by: Yan, Wenzhong, et al.
Published: (2025)

Distributed Low-Communication Training with Decoupled Momentum Optimization
by: Nedelkoski, Sasho, et al.
Published: (2025)

Dynamics of Stochastic Momentum with Sparse Updates in High Dimensions
by: Everett, Katie, et al.
Published: (2026)

Distillation Enhanced Time Series Forecasting Network with Momentum Contrastive Learning
by: Gao, Haozhi, et al.
Published: (2024)

SNOO: Step-K Nesterov Outer Optimizer - The Surprising Effectiveness of Nesterov Momentum Applied to Pseudo-Gradients
by: Kallusky, Dominik, et al.
Published: (2025)

Momentum-integrated Multi-task Stock Recommendation with Converge-based Optimization
by: Wang, Hao, et al.
Published: (2025)

FedLWS: Federated Learning with Adaptive Layer-wise Weight Shrinking
by: Shi, Changlong, et al.
Published: (2025)

Byzantine-Robust Distributed Sparse Learning Revisited
by: Wang, Yuxuan, et al.
Published: (2026)

Momentum-Based Federated Reinforcement Learning with Interaction and Communication Efficiency
by: Yue, Sheng, et al.
Published: (2024)

Provable Sparse Inversion and Token Relabel Enhanced One-shot Federated Learning with ViTs
by: Shen, Li, et al.
Published: (2026)

Reconciling Communication Compression and Byzantine-Robustness in Distributed Learning
by: Gupta, Diksha, et al.
Published: (2025)

On the Performance Analysis of Momentum Method: A Frequency Domain Perspective
by: Li, Xianliang, et al.
Published: (2024)

Momentum Capture and Prediction System Based on Wimbledon Open2023 Tournament Data
by: Liu, Chang, et al.
Published: (2024)

Beyond Communication Overhead: A Multilevel Monte Carlo Approach for Mitigating Compression Bias in Distributed Learning
by: Zukerman, Ze'ev, et al.
Published: (2025)

GraVAC: Adaptive Compression for Communication-Efficient Distributed DL Training
by: Tyagi, Sahil, et al.
Published: (2023)

FedVCK: Non-IID Robust and Communication-Efficient Federated Learning via Valuable Condensed Knowledge for Medical Image Analysis
by: Yan, Guochen, et al.
Published: (2024)

Heterogeneous Federated Learning System for Sparse Healthcare Time-Series Prediction
by: Syu, Jia-Hao, et al.
Published: (2025)

Global and Local Structure Learning for Sparse Tensor Completion
by: Ahn, Dawon, et al.
Published: (2025)