Copertina

Improved Quantization Strategies for Managing Heavy-tailed Gradients in Distributed Learning

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Yan, Guangfeng, Li, Tan, Xiao, Yuanzhang, Hou, Hanxu, Song, Linqi
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Machine Learning Distributed, Parallel, and Cluster Computing
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Truncated Non-Uniform Quantization for Distributed SGD
di: Yan, Guangfeng, et al.
Pubblicazione: (2024)

Distributed Learning based on 1-Bit Gradient Coding in the Presence of Stragglers
di: Li, Chengxi, et al.
Pubblicazione: (2024)

FedUHB: Accelerating Federated Unlearning via Polyak Heavy Ball Method
di: Jiang, Yu, et al.
Pubblicazione: (2024)

Distributed Deep Learning using Stochastic Gradient Staleness
di: Pham, Viet Hoang, et al.
Pubblicazione: (2025)

Preserving Near-Optimal Gradient Sparsification Cost for Scalable Distributed Deep Learning
di: Yoon, Daegun, et al.
Pubblicazione: (2024)

QSync: Quantization-Minimized Synchronous Distributed Training Across Hybrid Devices
di: Zhao, Juntao, et al.
Pubblicazione: (2024)

Efficient Federated Learning against Byzantine Attacks and Data Heterogeneity via Aggregating Normalized Gradients
di: Zuo, Shiyuan, et al.
Pubblicazione: (2024)

FedGCS: A Generative Framework for Efficient Client Selection in Federated Learning via Gradient-based Optimization
di: Ning, Zhiyuan, et al.
Pubblicazione: (2024)

Effectiveness of Distributed Gradient Descent with Local Steps for Overparameterized Models
di: Zhu, Heng, et al.
Pubblicazione: (2024)

A Dynamic Weighting Strategy to Mitigate Worker Node Failure in Distributed Deep Learning
di: Xu, Yuesheng, et al.
Pubblicazione: (2024)

MoE-SpeQ: Speculative Quantized Decoding with Proactive Expert Prefetching and Offloading for Mixture-of-Experts
di: Wang, Wenfeng, et al.
Pubblicazione: (2025)

A Joint Approach to Local Updating and Gradient Compression for Efficient Asynchronous Federated Learning
di: Song, Jiajun, et al.
Pubblicazione: (2024)

Accelerating Federated Learning by Selecting Beneficial Herd of Local Gradients
di: Luo, Ping, et al.
Pubblicazione: (2024)

Improving the Efficiency of a Deep Reinforcement Learning-Based Power Management System for HPC Clusters Using Curriculum Learning
di: Budiarjo, Thomas, et al.
Pubblicazione: (2025)

Virtual Nodes Can Help: Tackling Distribution Shifts in Federated Graph Learning
di: Fu, Xingbo, et al.
Pubblicazione: (2024)

Reinforcement Learning for Adaptive Resource Scheduling in Complex System Environments
di: Li, Pochun, et al.
Pubblicazione: (2024)

Trustworthiness of Stochastic Gradient Descent in Distributed Learning
di: Li, Hongyang, et al.
Pubblicazione: (2024)

Flexible Clustered Federated Learning for Client-Level Data Distribution Shift
di: Duan, Moming, et al.
Pubblicazione: (2021)

Local Gradient Regulation Stabilizes Federated Learning under Client Heterogeneity
di: Luo, Ping, et al.
Pubblicazione: (2026)

Energy-Efficient Quantized Federated Learning for Resource-constrained IoT devices
di: Compaoré, Wilfrid Sougrinoma, et al.
Pubblicazione: (2025)

FedQS: Optimizing Gradient and Model Aggregation for Semi-Asynchronous Federated Learning
di: Li, Yunbo, et al.
Pubblicazione: (2025)

FedReMa: Improving Personalized Federated Learning via Leveraging the Most Relevant Clients
di: Liang, Han, et al.
Pubblicazione: (2024)

Online Distributed Learning with Quantized Finite-Time Coordination
di: Bastianello, Nicola, et al.
Pubblicazione: (2023)

MiCRO: Near-Zero Cost Gradient Sparsification for Scaling and Accelerating Distributed DNN Training
di: Yoon, Daegun, et al.
Pubblicazione: (2023)

Distributed Stochastic Gradient Descent with Staleness: A Stochastic Delay Differential Equation Based Framework
di: Yu, Siyuan, et al.
Pubblicazione: (2024)

Robust Decentralized Learning with Local Updates and Gradient Tracking
di: Ghiasvand, Sajjad, et al.
Pubblicazione: (2024)

FedAgg: Adaptive Federated Learning with Aggregated Gradients
di: Yuan, Wenhao, et al.
Pubblicazione: (2023)

Buffer-based Gradient Projection for Continual Federated Learning
di: Dai, Shenghong, et al.
Pubblicazione: (2024)

An Experimental Comparison of Partitioning Strategies for Distributed Graph Neural Network Training
di: Merkel, Nikolai, et al.
Pubblicazione: (2023)

Enhancing Convergence, Privacy and Fairness for Wireless Personalized Federated Learning: Quantization-Assisted Min-Max Fair Scheduling
di: Zhao, Xiyu, et al.
Pubblicazione: (2025)

Communication-Efficient Federated Learning by Quantized Variance Reduction for Heterogeneous Wireless Edge Networks
di: Wang, Shuai, et al.
Pubblicazione: (2025)

First-Order Softmax Weighted Switching Gradient Method for Distributed Stochastic Minimax Optimization with Stochastic Constraints
di: Luo, Zhankun, et al.
Pubblicazione: (2026)

On the Convergence of Continual Federated Learning Using Incrementally Aggregated Gradients
di: Keshri, Satish Kumar, et al.
Pubblicazione: (2024)

The Panaceas for Improving Low-Rank Decomposition in Communication-Efficient Federated Learning
di: Li, Shiwei, et al.
Pubblicazione: (2025)

ECHO-2: A Large-Scale Distributed Rollout Framework for Cost-Efficient Reinforcement Learning
di: Song, Jingwei, et al.
Pubblicazione: (2026)

FedCGD: Collective Gradient Divergence Optimized Scheduling for Wireless Federated Learning
di: Chen, Tan, et al.
Pubblicazione: (2025)

Emulating Full Participation: An Effective and Fair Client Selection Strategy for Federated Learning
di: Li, Qingming, et al.
Pubblicazione: (2024)

An Efficient Gradient-Aware Error-Bounded Lossy Compressor for Federated Learning
di: Ye, Zhijing, et al.
Pubblicazione: (2025)

Age-of-Gradient Updates for Federated Learning over Random Access Channels
di: Wu, Yu Heng, et al.
Pubblicazione: (2024)

The Diversity Bonus: Learning from Dissimilar Distributed Clients in Personalized Federated Learning
di: Wu, Xinghao, et al.
Pubblicazione: (2024)