Cover Image

From promise to practice: realizing high-performance decentralized training

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wang, Zesen, Zhang, Jiaojiao, Wu, Xuyang, Johansson, Mikael
Format:	Preprint
Published:	2024
Subjects:	Distributed, Parallel, and Cluster Computing Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Controlled disagreement improves generalization in decentralized training
by: Wang, Zesen, et al.
Published: (2026)

Non-convex composite federated learning with heterogeneous data
by: Zhang, Jiaojiao, et al.
Published: (2025)

Locally Differentially Private Online Federated Learning With Correlated Noise
by: Zhang, Jiaojiao, et al.
Published: (2024)

Differentially Private Online Federated Learning with Correlated Noise
by: Zhang, Jiaojiao, et al.
Published: (2024)

Nonconvex Federated Learning on Compact Smooth Submanifolds With Heterogeneous Data
by: Zhang, Jiaojiao, et al.
Published: (2024)

Semi-decentralized Federated Time Series Prediction with Client Availability Budgets
by: Bao, Yunkai, et al.
Published: (2025)

An Efficient Subspace Algorithm for Federated Learning on Heterogeneous Data
by: Zhang, Jiaojiao, et al.
Published: (2025)

LoDAdaC: a unified local training-based decentralized framework with adaptive gradients and compressed communication
by: Liu, Wei, et al.
Published: (2026)

Distributed Learning based on 1-Bit Gradient Coding in the Presence of Stragglers
by: Li, Chengxi, et al.
Published: (2024)

FLMarket: Enabling Privacy-preserved Pre-training Data Pricing for Federated Learning
by: Wen, Zhenyu, et al.
Published: (2024)

GPT-FL: Generative Pre-trained Model-Assisted Federated Learning
by: Zhang, Tuo, et al.
Published: (2023)

Byzantine-Robust and Communication-Efficient Distributed Learning via Compressed Momentum Filtering
by: Liu, Changxin, et al.
Published: (2024)

Vertical Federated Learning Hybrid Local Pre-training
by: Li, Wenguo, et al.
Published: (2024)

Semi-decentralized Training of Spatio-Temporal Graph Neural Networks for Traffic Prediction
by: Kralj, Ivan, et al.
Published: (2024)

Computing in the Era of Large Generative Models: From Cloud-Native to AI-Native
by: Lu, Yao, et al.
Published: (2024)

Carbon-aware decentralized dynamic task offloading in MIMO-MEC networks via multi-agent reinforcement learning
by: Zulfiqar, Mubshra, et al.
Published: (2026)

Temporal Predictive Coding for Gradient Compression in Distributed Learning
by: Edin, Adrian, et al.
Published: (2024)

Exploiting Correlations in Federated Learning: Opportunities and Practical Limitations
by: Edin, Adrian, et al.
Published: (2026)

Optimizing Distributed Deployment of Mixture-of-Experts Model Inference in Serverless Computing
by: Liu, Mengfan, et al.
Published: (2025)

Large Language Model Aided QoS Prediction for Service Recommendation
by: Liu, Huiying, et al.
Published: (2024)

FedDD: Toward Communication-efficient Federated Learning with Differential Parameter Dropout
by: Feng, Zhiying, et al.
Published: (2023)

FedNS: A Fast Sketching Newton-Type Algorithm for Federated Learning
by: Li, Jian, et al.
Published: (2024)

Time-Series Learning for Proactive Fault Prediction in Distributed Systems with Deep Neural Structures
by: Wang, Yang, et al.
Published: (2025)

From Models to Operators: Rethinking Autoscaling Granularity for Large Generative Models
by: Cui, Xingqi, et al.
Published: (2025)

Emerging Trends in Federated Learning: From Model Fusion to Federated X Learning
by: Ji, Shaoxiong, et al.
Published: (2021)

FIKIT: Priority-Based Real-time GPU Multi-tasking Scheduling with Kernel Identification
by: Wu, Wenqing
Published: (2023)

When Swarm Learning meets energy series data: A decentralized collaborative learning design based on blockchain
by: Xu, Lei, et al.
Published: (2024)

Designing Large Foundation Models for Efficient Training and Inference: A Survey
by: Liu, Dong, et al.
Published: (2024)

ADP-VRSGP: Decentralized Learning with Adaptive Differential Privacy via Variance-Reduced Stochastic Gradient Push
by: Wu, Xiaoming, et al.
Published: (2025)

Neighborhood and Global Perturbations Supported SAM in Federated Learning: From Local Tweaks To Global Awareness
by: Li, Boyuan, et al.
Published: (2024)

From Tokens to Layers: Redefining Stall-Free Scheduling for MoE Serving with Layered Prefill
by: Lee, Gunjun, et al.
Published: (2025)

TDC-Cache: A Trustworthy Decentralized Cooperative Caching Framework for Web3.0
by: Chen, Jinyu, et al.
Published: (2025)

DualFed: Enjoying both Generalization and Personalization in Federated Learning via Hierachical Representations
by: Zhu, Guogang, et al.
Published: (2024)

AReaL-Hex: Accommodating Asynchronous RL Training over Heterogeneous GPUs
by: Yan, Ran, et al.
Published: (2025)

Rethinking Personalized Federated Learning with Clustering-based Dynamic Graph Propagation
by: Wang, Jiaqi, et al.
Published: (2024)

Lancet: Accelerating Mixture-of-Experts Training via Whole Graph Computation-Communication Overlapping
by: Jiang, Chenyu, et al.
Published: (2024)

DCP: Addressing Input Dynamism In Long-Context Training via Dynamic Context Parallelism
by: Jiang, Chenyu, et al.
Published: (2025)

SpecRouter: Adaptive Routing for Multi-Level Speculative Decoding in Large Language Models
by: Wu, Hang, et al.
Published: (2025)

A Fast Task Offloading Optimization Framework for IRS-Assisted Multi-Access Edge Computing System
by: Wu, Jianqiu, et al.
Published: (2023)

Echo: Simulating Distributed Training At Scale
by: Feng, Yicheng, et al.
Published: (2024)