Copertina

Communication-Efficient Distributed Training for Collaborative Flat Optima Recovery in Deep Learning

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Dimlioglu, Tolga, Choromanska, Anna
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning Distributed, Parallel, and Cluster Computing
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

GRAWA: Gradient-based Weighted Averaging for Distributed Training of Deep Learning Models
di: Dimlioglu, Tolga, et al.
Pubblicazione: (2024)

A Survey of Optimization Methods for Training DL Models: Theoretical Perspective on Convergence and Generalization
di: Wang, Jing, et al.
Pubblicazione: (2025)

SparDL: Distributed Deep Learning Training with Efficient Sparse Communication
di: Zhao, Minjun, et al.
Pubblicazione: (2023)

FedOptima: Optimizing Resource Utilization in Federated Learning
di: Zhang, Zihan, et al.
Pubblicazione: (2025)

Communication-Efficient Distributed Deep Learning via Federated Dynamic Averaging
di: Theologitis, Michail, et al.
Pubblicazione: (2024)

GraNNDis: Efficient Unified Distributed Training Framework for Deep GNNs on Large Clusters
di: Song, Jaeyong, et al.
Pubblicazione: (2023)

Communication-Efficient Distributed Learning with Local Immediate Error Compensation
di: Cheng, Yifei, et al.
Pubblicazione: (2024)

Hybrid Dual-Batch and Cyclic Progressive Learning for Efficient Distributed Training
di: Lu, Kuan-Wei, et al.
Pubblicazione: (2025)

Communication Optimization for Distributed Training: Architecture, Advances, and Opportunities
di: Wei, Yunze, et al.
Pubblicazione: (2024)

An All-Reduce Compatible Top-K Compressor for Communication-Efficient Distributed Learning
di: Chen, Chuyan, et al.
Pubblicazione: (2025)

Byzantine-Robust and Communication-Efficient Distributed Learning via Compressed Momentum Filtering
di: Liu, Changxin, et al.
Pubblicazione: (2024)

Accelerating Communication in Deep Learning Recommendation Model Training with Dual-Level Adaptive Lossy Compression
di: Feng, Hao, et al.
Pubblicazione: (2024)

Communication-Efficient Federated Group Distributionally Robust Optimization
di: Guo, Zhishuai, et al.
Pubblicazione: (2024)

SignMuon: Communication-Efficient Distributed Muon Optimization
di: Mishra, Neel, et al.
Pubblicazione: (2026)

Occult: Optimizing Collaborative Communication across Experts for Accelerated Parallel MoE Training and Inference
di: Luo, Shuqing, et al.
Pubblicazione: (2025)

Efficient Parallelization Layouts for Large-Scale Distributed Model Training
di: Hagemann, Johannes, et al.
Pubblicazione: (2023)

DG-CoLearn: An Efficient Collaborative Learning Framework for Dynamic Graphs
di: Au, Ashley Hoi-Ting, et al.
Pubblicazione: (2026)

AB-Training: A Communication-Efficient Approach for Distributed Low-Rank Learning
di: Coquelin, Daniel, et al.
Pubblicazione: (2024)

LoCoDL: Communication-Efficient Distributed Learning with Local Training and Compression
di: Condat, Laurent, et al.
Pubblicazione: (2024)

Going Forward-Forward in Distributed Deep Learning
di: Aktemur, Ege, et al.
Pubblicazione: (2024)

TokenWeave: Efficient Compute-Communication Overlap for Distributed LLM Inference
di: Gond, Raja, et al.
Pubblicazione: (2025)

Armada: Memory-Efficient Distributed Training of Large-Scale Graph Neural Networks
di: Waleffe, Roger, et al.
Pubblicazione: (2025)

Update Estimation and Scheduling for Over-the-Air Federated Learning with Energy Harvesting Devices
di: Bagci, Furkan, et al.
Pubblicazione: (2025)

Distributed Deep Learning using Stochastic Gradient Staleness
di: Pham, Viet Hoang, et al.
Pubblicazione: (2025)

OpenDiLoCo: An Open-Source Framework for Globally Distributed Low-Communication Training
di: Jaghouar, Sami, et al.
Pubblicazione: (2024)

High-Dimensional Distributed Sparse Classification with Scalable Communication-Efficient Global Updates
di: Lu, Fred, et al.
Pubblicazione: (2024)

TensorSocket: Shared Data Loading for Deep Learning Training
di: Robroek, Ties, et al.
Pubblicazione: (2024)

Spindle: Efficient Distributed Training of Multi-Task Large Models via Wavefront Scheduling
di: Wang, Yujie, et al.
Pubblicazione: (2024)

Traversal Learning: A Lossless And Efficient Distributed Learning Framework
di: Batbaatar, Erdenebileg, et al.
Pubblicazione: (2025)

Agglomerative Federated Learning: Empowering Larger Model Training via End-Edge-Cloud Collaboration
di: Wu, Zhiyuan, et al.
Pubblicazione: (2023)

Federated Learning based on Pruning and Recovery
di: Ma, Chengjie
Pubblicazione: (2024)

Collaborative and Distributed Bayesian Optimization via Consensus: Showcasing the Power of Collaboration for Optimal Design
di: Yue, Xubo, et al.
Pubblicazione: (2023)

HelixPipe: Efficient Distributed Training of Long Sequence Transformers with Attention Parallel Pipeline Parallelism
di: Zhang, Geng, et al.
Pubblicazione: (2025)

Efficient Data Distribution Estimation for Accelerated Federated Learning
di: Wang, Yuanli, et al.
Pubblicazione: (2024)

Taming Unbalanced Training Workloads in Deep Learning with Partial Collective Operations
di: Li, Shigang, et al.
Pubblicazione: (2019)

Masked Random Noise for Communication Efficient Federated Learning
di: Li, Shiwei, et al.
Pubblicazione: (2024)

NEST: Network- and Memory-Aware Device Placement For Distributed Deep Learning
di: Wang, Irene, et al.
Pubblicazione: (2026)

Towards Federated Learning with On-device Training and Communication in 8-bit Floating Point
di: Wang, Bokun, et al.
Pubblicazione: (2024)

MegaScale-MoE: Large-Scale Communication-Efficient Training of Mixture-of-Experts Models in Production
di: Jin, Chao, et al.
Pubblicazione: (2025)

Maya: Optimizing Deep Learning Training Workloads using GPU Runtime Emulation
di: Yarlagadda, Srihas, et al.
Pubblicazione: (2025)