Image de couverture de livre

PacTrain: Pruning and Adaptive Sparse Gradient Compression for Efficient Collective Communication in Distributed Deep Learning

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Wang, Yisu, Wu, Ruilong, Li, Xinjiao, Kutscher, Dirk
Format:	Preprint
Publié:	2025
Sujets:	Distributed, Parallel, and Cluster Computing Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

NetSenseML: Network-Adaptive Compression for Efficient Distributed Machine Learning
par: Wang, Yisu, et autres
Publié: (2025)

Rethinking Dynamic Networks and Heterogeneous Computing with Automatic Parallelization
par: Wu, Ruilong, et autres
Publié: (2025)

Byzantine-Robust and Communication-Efficient Distributed Training: Compressive and Cyclic Gradient Coding
par: Li, Chengxi, et autres
Publié: (2026)

Communication-Efficient Distributed Learning via Sparse and Adaptive Stochastic Gradient
par: Deng, Xiaoge, et autres
Publié: (2021)

Communication-Efficient Large-Scale Distributed Deep Learning: A Comprehensive Survey
par: Liang, Feng, et autres
Publié: (2024)

TACO: Efficient Communication Compression of Intermediate Tensors for Scalable Tensor-Parallel LLM Training
par: Liu, Man, et autres
Publié: (2026)

FedComLoc: Communication-Efficient Distributed Training of Sparse and Quantized Models
par: Yi, Kai, et autres
Publié: (2024)

Enhancing Communication Efficiency in FL with Adaptive Gradient Quantization and Communication Frequency Optimization
par: Tariq, Asadullah, et autres
Publié: (2025)

Adaptive Consensus Gradients Aggregation for Scaled Distributed Training
par: Choukroun, Yoni, et autres
Publié: (2024)

Communication-Efficient Split Learning via Adaptive Feature-Wise Compression
par: Oh, Yongjeong, et autres
Publié: (2023)

AB-Training: A Communication-Efficient Approach for Distributed Low-Rank Learning
par: Coquelin, Daniel, et autres
Publié: (2024)

MoESys: A Distributed and Efficient Mixture-of-Experts Training and Inference System for Internet Services
par: Yu, Dianhai, et autres
Publié: (2022)

UCCL-Zip: Lossless Compression Supercharged GPU Communication
par: Ma, Shuang, et autres
Publié: (2026)

PruneX: A Hierarchical Communication-Efficient System for Distributed CNN Training with Structured Pruning
par: Olama, Alireza, et autres
Publié: (2025)

SparDL: Distributed Deep Learning Training with Efficient Sparse Communication
par: Zhao, Minjun, et autres
Publié: (2023)

Communication Efficient Distributed Training with Distributed Lion
par: Liu, Bo, et autres
Publié: (2024)

Efficient Federated Learning Using Dynamic Update and Adaptive Pruning with Momentum on Shared Server Data
par: Liu, Ji, et autres
Publié: (2024)

Mesh-Attention: A New Communication-Efficient Distributed Attention with Improved Data Locality
par: Chen, Sirui, et autres
Publié: (2025)

EDiT: A Local-SGD-Based Efficient Distributed Training Method for Large Language Models
par: Cheng, Jialiang, et autres
Publié: (2024)

Mist: Efficient Distributed Training of Large Language Models via Memory-Parallelism Co-Optimization
par: Zhu, Zhanda, et autres
Publié: (2025)

Trustworthiness of Stochastic Gradient Descent in Distributed Learning
par: Li, Hongyang, et autres
Publié: (2024)

Accelerating Large Language Model Training with Hybrid GPU-based Compression
par: Xu, Lang, et autres
Publié: (2024)

Deep Reinforcement Learning for Fault-Adaptive Routing in Eisenstein-Jacobi Interconnection Topologies
par: Charrwi, Mohammad Walid, et autres
Publié: (2026)

Optimizing Data Distribution and Kernel Performance for Efficient Training of Chemistry Foundation Models: A Case Study with MACE
par: Firoz, Jesun, et autres
Publié: (2025)

Verify Distributed Deep Learning Model Implementation Refinement with Iterative Relation Inference
par: Wang, Zhanghan, et autres
Publié: (2025)

High-Dimensional Data Processing: Benchmarking Machine Learning and Deep Learning Architectures in Local and Distributed Environments
par: Rodriguez, Julian, et autres
Publié: (2025)

Cloudless-Training: A Framework to Improve Efficiency of Geo-Distributed ML Training
par: Tan, Wenting, et autres
Publié: (2023)

Placement Semantics for Distributed Deep Learning: A Systematic Framework for Analyzing Parallelism Strategies
par: Mehta, Deep Pankajbhai
Publié: (2026)

Resource Allocation and Workload Scheduling for Large-Scale Distributed Deep Learning: A Survey
par: Liang, Feng, et autres
Publié: (2024)

Failure-Resilient Distributed Inference with Model Compression over Heterogeneous Edge Devices
par: Wang, Li, et autres
Publié: (2024)

SpaFL: Communication-Efficient Federated Learning with Sparse Models and Low computational Overhead
par: Kim, Minsu, et autres
Publié: (2024)

Demystifying the Communication Characteristics for Distributed Transformer Models
par: Anthony, Quentin, et autres
Publié: (2024)

COMET: A Comprehensive Cluster Design Methodology for Distributed Deep Learning Training
par: Kadiyala, Divya Kiran, et autres
Publié: (2022)

Why Should the Server Do It All?: A Scalable, Versatile, and Model-Agnostic Framework for Server-Light DNN Inference over Massively Distributed Clients via Training-Free Intermediate Feature Compression
par: Sung, Mingyu, et autres
Publié: (2025)

Gradient Correction in Federated Learning with Adaptive Optimization
par: Chen, Evan, et autres
Publié: (2025)

A HPX Communication Benchmark: Distributed FFT using Collectives
par: Strack, Alexander, et autres
Publié: (2025)

ModTrans: Translating Real-world Models for Distributed Training Simulator
par: Lyu, Yi
Publié: (2026)

Profiling-Driven Adaptive Distributed Transformer Inference on Embedded Edge Deployment
par: Qazi, Muhammad Azlan, et autres
Publié: (2026)

Sparse Training for Federated Learning with Regularized Error Correction
par: Greidi, Ran, et autres
Publié: (2023)

Distributed Low-Communication Training with Decoupled Momentum Optimization
par: Nedelkoski, Sasho, et autres
Publié: (2025)