Copertina

MiCRO: Near-Zero Cost Gradient Sparsification for Scaling and Accelerating Distributed DNN Training

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Yoon, Daegun, Oh, Sangyoon
Natura:	Preprint
Pubblicazione:	2023
Soggetti:	Machine Learning Distributed, Parallel, and Cluster Computing
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Preserving Near-Optimal Gradient Sparsification Cost for Scalable Distributed Deep Learning
di: Yoon, Daegun, et al.
Pubblicazione: (2024)

Universal Checkpointing: A Flexible and Efficient Distributed Checkpointing System for Large-Scale DNN Training with Reconfigurable Parallelis
di: Lian, Xinyu, et al.
Pubblicazione: (2024)

Towards a Flexible and High-Fidelity Approach to Distributed DNN Training Emulation
di: Liu, Banruo, et al.
Pubblicazione: (2024)

Mask-Encoded Sparsification: Mitigating Biased Gradients in Communication-Efficient Split Learning
di: Zhou, Wenxuan, et al.
Pubblicazione: (2024)

PaSE: Parallelization Strategies for Efficient DNN Training
di: Elango, Venmugil
Pubblicazione: (2024)

Echo: Simulating Distributed Training At Scale
di: Feng, Yicheng, et al.
Pubblicazione: (2024)

Adaptive Consensus Gradients Aggregation for Scaled Distributed Training
di: Choukroun, Yoni, et al.
Pubblicazione: (2024)

Hardware Scaling Trends and Diminishing Returns in Large-Scale Distributed Training
di: Fernandez, Jared, et al.
Pubblicazione: (2024)

TurboGR: An Accelerated Training System for Large-Scale Generative Recommendation
di: Chai, Huichao, et al.
Pubblicazione: (2026)

Efficient Parallelization Layouts for Large-Scale Distributed Model Training
di: Hagemann, Johannes, et al.
Pubblicazione: (2023)

Reducing Communication for Split Learning by Randomized Top-k Sparsification
di: Zheng, Fei, et al.
Pubblicazione: (2023)

Resource-Aware Aggregation and Sparsification in Heterogeneous Ensemble Federated Learning
di: Ryum, Keumseo, et al.
Pubblicazione: (2025)

Enabling Large Batch Size Training for DNN Models Beyond the Memory Limit While Maintaining Performance
di: Piao, XinYu, et al.
Pubblicazione: (2021)

ECHO-2: A Large-Scale Distributed Rollout Framework for Cost-Efficient Reinforcement Learning
di: Song, Jingwei, et al.
Pubblicazione: (2026)

Poplar: Efficient Scaling of Distributed DNN Training on Heterogeneous GPU Clusters
di: Zhang, WenZheng, et al.
Pubblicazione: (2024)

NestPipe: Large-Scale Recommendation Training on 1,500+ Accelerators via Nested Pipelining
di: Jiang, Zhida, et al.
Pubblicazione: (2026)

Armada: Memory-Efficient Distributed Training of Large-Scale Graph Neural Networks
di: Waleffe, Roger, et al.
Pubblicazione: (2025)

Practical Performance Guarantees for Pipelined DNN Inference
di: Archer, Aaron, et al.
Pubblicazione: (2023)

Accelerating Federated Learning by Selecting Beneficial Herd of Local Gradients
di: Luo, Ping, et al.
Pubblicazione: (2024)

A Survey on Collaborative DNN Inference for Edge Intelligence
di: Ren, Weiqing, et al.
Pubblicazione: (2022)

Learning the Optimal Path and DNN Partition for Collaborative Edge Inference
di: Huang, Yin, et al.
Pubblicazione: (2024)

Near-Zero-Overhead Freshness for Recommendation Systems via Inference-Side Model Updates
di: Yu, Wenjun, et al.
Pubblicazione: (2025)

Distributed Deep Learning using Stochastic Gradient Staleness
di: Pham, Viet Hoang, et al.
Pubblicazione: (2025)

HetCCL: Accelerating LLM Training with Heterogeneous GPUs
di: Kim, Heehoon, et al.
Pubblicazione: (2026)

Symphony: Optimized DNN Model Serving using Deferred Batch Scheduling
di: Chen, Lequn, et al.
Pubblicazione: (2023)

Performance Characterization of Containerized DNN Training and Inference on Edge Accelerators
di: K., Prashanthi S., et al.
Pubblicazione: (2023)

Fulcrum: Optimizing Concurrent DNN Training and Inferencing on Edge Accelerators
di: K., Prashanthi S., et al.
Pubblicazione: (2025)

Efficient Data Distribution Estimation for Accelerated Federated Learning
di: Wang, Yuanli, et al.
Pubblicazione: (2024)

Effectiveness of Distributed Gradient Descent with Local Steps for Overparameterized Models
di: Zhu, Heng, et al.
Pubblicazione: (2024)

Grappa: Gradient-Only Communication for Scalable Graph Neural Network Training
di: Xu, Chongyang, et al.
Pubblicazione: (2026)

Distributed Training under Packet Loss
di: Weintraub, Erez, et al.
Pubblicazione: (2025)

Traversal Learning: A Lossless And Efficient Distributed Learning Framework
di: Batbaatar, Erdenebileg, et al.
Pubblicazione: (2025)

FlashMem: Supporting Modern DNN Workloads on Mobile with GPU Memory Hierarchy Optimizations
di: Shu, Zhihao, et al.
Pubblicazione: (2026)

CARIn: Constraint-Aware and Responsive Inference on Heterogeneous Devices for Single- and Multi-DNN Workloads
di: Panopoulos, Ioannis, et al.
Pubblicazione: (2024)

Improved Quantization Strategies for Managing Heavy-tailed Gradients in Distributed Learning
di: Yan, Guangfeng, et al.
Pubblicazione: (2024)

Distributed Learning based on 1-Bit Gradient Coding in the Presence of Stragglers
di: Li, Chengxi, et al.
Pubblicazione: (2024)

MLTCP: Congestion Control for DNN Training
di: Rajasekaran, Sudarsanan, et al.
Pubblicazione: (2024)

MinatoLoader: Accelerating Machine Learning Training Through Efficient Data Preprocessing
di: Nouaji, Rahma, et al.
Pubblicazione: (2025)

NestQuant: Post-Training Integer-Nesting Quantization for On-Device DNN
di: Xie, Jianhang, et al.
Pubblicazione: (2025)

Unicron: Economizing Self-Healing LLM Training at Scale
di: He, Tao, et al.
Pubblicazione: (2023)