Cover Image

Quantize Once, Train Fast: Allreduce-Compatible Compression with Provable Guarantees

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Xin, Jihao, Canini, Marco, Richtárik, Peter, Horváth, Samuel
Format:	Preprint
Published:	2023
Subjects:	Machine Learning Distributed, Parallel, and Cluster Computing I.2.11
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Near-Optimal Sparse Allreduce for Distributed Deep Learning
by: Li, Shigang, et al.
Published: (2022)

Training Diffusion Models with Federated Learning
by: de Goede, Matthijs, et al.
Published: (2024)

Aergia: Leveraging Heterogeneity in Federated Learning Systems
by: Cox, Bart, et al.
Published: (2022)

Towards Optimal Heterogeneous Client Sampling in Multi-Model Federated Learning
by: Zhang, Haoran, et al.
Published: (2025)

Parameterizing Federated Continual Learning for Reproducible Research
by: Cox, Bart, et al.
Published: (2024)

Asynchronous Byzantine Federated Learning
by: Cox, Bart, et al.
Published: (2024)

Hyper-parameter Optimization for Federated Learning with Step-wise Adaptive Mechanism
by: Saadati, Yasaman, et al.
Published: (2024)

Asynchronous Multi-Server Federated Learning for Geo-Distributed Clients
by: Zuo, Yuncong, et al.
Published: (2024)

A Full Compression Pipeline for Green Federated Learning in Communication-Constrained Environments
by: Colybes, Elouan, et al.
Published: (2026)

WLB-LLM: Workload-Balanced 4D Parallelism for Large Language Model Training
by: Wang, Zheng, et al.
Published: (2025)

FedPLT: Scalable, Resource-Efficient, and Heterogeneity-Aware Federated Learning via Partial Layer Training
by: Dabaja, Ahmad, et al.
Published: (2026)

FlashSparse: Minimizing Computation Redundancy for Fast Sparse Matrix Multiplications on Tensor Cores
by: Shi, Jinliang, et al.
Published: (2024)

ADF-LoRA: Alternating Low-Rank Aggregation for Decentralized Federated Fine-Tuning
by: Wang, Xiaoyu, et al.
Published: (2025)

DAGER: Exact Gradient Inversion for Large Language Models
by: Petrov, Ivo, et al.
Published: (2024)

Chimera: Efficiently Training Large-Scale Neural Networks with Bidirectional Pipelines
by: Li, Shigang, et al.
Published: (2021)

Comparison of Autoscaling Frameworks for Containerised Machine-Learning-Applications in a Local and Cloud Environment
by: Schroeder, Christian, et al.
Published: (2023)

Roadmap for Edge AI: A Dagstuhl Perspective
by: Ding, Aaron Yi, et al.
Published: (2021)

Connecting Large Language Model Agent to High Performance Computing Resource
by: Ma, Heng, et al.
Published: (2025)

Uncertainty Estimation in Multi-Agent Distributed Learning for AI-Enabled Edge Devices
by: Radchenko, Gleb, et al.
Published: (2024)

Edge AI Collaborative Learning: Bayesian Approaches to Uncertainty Estimation
by: Radchenko, Gleb, et al.
Published: (2024)

SPEAR++: Scaling Gradient Inversion via Sparsely-Used Dictionary Learning
by: Bakarsky, Alexander, et al.
Published: (2025)

SPEAR:Exact Gradient Inversion of Batches in Federated Learning
by: Dimitrov, Dimitar I., et al.
Published: (2024)

Federated Learning with Differential Privacy
by: Banse, Adrien, et al.
Published: (2024)

SI-ChainFL: Shapley-Incentivized Secure Federated Learning for High-Speed Rail Data Sharing
by: Zhao, Mingjie, et al.
Published: (2026)

CodeCRDT: Observation-Driven Coordination for Multi-Agent LLM Code Generation
by: Pugachev, Sergey
Published: (2025)

Separating Intelligence from Execution: A Workflow Engine for the Model Context Protocol
by: Parmar, Abhinav Singh
Published: (2026)

A Survey on Efficient Federated Learning Methods for Foundation Model Training
by: Woisetschläger, Herbert, et al.
Published: (2024)

Learning In Chaos: Efficient Autoscaling and Self-Healing for Multi-Party Distributed Training
by: Feng, Wenjiao, et al.
Published: (2025)

TAGC: Optimizing Gradient Communication in Distributed Transformer Training
by: Polyakov, Igor, et al.
Published: (2025)

AutoDDL: Automatic Distributed Deep Learning with Near-Optimal Bandwidth Cost
by: Chen, Jinfan, et al.
Published: (2023)

Robustness of deep learning classification to adversarial input on GPUs: asynchronous parallel accumulation is a source of vulnerability
by: Shanmugavelu, Sanjif, et al.
Published: (2025)

GRAIN: Exact Graph Reconstruction from Gradients
by: Drencheva, Maria, et al.
Published: (2025)

HFedATM: Hierarchical Federated Domain Generalization via Optimal Transport and Regularized Mean Aggregation
by: Nguyen, Thinh, et al.
Published: (2025)

SparkAttention: High-Performance Multi-Head Attention for Large Models on Volta GPU Architecture
by: Xu, Youxuan, et al.
Published: (2025)

How Machine Learning-Data Driven Replication Strategies Enhance Fault Tolerance in Large-Scale Distributed Systems
by: Murimi, Almond Kiruthu
Published: (2025)

Federated Learning for Traffic Flow Prediction with Synthetic Data Augmentation
by: Orozco, Fermin, et al.
Published: (2024)

SPARK: Igniting Communication-Efficient Decentralized Learning via Stage-wise Projected NTK and Accelerated Regularization
by: Xia, Li
Published: (2025)

Libra: Unleashing GPU Heterogeneity for High-Performance Sparse Matrix Multiplication
by: Shi, Jinliang, et al.
Published: (2025)

Risk Management for Distributed Arbitrage Systems: Integrating Artificial Intelligence
by: Hazarika, Akaash Vishal, et al.
Published: (2025)

LayerKV: Optimizing Large Language Model Serving with Layer-wise KV Cache Management
by: Xiong, Yi, et al.
Published: (2024)