Imagen de Portada

OpenDiLoCo: An Open-Source Framework for Globally Distributed Low-Communication Training

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Jaghouar, Sami, Ong, Jack Min, Hagemann, Johannes
Formato:	Preprint
Publicado:	2024
Materias:	Machine Learning Distributed, Parallel, and Cluster Computing
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Prime Collective Communications Library -- Technical Report
por: Keiblinger, Michael, et al.
Publicado: (2025)

INTELLECT-2: A Reasoning Model Trained Through Globally Decentralized Reinforcement Learning
por: Prime Intellect Team, et al.
Publicado: (2025)

Efficient Parallelization Layouts for Large-Scale Distributed Model Training
por: Hagemann, Johannes, et al.
Publicado: (2023)

TOPLOC: A Locality Sensitive Hashing Scheme for Trustless Verifiable Inference
por: Ong, Jack Min, et al.
Publicado: (2025)

Communication-Efficient Language Model Training Scales Reliably and Robustly: Scaling Laws for DiLoCo
por: Charles, Zachary, et al.
Publicado: (2025)

INTELLECT-1 Technical Report
por: Jaghouar, Sami, et al.
Publicado: (2024)

LoCoDL: Communication-Efficient Distributed Learning with Local Training and Compression
por: Condat, Laurent, et al.
Publicado: (2024)

What happens when nanochat meets DiLoCo?
por: Acker, Alexander, et al.
Publicado: (2025)

Communication Optimization for Distributed Training: Architecture, Advances, and Opportunities
por: Wei, Yunze, et al.
Publicado: (2024)

Arctic Inference with Shift Parallelism: Fast and Efficient Open Source Inference System for Enterprise AI
por: Rajbhandari, Samyam, et al.
Publicado: (2025)

Detection of Global Anomalies on Distributed IoT Edges with Device-to-Device Communication
por: Ochiai, Hideya, et al.
Publicado: (2024)

Distributed Low-Communication Training with Decoupled Momentum Optimization
por: Nedelkoski, Sasho, et al.
Publicado: (2025)

Federated LoRA with Sparse Communication
por: Kuo, Kevin, et al.
Publicado: (2024)

SparDL: Distributed Deep Learning Training with Efficient Sparse Communication
por: Zhao, Minjun, et al.
Publicado: (2023)

Low-Communication Resilient Distributed Estimation Algorithm Based on Memory Mechanism
por: Li, Wei, et al.
Publicado: (2025)

High-Dimensional Distributed Sparse Classification with Scalable Communication-Efficient Global Updates
por: Lu, Fred, et al.
Publicado: (2024)

Communication-Efficient Distributed Training for Collaborative Flat Optima Recovery in Deep Learning
por: Dimlioglu, Tolga, et al.
Publicado: (2025)

Paris: A Decentralized Trained Open-Weight Diffusion Model
por: Jiang, Zhiying, et al.
Publicado: (2025)

EarthSight: A Distributed Framework for Low-Latency Satellite Intelligence
por: Erol, Ansel Kaplan, et al.
Publicado: (2025)

DiOMP-Offloading: Toward Portable Distributed Heterogeneous OpenMP
por: Shan, Baodi, et al.
Publicado: (2025)

SDT-GNN: Streaming-based Distributed Training Framework for Graph Neural Networks
por: Huang, Xin, et al.
Publicado: (2024)

BOOST: BOttleneck-Optimized Scalable Training Framework for Low-Rank Large Language Models
por: Wang, Zhengyang, et al.
Publicado: (2025)

AntDT: A Self-Adaptive Distributed Training Framework for Leader and Straggler Nodes
por: Xiao, Youshao, et al.
Publicado: (2024)

GraNNDis: Efficient Unified Distributed Training Framework for Deep GNNs on Large Clusters
por: Song, Jaeyong, et al.
Publicado: (2023)

LoGoFair: Post-Processing for Local and Global Fairness in Federated Learning
por: Zhang, Li, et al.
Publicado: (2025)

Heterogeneity: An Open Challenge for Federated On-board Machine Learning
por: Hartmann, Maria, et al.
Publicado: (2024)

AB-Training: A Communication-Efficient Approach for Distributed Low-Rank Learning
por: Coquelin, Daniel, et al.
Publicado: (2024)

CDFGNN: a Systematic Design of Cache-based Distributed Full-Batch Graph Neural Network Training with Communication Reduction
por: Zhang, Shuai, et al.
Publicado: (2024)

Distributed Training under Packet Loss
por: Weintraub, Erez, et al.
Publicado: (2025)

Echo: Simulating Distributed Training At Scale
por: Feng, Yicheng, et al.
Publicado: (2024)

AutoRank: MCDA Based Rank Personalization for LoRA-Enabled Distributed Learning
por: Chen, Shuaijun, et al.
Publicado: (2024)

Privacy-Enhanced Training-as-a-Service for On-Device Intelligence: Concept, Architectural Scheme, and Open Problems
por: Wu, Zhiyuan, et al.
Publicado: (2024)

Accelerating Communication in Deep Learning Recommendation Model Training with Dual-Level Adaptive Lossy Compression
por: Feng, Hao, et al.
Publicado: (2024)

Empowering Distributed Training with Sparsity-driven Data Synchronization
por: Wang, Zhuang, et al.
Publicado: (2023)

Distributed Convolutional Neural Network Training on Mobile and Edge Clusters
por: Rama, Pranav, et al.
Publicado: (2024)

Communication-Efficient Federated Group Distributionally Robust Optimization
por: Guo, Zhishuai, et al.
Publicado: (2024)

SignMuon: Communication-Efficient Distributed Muon Optimization
por: Mishra, Neel, et al.
Publicado: (2026)

Lion Cub: Minimizing Communication Overhead in Distributed Lion
por: Ishikawa, Satoki, et al.
Publicado: (2024)

TetriServe: Efficient DiT Serving for Heterogeneous Image Generation
por: Lu, Runyu, et al.
Publicado: (2025)

Fed-BioMed: Open, Transparent and Trusted Federated Learning for Real-world Healthcare Applications
por: Cremonesi, Francesco, et al.
Publicado: (2023)