Cover Image

Go With The Flow: Churn-Tolerant Decentralized Training of Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Blagoev, Nikolay, Cox, Bart, Decouchant, Jérémie, Chen, Lydia Y.
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Distributed, Parallel, and Cluster Computing
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Dynamic Topology Optimization for Non-IID Data in Decentralized Learning
by: Cox, Bart, et al.
Published: (2026)

Aergia: Leveraging Heterogeneity in Federated Learning Systems
by: Cox, Bart, et al.
Published: (2022)

Training Diffusion Models with Federated Learning
by: de Goede, Matthijs, et al.
Published: (2024)

Asynchronous Byzantine Federated Learning
by: Cox, Bart, et al.
Published: (2024)

Asynchronous Multi-Server Federated Learning for Geo-Distributed Clients
by: Zuo, Yuncong, et al.
Published: (2024)

SkipPipe: Partial and Reordered Pipelining Framework for Training LLMs in Heterogeneous Networks
by: Blagoev, Nikolay, et al.
Published: (2025)

Parameterizing Federated Continual Learning for Reproducible Research
by: Cox, Bart, et al.
Published: (2024)

All is Not Lost: LLM Recovery without Checkpoints
by: Blagoev, Nikolay, et al.
Published: (2025)

TabVFL: Improving Latent Representation in Vertical Federated Learning
by: Rashad, Mohamed, et al.
Published: (2024)

Reliable Communication in Hybrid Authentication and Trust Models
by: Chotkan, Rowdy, et al.
Published: (2024)

Share Secrets for Privacy: Confidential Forecasting with Vertical Federated Learning
by: Shankar, Aditya, et al.
Published: (2024)

Tolerating Disasters with Hierarchical Consensus
by: Yahyaoui, Wassim, et al.
Published: (2025)

Herring: Parallel Batch-Order-Fairness on DAG-based Blockchain Consensus
by: Putnik, Marko, et al.
Published: (2026)

Optimizing Streamlined Blockchain Consensus with Generalized Weighted Voting and Enhanced Leader Rotation
by: Micloiu, Diana, et al.
Published: (2024)

Energy-Aware Decentralized Learning with Intermittent Model Training
by: Dhasade, Akash, et al.
Published: (2024)

NEMO: Faster Parallel Execution for Highly Contended Blockchain Workloads (Full version)
by: Ezard, François, et al.
Published: (2025)

SoK: Microservice Architectures from a Dependability Perspective
by: Kažemaks, Dāvis, et al.
Published: (2025)

The Blockchain Execution Dilemma: Optimizing Revenue XOR Fair Ordering
by: Pugatsov, Artjom, et al.
Published: (2026)

SmartShards: Churn-Tolerant Continuously Available Distributed Ledger
by: Oglio, Joseph, et al.
Published: (2025)

Heterogeneous Parallelism for Multimodal Large Language Model Training
by: Karnati, Yashaswi, et al.
Published: (2026)

INTELLECT-2: A Reasoning Model Trained Through Globally Decentralized Reinforcement Learning
by: Prime Intellect Team, et al.
Published: (2025)

MoC-System: Efficient Fault Tolerance for Sparse Mixture-of-Experts Model Training
by: Cai, Weilin, et al.
Published: (2024)

Partition Detection in Byzantine Networks
by: Bromberg, Yérom-David, et al.
Published: (2024)

MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs
by: Jiang, Ziheng, et al.
Published: (2024)

Optimizing Large Model Training through Overlapped Activation Recomputation
by: Chen, Ping, et al.
Published: (2024)

BOOST: BOttleneck-Optimized Scalable Training Framework for Low-Rank Large Language Models
by: Wang, Zhengyang, et al.
Published: (2025)

FlexSP: Accelerating Large Language Model Training via Flexible Sequence Parallelism
by: Wang, Yujie, et al.
Published: (2024)

Accelerating Large Language Model Training with 4D Parallelism and Memory Consumption Estimator
by: Fujii, Kazuki, et al.
Published: (2024)

Paris: A Decentralized Trained Open-Weight Diffusion Model
by: Jiang, Zhiying, et al.
Published: (2025)

Pipette: Automatic Fine-grained Large Language Model Training Configurator for Real-World Clusters
by: Yim, Jinkyu, et al.
Published: (2024)

AMDP: Asynchronous Multi-Directional Pipeline Parallelism for Large-Scale Models Training
by: Chen, Ling, et al.
Published: (2026)

Decentralized Learning Made Easy with DecentralizePy
by: Dhasade, Akash, et al.
Published: (2023)

Reducing Energy Bloat in Large Model Training
by: Chung, Jae-Won, et al.
Published: (2023)

The Streaming Batch Model for Efficient and Fault-Tolerant Heterogeneous Execution
by: Luan, Frank Sifei, et al.
Published: (2025)

Efficient Heterogeneous Large Language Model Decoding with Model-Attention Disaggregation
by: Chen, Shaoyuan, et al.
Published: (2024)

Efficient Long-context Language Model Training by Core Attention Disaggregation
by: Zhuang, Yonghao, et al.
Published: (2025)

Arena: Efficiently Training Large Models via Dynamic Scheduling and Adaptive Parallelism Co-Design
by: Xue, Chunyu, et al.
Published: (2024)

Decentralized Federated Learning with Model Caching on Mobile Agents
by: Wang, Xiaoyu, et al.
Published: (2024)

Characterization of Large Language Model Development in the Datacenter
by: Hu, Qinghao, et al.
Published: (2024)

Understanding Stragglers in Large Model Training Using What-if Analysis
by: Lin, Jinkun, et al.
Published: (2025)