Cover Image

Data movement limits to frontier model training

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Erdil, Ege, Schneider-Joseph, David
Format:	Preprint
Published:	2024
Subjects:	Distributed, Parallel, and Cluster Computing Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Inference economics of language models
by: Erdil, Ege
Published: (2025)

Improving training time and GPU utilization in geo-distributed language model training
by: Palak, et al.
Published: (2024)

Recipes for Pre-training LLMs with MXFP8
by: Mishra, Asit, et al.
Published: (2025)

The Future of Large Language Model Pre-training is Federated
by: Sani, Lorenzo, et al.
Published: (2024)

FLStore: Efficient Federated Learning Storage for non-training workloads
by: Khan, Ahmad Faraz, et al.
Published: (2025)

Stable Diffusion-based Data Augmentation for Federated Learning with Non-IID Data
by: Morafah, Mahdi, et al.
Published: (2024)

2BP: 2-Stage Backpropagation
by: Rae, Christopher, et al.
Published: (2024)

Redefining Data-Centric Design: A New Approach with a Domain Model and Core Data Ontology for Computational Systems
by: Johnson, William, et al.
Published: (2024)

On the Fragility of Data Attribution When Learning Is Distributed
by: Gao, Xian, et al.
Published: (2026)

Efficient Federated Learning with Heterogeneous Data and Adaptive Dropout
by: Liu, Ji, et al.
Published: (2025)

The Built-In Robustness of Decentralized Federated Averaging to Bad Data
by: Sabella, Samuele, et al.
Published: (2025)

Enhancing Data Quality in Federated Fine-Tuning of Foundation Models
by: Zhao, Wanru, et al.
Published: (2024)

LCFed: An Efficient Clustered Federated Learning Framework for Heterogeneous Data
by: Zhang, Yuxin, et al.
Published: (2025)

Full Scaling Automation for Sustainable Development of Green Data Centers
by: Wang, Shiyu, et al.
Published: (2023)

Learn How to Query from Unlabeled Data Streams in Federated Learning
by: Sun, Yuchang, et al.
Published: (2024)

FedAC: An Adaptive Clustered Federated Learning Framework for Heterogeneous Data
by: Zhang, Yuxin, et al.
Published: (2024)

Task-Agnostic Federation over Decentralized Data: Research Landscape and Visions
by: Wu, Wentai, et al.
Published: (2025)

Personalizing Federated Learning for Hierarchical Edge Networks with Non-IID Data
by: Lee, Seunghyun, et al.
Published: (2025)

FedACT: Concurrent Federated Intelligence across Heterogeneous Data Sources
by: Islam, Md Sirajul, et al.
Published: (2026)

Measuring Heterogeneity in Machine Learning with Distributed Energy Distance
by: Fan, Mengchen, et al.
Published: (2025)

Data-Free Client Contribution Estimation via Logit Maximization for Federated Learning
by: Ukaye, Asim, et al.
Published: (2026)

Mosaic: Data-Free Knowledge Distillation via Mixture-of-Experts for Heterogeneous Distributed Environments
by: Liu, Junming, et al.
Published: (2025)

FedClust: Tackling Data Heterogeneity in Federated Learning through Weight-Driven Client Clustering
by: Islam, Md Sirajul, et al.
Published: (2024)

Efficient Federated Learning Using Dynamic Update and Adaptive Pruning with Momentum on Shared Server Data
by: Liu, Ji, et al.
Published: (2024)

Compare Where It Matters: Using Layer-Wise Regularization To Improve Federated Learning on Heterogeneous Data
by: Son, Ha Min, et al.
Published: (2021)

FedDAG: Clustered Federated Learning via Global Data and Gradient Integration for Heterogeneous Environments
by: Pramanik, Anik, et al.
Published: (2026)

Semantic Parallelism: Redefining Efficient MoE Inference via Model-Data Co-Scheduling
by: Li, Yan, et al.
Published: (2025)

Federated Learning with Workload Reduction through Partial Training of Client Models and Entropy-Based Data Selection
by: Shi, Hongrui, et al.
Published: (2024)

Loop Improvement: An Efficient Approach for Extracting Shared Features from Heterogeneous Data without Central Server
by: Li, Fei, et al.
Published: (2024)

HybridEP: Scaling Expert Parallelism to Cross-Datacenter Scenario via Hybrid Expert/Data Transmission
by: Yang, Weihao, et al.
Published: (2025)

FedLECC: Cluster- and Loss-Guided Client Selection for Federated Learning under Non-IID Data
by: Jimenez-Gutierrez, Daniel M., et al.
Published: (2026)

Towards the Next Frontier of LLMs, Training on Private Data: A Cross-Domain Benchmark for Federated Fine-Tuning
by: Jimenez-Gutierrez, Daniel M., et al.
Published: (2026)

Multi-Worker Selection based Distributed Swarm Learning for Edge IoT with Non-i.i.d. Data
by: Yao, Zhuoyu, et al.
Published: (2025)

FedPBS: Proximal-Balanced Scaling Federated Learning Model for Robust Personalized Training for Non-IID Data
by: AbouNassar, Eman M., et al.
Published: (2026)

DISTFLASHATTN: Distributed Memory-efficient Attention for Long-context LLMs Training
by: Li, Dacheng, et al.
Published: (2023)

MoE-Lightning: High-Throughput MoE Inference on Memory-constrained GPUs
by: Cao, Shiyi, et al.
Published: (2024)

Game-Theoretic Deep Reinforcement Learning to Minimize Carbon Emissions and Energy Costs for AI Inference Workloads in Geo-Distributed Data Centers
by: Hogade, Ninad, et al.
Published: (2024)

Autellix: An Efficient Serving Engine for LLM Agents as General Programs
by: Luo, Michael, et al.
Published: (2025)

Conflict-Free Replicated Data Types for Neural Network Model Merging: A Two-Layer Architecture Enabling CRDT-Compliant Model Merging Across 26 Strategies
by: Gillespie, Ryan
Published: (2026)

S-LoRA: Serving Thousands of Concurrent LoRA Adapters
by: Sheng, Ying, et al.
Published: (2023)