Copertina

WLB-LLM: Workload-Balanced 4D Parallelism for Large Language Model Training

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Wang, Zheng, Cai, Anna, Xie, Xinfeng, Pan, Zaifeng, Guan, Yue, Chu, Weiwei, Wang, Jie, Li, Shikai, Huang, Jianyu, Cai, Chris, Hao, Yuchen, Ding, Yufei
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Distributed, Parallel, and Cluster Computing Artificial Intelligence Machine Learning I.2.11
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Zeppelin: Balancing Variable-length Workloads in Data Parallel Large Model Training
di: Chen, Chang, et al.
Pubblicazione: (2025)

Training Diffusion Models with Federated Learning
di: de Goede, Matthijs, et al.
Pubblicazione: (2024)

Quantize Once, Train Fast: Allreduce-Compatible Compression with Provable Guarantees
di: Xin, Jihao, et al.
Pubblicazione: (2023)

KVFlow: Efficient Prefix Caching for Accelerating LLM-Based Multi-Agent Workflows
di: Pan, Zaifeng, et al.
Pubblicazione: (2025)

Accelerating Compound LLM Training Workloads with Maestro
di: Yuan, Xiulong, et al.
Pubblicazione: (2026)

CodeCRDT: Observation-Driven Coordination for Multi-Agent LLM Code Generation
di: Pugachev, Sergey
Pubblicazione: (2025)

Roadmap for Edge AI: A Dagstuhl Perspective
di: Ding, Aaron Yi, et al.
Pubblicazione: (2021)

SparkAttention: High-Performance Multi-Head Attention for Large Models on Volta GPU Architecture
di: Xu, Youxuan, et al.
Pubblicazione: (2025)

Improving Automatic Parallel Training via Balanced Memory Workload Optimization
di: Wang, Yujie, et al.
Pubblicazione: (2023)

ScaleSim: Serving Large-Scale Multi-Agent Simulation with Invocation Distance-Based Memory Management
di: Pan, Zaifeng, et al.
Pubblicazione: (2026)

Aergia: Leveraging Heterogeneity in Federated Learning Systems
di: Cox, Bart, et al.
Pubblicazione: (2022)

Towards Optimal Heterogeneous Client Sampling in Multi-Model Federated Learning
di: Zhang, Haoran, et al.
Pubblicazione: (2025)

Parameterizing Federated Continual Learning for Reproducible Research
di: Cox, Bart, et al.
Pubblicazione: (2024)

Asynchronous Byzantine Federated Learning
di: Cox, Bart, et al.
Pubblicazione: (2024)

Hyper-parameter Optimization for Federated Learning with Step-wise Adaptive Mechanism
di: Saadati, Yasaman, et al.
Pubblicazione: (2024)

A Domain-Driven Design Simulator for Business Logic-Rich Microservice Systems
di: Pereira, Daniel da Palma, et al.
Pubblicazione: (2026)

Connecting Large Language Model Agent to High Performance Computing Resource
di: Ma, Heng, et al.
Pubblicazione: (2025)

Streaming REST APIs for Large Financial Transaction Exports from Relational Databases
di: Kandiraju, Abhiram
Pubblicazione: (2026)

Comparison of Autoscaling Frameworks for Containerised Machine-Learning-Applications in a Local and Cloud Environment
di: Schroeder, Christian, et al.
Pubblicazione: (2023)

Asynchronous Multi-Server Federated Learning for Geo-Distributed Clients
di: Zuo, Yuncong, et al.
Pubblicazione: (2024)

Learning In Chaos: Efficient Autoscaling and Self-Healing for Multi-Party Distributed Training
di: Feng, Wenjiao, et al.
Pubblicazione: (2025)

Accelerating Heterogeneous Tensor Parallelism via Flexible Workload Control
di: Wang, Zhigang, et al.
Pubblicazione: (2024)

Shift Parallelism: Low-Latency, High-Throughput LLM Inference for Dynamic Workloads
di: Hidayetoglu, Mert, et al.
Pubblicazione: (2025)

FedPLT: Scalable, Resource-Efficient, and Heterogeneity-Aware Federated Learning via Partial Layer Training
di: Dabaja, Ahmad, et al.
Pubblicazione: (2026)

MegaScale-Omni: A Hyper-Scale, Workload-Resilient System for MultiModal LLM Training in Production
di: Xue, Chunyu, et al.
Pubblicazione: (2026)

TPI-LLM: Serving 70B-scale LLMs Efficiently on Low-resource Edge Devices
di: Li, Zonghang, et al.
Pubblicazione: (2024)

Serial Parallel Reliability Redundancy Allocation Optimization for Energy Efficient and Fault Tolerant Cloud Computing
di: Krishna, Gutha Jaya
Pubblicazione: (2024)

DreamDDP: Accelerating Data Parallel Distributed LLM Training with Layer-wise Scheduled Partial Synchronization
di: Tang, Zhenheng, et al.
Pubblicazione: (2025)

EPARA: Parallelizing Categorized AI Inference in Edge Clouds
di: Wang, Yubo, et al.
Pubblicazione: (2025)

Separating Intelligence from Execution: A Workflow Engine for the Model Context Protocol
di: Parmar, Abhinav Singh
Pubblicazione: (2026)

Accelerating Geo-distributed Machine Learning with Network-Aware Adaptive Tree and Auxiliary Route
di: Li, Zonghang, et al.
Pubblicazione: (2024)

nvidia-pcm: A D-Bus-Driven Platform Configuration Manager for OpenBMC Environments
di: Singh, Harinder
Pubblicazione: (2026)

AutoDDL: Automatic Distributed Deep Learning with Near-Optimal Bandwidth Cost
di: Chen, Jinfan, et al.
Pubblicazione: (2023)

SI-ChainFL: Shapley-Incentivized Secure Federated Learning for High-Speed Rail Data Sharing
di: Zhao, Mingjie, et al.
Pubblicazione: (2026)

How Machine Learning-Data Driven Replication Strategies Enhance Fault Tolerance in Large-Scale Distributed Systems
di: Murimi, Almond Kiruthu
Pubblicazione: (2025)

Intelligent Router for LLM Workloads: Improving Performance Through Workload-Aware Load Balancing
di: Jain, Kunal, et al.
Pubblicazione: (2024)

FlashEvolve: Accelerating Agent Self-Evolution with Asynchronous Stage Orchestration
di: Hu, Zhengding, et al.
Pubblicazione: (2026)

Chimera: Efficiently Training Large-Scale Neural Networks with Bidirectional Pipelines
di: Li, Shigang, et al.
Pubblicazione: (2021)

A Survey on Efficient Federated Learning Methods for Foundation Model Training
di: Woisetschläger, Herbert, et al.
Pubblicazione: (2024)

WRATH: Workload Resilience Across Task Hierarchies in Task-based Parallel Programming Frameworks
di: Zhou, Sicheng, et al.
Pubblicazione: (2025)