Image de couverture de livre

LuWu: An End-to-End In-Network Out-of-Core Optimizer for 100B-Scale Model-in-Network Data-Parallel Training on Distributed GPUs

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Sun, Mo, Yang, Zihan, Liao, Changyue, Li, Yingtao, Wu, Fei, Wang, Zeke
Format:	Preprint
Publié:	2024
Sujets:	Distributed, Parallel, and Cluster Computing
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

LoHan: Low-Cost High-Performance Framework to Fine-Tune 100B Model on a Consumer GPU
par: Liao, Changyue, et autres
Publié: (2024)

A Survey of End-to-End Modeling for Distributed DNN Training: Workloads, Simulators, and TCO
par: Svedas, Jonas, et autres
Publié: (2025)

Optimizing Distributed Training Approaches for Scaling Neural Networks
par: Baligodugula, Vishnu Vardhan, et autres
Publié: (2025)

Parallelizing Large-Scale Tensor Network Contraction on Multiple GPUs
par: Pan, Feng, et autres
Publié: (2026)

Cost-Effective Edge Data Distribution with End-To-End Delay Guarantees in Edge Computing
par: Shankar, Ravi, et autres
Publié: (2025)

A Proposed End-To-End Principle for Data Commons
par: Grossman, Robert L.
Publié: (2025)

End-to-End and Phase-Level Performance Optimization for Hyperledger Fabric
par: Sollu, Pavan, et autres
Publié: (2026)

ATLAS: Efficient Out-of-Core Inference for Billion-Scale Graph Neural Networks
par: Naman, Pranjal, et autres
Publié: (2026)

Saarthi: An End-to-End Intelligent Platform for Optimising Distributed Serverless Workloads
par: Agarwal, Siddharth, et autres
Publié: (2025)

Parallelizing Maximal Clique Enumeration on GPUs
par: Almasri, Mohammad, et autres
Publié: (2022)

Beyond End-to-End: Dynamic Chain Optimization for Private LLM Adaptation on the Edge
par: Wu, Yebo, et autres
Publié: (2026)

DeepCEE: Efficient Cross-Region Model Distributed Training System under Heterogeneous GPUs and Networks
par: Wang, Jinquan, et autres
Publié: (2025)

Nezha: Breaking Multi-Rail Network Barriers for Distributed DNN Training
par: Yu, Enda, et autres
Publié: (2024)

FpgaHub: Fpga-centric Hyper-heterogeneous Computing Platform for Big Data Analytics
par: Wang, Zeke, et autres
Publié: (2025)

ScaleLLM: A Resource-Frugal LLM Serving Framework by Optimizing End-to-End Efficiency
par: Yao, Yuhang, et autres
Publié: (2024)

Hyperion: Hierarchical Scheduling for Parallel LLM Acceleration in Multi-tier Networks
par: Ma, Mulei, et autres
Publié: (2025)

Scaling State-Space Models on Multiple GPUs with Tensor Parallelism
par: Dutt, Anurag, et autres
Publié: (2026)

An Adaptive Distributed Stencil Abstraction for GPUs
par: Bhosale, Aditya, et autres
Publié: (2025)

Training LLMs with Fault Tolerant HSDP on 100,000 GPUs
par: Salpekar, Omkar, et autres
Publié: (2026)

RapidGNN: Communication Efficient Large-Scale Distributed Training of Graph Neural Networks
par: Niam, Arefin, et autres
Publié: (2025)

CaPGNN: Optimizing Parallel Graph Neural Network Training with Joint Caching and Resource-Aware Graph Partitioning
par: Song, Xianfeng, et autres
Publié: (2025)

ClusterRCA: An End-to-End Approach for Network Fault Localization and Classification for HPC System
par: Sun, Yongqian, et autres
Publié: (2025)

Deal: Distributed End-to-End GNN Inference for All Nodes
par: Chen, Shiyang, et autres
Publié: (2025)

Combining Performance and Productivity: Accelerating the Network Sensing Graph Challenge with GPUs and Commodity Data Science Software
par: Samsi, Siddharth, et autres
Publié: (2025)

Optimizing sDTW for AMD GPUs
par: Latta-Lin, Daniel, et autres
Publié: (2024)

Anonymized Network Sensing using C++26 std::execution on GPUs
par: Mandulak, Michael, et autres
Publié: (2025)

Straggler Tolerant and Resilient DL Training on Homogeneous GPUs
par: Zhang, Zeyu, et autres
Publié: (2025)

Opara: Exploiting Operator Parallelism for Expediting DNN Inference on GPUs
par: Chen, Aodong, et autres
Publié: (2023)

APEX: Asynchronous Parallel CPU-GPU Execution for Online LLM Inference on Constrained GPUs
par: Fan, Jiakun, et autres
Publié: (2025)

Story of Two GPUs: Characterizing the Resilience of Hopper H100 and Ampere A100 GPUs
par: Cui, Shengkun, et autres
Publié: (2025)

KubeIntellect: A Modular LLM-Orchestrated Agent Framework for End-to-End Kubernetes Management
par: Ardebili, Mohsen Seyedkazemi, et autres
Publié: (2025)

A Parallel and Distributed Rust Library for Core Decomposition on Large Graphs
par: Rucci, Davide, et autres
Publié: (2025)

Heta: Distributed Training of Heterogeneous Graph Neural Networks
par: Zhong, Yuchen, et autres
Publié: (2024)

DreamDDP: Accelerating Data Parallel Distributed LLM Training with Layer-wise Scheduled Partial Synchronization
par: Tang, Zhenheng, et autres
Publié: (2025)

Hetis: Serving LLMs in Heterogeneous GPU Clusters with Fine-grained and Dynamic Parallelism
par: Mo, Zizhao, et autres
Publié: (2025)

SPARe: Stacked Parallelism with Adaptive Reordering for Fault-Tolerant LLM Pretraining Systems with 100k+ GPUs
par: Lee, Jin, et autres
Publié: (2026)

Parallel Collaborative ADMM Privacy Computing and Adaptive GPU Acceleration for Distributed Edge Networks
par: Xia, Mengchun, et autres
Publié: (2026)

Accelerating End-Cloud Collaborative Inference via Near Bubble-free Pipeline Optimization
par: Gao, Luyao, et autres
Publié: (2024)

Efficient Parallelization Layouts for Large-Scale Distributed Model Training
par: Hagemann, Johannes, et autres
Publié: (2023)

Distributed Consensus Network: A Modularized Communication Framework and Reliability Probabilistic Analysis
par: Li, Yuetai, et autres
Publié: (2025)