Image de couverture de livre

Cephalo: Harnessing Heterogeneous GPU Clusters for Training Transformer Models

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Guo, Runsheng Benson, Anand, Utkarsh, Chen, Arthur, Daudjee, Khuzaima
Format:	Preprint
Publié:	2024
Sujets:	Distributed, Parallel, and Cluster Computing
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Zorse: Optimizing LLM Training Efficiency on Heterogeneous GPU Clusters
par: Guo, Runsheng Benson, et autres
Publié: (2025)

FreeRide: Harvesting Bubbles in Pipeline Parallelism
par: Zhang, Jiashu, et autres
Publié: (2024)

HARP: Orchestrating Automated Parallel Training on Heterogeneous GPU Clusters
par: Liang, Antian, et autres
Publié: (2025)

Poplar: Efficient Scaling of Distributed DNN Training on Heterogeneous GPU Clusters
par: Zhang, WenZheng, et autres
Publié: (2024)

HAP: SPMD DNN Training on Heterogeneous GPU Clusters with Automated Program Synthesis
par: Zhang, Shiwei, et autres
Publié: (2024)

Frenzy: A Memory-Aware Serverless LLM Training System for Heterogeneous GPU Clusters
par: Chang, Zihan, et autres
Publié: (2024)

HeteroSTA: A CPU-GPU Heterogeneous Static Timing Analysis Engine with Holistic Industrial Design Support
par: Guo, Zizheng, et autres
Publié: (2025)

Optimal Resource Efficiency with Fairness in Heterogeneous GPU Clusters
par: Mo, Zizhao, et autres
Publié: (2024)

Hetis: Serving LLMs in Heterogeneous GPU Clusters with Fine-grained and Dynamic Parallelism
par: Mo, Zizhao, et autres
Publié: (2025)

Training DNN Models over Heterogeneous Clusters with Optimal Performance
par: Nie, Chengyi, et autres
Publié: (2024)

An Efficient, Reliable and Observable Collective Communication Library in Large-scale GPU Training Clusters
par: Zhang, Mingjun, et autres
Publié: (2025)

Cronus: Efficient LLM inference on Heterogeneous GPU Clusters via Partially Disaggregated Prefill
par: Liu, Yunzhao, et autres
Publié: (2025)

Towards Affordable, Adaptive and Automatic GNN Training on CPU-GPU Heterogeneous Platforms
par: Qiao, Tong, et autres
Publié: (2025)

HarMoEny: Efficient Multi-GPU Inference of MoE Models
par: Doucet, Zachary, et autres
Publié: (2025)

PPipe: Efficient Video Analytics Serving on Heterogeneous GPU Clusters via Pool-Based Pipeline Parallelism
par: Kong, Z. Jonny, et autres
Publié: (2025)

Scaling Up Throughput-oriented LLM Inference Applications on Heterogeneous Opportunistic GPU Clusters with Pervasive Context Management
par: Phung, Thanh Son, et autres
Publié: (2025)

Sailor: Automating Distributed Training over Dynamic, Heterogeneous, and Geo-distributed Clusters
par: Strati, Foteini, et autres
Publié: (2025)

Predictable LLM Serving on GPU Clusters
par: Darzi, Erfan, et autres
Publié: (2025)

Efficiently Executing High-throughput Lightweight LLM Inference Applications on Heterogeneous Opportunistic GPU Clusters with Pervasive Context Management
par: Phung, Thanh Son, et autres
Publié: (2025)

gZCCL: Compression-Accelerated Collective Communication Framework for GPU Clusters
par: Huang, Jiajun, et autres
Publié: (2023)

BandPilot: Towards Performance- and Contention-Aware GPU Dispatching in AI Clusters
par: Zhang, Kunming, et autres
Publié: (2025)

Fantasy: Efficient Large-scale Vector Search on GPU Clusters with GPUDirect Async
par: Liu, Yi, et autres
Publié: (2025)

DeepOps & SLURM: Your GPU Cluster Guide
par: Majee, Arindam
Publié: (2024)

PAL: A Variability-Aware Policy for Scheduling ML Workloads in GPU Clusters
par: Jain, Rutwik, et autres
Publié: (2024)

Harnessing Integrated CPU-GPU System Memory for HPC: a first look into Grace Hopper
par: Schieffer, Gabin, et autres
Publié: (2024)

A Unified CPU-GPU Protocol for GNN Training
par: Lin, Yi-Chien, et autres
Publié: (2024)

GMLake: Efficient and Transparent GPU Memory Defragmentation for Large-scale DNN Training with Virtual Memory Stitching
par: Guo, Cong, et autres
Publié: (2024)

Optimizing Allreduce Operations for Modern Heterogeneous Architectures with Multiple Processes per GPU
par: Adams, Michael, et autres
Publié: (2025)

DistTrain: Addressing Model and Data Heterogeneity with Disaggregated Training for Multimodal Large Language Models
par: Zhang, Zili, et autres
Publié: (2024)

PRISM: Dynamic Primitive-Based Forecasting for Large-Scale GPU Cluster Workloads
par: Wu, Xin, et autres
Publié: (2026)

ClusterFusion++: Expanding Cluster-Level Fusion to Full Transformer-Block Decoding
par: Jin, ChiHeng, et autres
Publié: (2026)

Mell: Memory-Efficient Large Language Model Serving via Multi-GPU KV Cache Management
par: Qianli, Liu, et autres
Publié: (2025)

Dalek: An Unconventional and Energy-Aware Heterogeneous Cluster
par: Cassagne, Adrien, et autres
Publié: (2025)

H2:Towards Efficient Large-Scale LLM Training on Hyper-Heterogeneous Cluster over 1,000 Chips
par: Tang, Ding, et autres
Publié: (2025)

GFS: A Preemption-aware Scheduling Framework for GPU Clusters with Predictive Spot Instance Management
par: Duan, Jiaang, et autres
Publié: (2025)

PowerTrip: Exploiting Federated Heterogeneous Datacenter Power for Distributed ML Training
par: Mehboob, Talha, et autres
Publié: (2025)

TurboFFT: A High-Performance Fast Fourier Transform with Fault Tolerance on GPU
par: Wu, Shixun, et autres
Publié: (2024)

The Energy Cost of Execution-Idle in GPU Clusters
par: Lei, Yiran, et autres
Publié: (2026)

Cultivating Multidisciplinary AI Workforce Development on iTiger GPU Cluster: Practices and Challenges
par: Sharif, Mayira, et autres
Publié: (2025)

GPU Memory and Utilization Estimation for Training-Aware Resource Management: Opportunities and Limitations
par: Yousefzadeh-Asl-Miandoab, Ehsan, et autres
Publié: (2026)