Cover Image

HETHUB: A Distributed Training System with Heterogeneous Cluster for Large-Scale Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Xu, Si, Huang, Zixiao, Zeng, Yan, Yan, Shengen, Ning, Xuefei, Zhang, Quanlu, Ye, Haolin, Gu, Sipei, Shui, Chunsheng, Lin, Zhezheng, Zhang, Hao, Wang, Sheng, Dai, Guohao, Wang, Yu
Format:	Preprint
Published:	2024
Subjects:	Distributed, Parallel, and Cluster Computing Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

STAlloc: Enhancing Memory Efficiency in Large-Scale Model Training with Spatio-Temporal Planning
by: Huang, Zixiao, et al.
Published: (2025)

DynaTrain: Fast Online Parallelism Switching for Elastic LLM Training
by: Wang, Yuanqing, et al.
Published: (2026)

HARP: Orchestrating Automated Parallel Training on Heterogeneous GPU Clusters
by: Liang, Antian, et al.
Published: (2025)

HAP: SPMD DNN Training on Heterogeneous GPU Clusters with Automated Program Synthesis
by: Zhang, Shiwei, et al.
Published: (2024)

FUSCO: High-Performance Distributed Data Shuffling via Transformation-Communication Fusion
by: Zhu, Zhuoran, et al.
Published: (2025)

HexiSeq: Accommodating Long Context Training of LLMs over Heterogeneous Hardware
by: Liang, Yan, et al.
Published: (2026)

Poplar: Efficient Scaling of Distributed DNN Training on Heterogeneous GPU Clusters
by: Zhang, WenZheng, et al.
Published: (2024)

Frenzy: A Memory-Aware Serverless LLM Training System for Heterogeneous GPU Clusters
by: Chang, Zihan, et al.
Published: (2024)

Accelerating Heterogeneous Tensor Parallelism via Flexible Workload Control
by: Wang, Zhigang, et al.
Published: (2024)

Holmes: Towards Distributed Training Across Clusters with Heterogeneous NIC Environment
by: Yang, Fei, et al.
Published: (2023)

Sailor: Automating Distributed Training over Dynamic, Heterogeneous, and Geo-distributed Clusters
by: Strati, Foteini, et al.
Published: (2025)

H2:Towards Efficient Large-Scale LLM Training on Hyper-Heterogeneous Cluster over 1,000 Chips
by: Tang, Ding, et al.
Published: (2025)

Training DNN Models over Heterogeneous Clusters with Optimal Performance
by: Nie, Chengyi, et al.
Published: (2024)

Cephalo: Harnessing Heterogeneous GPU Clusters for Training Transformer Models
by: Guo, Runsheng Benson, et al.
Published: (2024)

Zorse: Optimizing LLM Training Efficiency on Heterogeneous GPU Clusters
by: Guo, Runsheng Benson, et al.
Published: (2025)

Federated Incomplete Multi-View Clustering with Heterogeneous Graph Neural Networks
by: Yan, Xueming, et al.
Published: (2024)

DawnPiper: A Memory-scablable Pipeline Parallel Training Framework
by: Peng, Xuan, et al.
Published: (2025)

An Efficient, Reliable and Observable Collective Communication Library in Large-scale GPU Training Clusters
by: Zhang, Mingjun, et al.
Published: (2025)

HexiScale: Facilitating Large Language Model Training over Heterogeneous Hardware
by: Yan, Ran, et al.
Published: (2024)

AcOrch: Accelerating Sampling-based GNN Training under CPU-NPU Heterogeneous Environments
by: Chen, Kefu, et al.
Published: (2026)

Heta: Distributed Training of Heterogeneous Graph Neural Networks
by: Zhong, Yuchen, et al.
Published: (2024)

Hexa-MoE: Efficient and Heterogeneous-aware Training for Mixture-of-Experts
by: Luo, Shuqing, et al.
Published: (2024)

KUBEDIRECT: Unleashing the Full Power of the Cluster Manager for Serverless Computing
by: Qi, Sheng, et al.
Published: (2026)

Dalek: An Unconventional and Energy-Aware Heterogeneous Cluster
by: Cassagne, Adrien, et al.
Published: (2025)

DistTrain: Addressing Model and Data Heterogeneity with Disaggregated Training for Multimodal Large Language Models
by: Zhang, Zili, et al.
Published: (2024)

Optimal Resource Efficiency with Fairness in Heterogeneous GPU Clusters
by: Mo, Zizhao, et al.
Published: (2024)

HexGen-2: Disaggregated Generative Inference of LLMs in Heterogeneous Environment
by: Jiang, Youhe, et al.
Published: (2025)

Towards Affordable, Adaptive and Automatic GNN Training on CPU-GPU Heterogeneous Platforms
by: Qiao, Tong, et al.
Published: (2025)

Graph for Science: From API based Programming to Graph Engine based Programming for HPC
by: Zhang, Yu, et al.
Published: (2023)

Hamava: Fault-tolerant Reconfigurable Geo-Replication on Heterogeneous Clusters
by: Mane, Tejas, et al.
Published: (2024)

Efficient Training of Large Language Models on Distributed Infrastructures: A Survey
by: Duan, Jiangfei, et al.
Published: (2024)

JanusPipe: Efficient Pipeline Parallel Training for Machine Learning Interatomic Potentials
by: Wang, Hongyu, et al.
Published: (2026)

Robust Federated Fine-Tuning in Heterogeneous Networks with Unreliable Connections: An Aggregation View
by: Wang, Yanmeng, et al.
Published: (2025)

Parallel Cluster-BFS and Applications to Shortest Paths
by: Wang, Letong, et al.
Published: (2024)

Beyond A Single AI Cluster: A Survey of Decentralized LLM Training
by: Dong, Haotian, et al.
Published: (2025)

Addressing Variable Heterogeneity in Distributed Multimodal Training with Entrain
by: Jang, Insu, et al.
Published: (2026)

Augur: Pre-Execution Energy Prediction for Workflow Tasks in Heterogeneous Clusters
by: West, Kathleen, et al.
Published: (2026)

Monadring: A lightweight consensus protocol to offer Validation-as-a-Service to AVS nodes
by: Zhang, Yu, et al.
Published: (2024)

TSUE: A Two-Stage Data Update Method for an Erasure Coded Cluster File System
by: Wei, Zheng, et al.
Published: (2025)

HetRL: Efficient Reinforcement Learning for LLMs in Heterogeneous Environments
by: He, Yongjun, et al.
Published: (2025)