Imagen de Portada

Hydraulis: Balancing Large Transformer Model Training via Co-designing Parallel Strategies and Data Assignment

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Li, Haoyang, Fu, Fangcheng, Lin, Sheng, Ge, Hao, Wang, Xuanyu, Niu, Jiawen, Xue, Jinbao, Tao, Yangyu, Wang, Di, Jiang, Jie, Cui, Bin
Formato:	Preprint
Publicado:	2024
Materias:	Distributed, Parallel, and Cluster Computing Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Malleus: Straggler-Resilient Hybrid Parallel Training of Large-scale Models via Malleable Data and Model Parallelization
por: Li, Haoyang, et al.
Publicado: (2024)

LobRA: Multi-tenant Fine-tuning over Heterogeneous Data
por: Lin, Sheng, et al.
Publicado: (2025)

Hetu v2: A General and Scalable Deep Learning System with Hierarchical and Heterogeneous Single Program Multiple Data Annotations
por: Li, Haoyang, et al.
Publicado: (2025)

Unleashing Efficient Asynchronous RL Post-Training via Staleness-Constrained Rollout Coordination
por: Li, Haoyang, et al.
Publicado: (2026)

Improving Automatic Parallel Training via Balanced Memory Workload Optimization
por: Wang, Yujie, et al.
Publicado: (2023)

MEMO: Fine-grained Tensor Management For Ultra-long Context LLM Training
por: Zhao, Pinxue, et al.
Publicado: (2024)

InfiniPipe: Elastic Pipeline Parallelism for Efficient Variable-Length Long-Context LLM Training
por: Wang, Shiju, et al.
Publicado: (2025)

NeutronTP: Load-Balanced Distributed Full-Graph GNN Training with Tensor Parallelism
por: Ai, Xin, et al.
Publicado: (2024)

HexiScale: Facilitating Large Language Model Training over Heterogeneous Hardware
por: Yan, Ran, et al.
Publicado: (2024)

FlexSP: Accelerating Large Language Model Training via Flexible Sequence Parallelism
por: Wang, Yujie, et al.
Publicado: (2024)

Pro-Prophet: A Systematic Load Balancing Method for Efficient Parallel Training of Large-scale MoE Models
por: Wang, Wei, et al.
Publicado: (2024)

Zeppelin: Balancing Variable-length Workloads in Data Parallel Large Model Training
por: Chen, Chang, et al.
Publicado: (2025)

Balancing Pipeline Parallelism with Vocabulary Parallelism
por: Yeung, Man Tsung, et al.
Publicado: (2024)

ThunderServe: High-performance and Cost-efficient LLM Serving in Cloud Environments
por: Jiang, Youhe, et al.
Publicado: (2025)

LSH-MoE: Communication-efficient MoE Training via Locality-Sensitive Hashing
por: Nie, Xiaonan, et al.
Publicado: (2024)

LAER-MoE: Load-Adaptive Expert Re-layout for Efficient Mixture-of-Experts Training
por: Liu, Xinyi, et al.
Publicado: (2026)

Efficient Parallel Implementation of the Pilot Assignment Problem in Massive MIMO Systems
por: Alqudah, Eman, et al.
Publicado: (2025)

Joint Optimization of Offloading, Batching and DVFS for Multiuser Co-Inference
por: Xu, Yaodan, et al.
Publicado: (2025)

Efficient Multi-round LLM Inference over Disaggregated Serving
por: He, Wenhao, et al.
Publicado: (2026)

Arena: Efficiently Training Large Models via Dynamic Scheduling and Adaptive Parallelism Co-Design
por: Xue, Chunyu, et al.
Publicado: (2024)

COPUS: Co-adaptive Parallelism and Batch Size Selection in Large Language Model Training
por: Sakip, Akhmed, et al.
Publicado: (2026)

CoCoDiff: Optimizing Collective Communications for Distributed Diffusion Transformer Inference Under Ulysses Sequence Parallelism
por: Ma, Bin, et al.
Publicado: (2026)

S-HPLB: Efficient LLM Attention Serving via Sparsity-Aware Head Parallelism Load Balance
por: Liu, Di, et al.
Publicado: (2026)

HARP: Orchestrating Automated Parallel Training on Heterogeneous GPU Clusters
por: Liang, Antian, et al.
Publicado: (2025)

Large Scale Multi-GPU Based Parallel Traffic Simulation for Accelerated Traffic Assignment and Propagation
por: Jiang, Xuan, et al.
Publicado: (2024)

Adaptra: Straggler-Resilient Hybrid-Parallel Training with Pipeline Adaptation
por: Wu, Tianyuan, et al.
Publicado: (2025)

Spindle: Efficient Distributed Training of Multi-Task Large Models via Wavefront Scheduling
por: Wang, Yujie, et al.
Publicado: (2024)

Hecate: Unlocking Efficient Sparse Model Training via Fully Sharded Sparse Data Parallelism
por: Qing, Yuhao, et al.
Publicado: (2025)

JanusPipe: Efficient Pipeline Parallel Training for Machine Learning Interatomic Potentials
por: Wang, Hongyu, et al.
Publicado: (2026)

LoongTrain: Efficient Training of Long-Sequence LLMs with Head-Context Parallelism
por: Gu, Diandian, et al.
Publicado: (2024)

BOute: Cost-Efficient LLM Serving with Heterogeneous LLMs and GPUs via Multi-Objective Bayesian Optimization
por: Jiang, Youhe, et al.
Publicado: (2026)

Load Balanced Parallel Node Generation for Meshless Numerical Methods
por: Vehovar, Jon, et al.
Publicado: (2026)

OServe: Accelerating LLM Serving via Spatial-Temporal Workload Orchestration
por: Jiang, Youhe, et al.
Publicado: (2026)

Para-B&B: Load-Balanced Deterministic Parallelization of Solving MIP
por: Zhang, Jinyu, et al.
Publicado: (2026)

ResiHP: Taming LLM Training Failures with Dynamic Hybrid Parallelism
por: Ma, Tenghui, et al.
Publicado: (2026)

CFP: Efficient Optimization of Intra-Operator Parallelism Plans for Large Model Training
por: Hu, Weifang, et al.
Publicado: (2025)

Accelerating Heterogeneous Tensor Parallelism via Flexible Workload Control
por: Wang, Zhigang, et al.
Publicado: (2024)

TridentServe: A Stage-level Serving System for Diffusion Pipelines
por: Xia, Yifei, et al.
Publicado: (2025)

Demystifying Cost-Efficiency in LLM Serving over Heterogeneous GPUs
por: Jiang, Youhe, et al.
Publicado: (2025)

ElasWave: An Elastic-Native System for Scalable Hybrid-Parallel Training
por: Kang, Xueze, et al.
Publicado: (2025)