Copertina

NeutronTP: Load-Balanced Distributed Full-Graph GNN Training with Tensor Parallelism

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Ai, Xin, Yuan, Hao, Ling, Zeyu, Wang, Qiange, Zhang, Yanfeng, Fu, Zhenbo, Chen, Chaoyi, Gu, Yu, Yu, Ge
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Distributed, Parallel, and Cluster Computing
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Comprehensive Evaluation of GNN Training Systems: A Data Management Perspective
di: Yuan, Hao, et al.
Pubblicazione: (2023)

AcOrch: Accelerating Sampling-based GNN Training under CPU-NPU Heterogeneous Environments
di: Chen, Kefu, et al.
Pubblicazione: (2026)

Incremental GNN Embedding Computation on Streaming Graphs
di: Wang, Qiange, et al.
Pubblicazione: (2026)

AnchorTP: Resilient LLM Inference with State-Preserving Elastic Tensor Parallelism
di: Xu, Wendong, et al.
Pubblicazione: (2025)

Accelerating Heterogeneous Tensor Parallelism via Flexible Workload Control
di: Wang, Zhigang, et al.
Pubblicazione: (2024)

Pro-Prophet: A Systematic Load Balancing Method for Efficient Parallel Training of Large-scale MoE Models
di: Wang, Wei, et al.
Pubblicazione: (2024)

S-HPLB: Efficient LLM Attention Serving via Sparsity-Aware Head Parallelism Load Balance
di: Liu, Di, et al.
Pubblicazione: (2026)

GreenDyGNN: Runtime-Adaptive Energy-Efficient Communication for Distributed GNN Training
di: Niam, Arefin, et al.
Pubblicazione: (2026)

Load Balanced Parallel Node Generation for Meshless Numerical Methods
di: Vehovar, Jon, et al.
Pubblicazione: (2026)

HopGNN: Boosting Distributed GNN Training Efficiency via Feature-Centric Model Migration
di: Chen, Weijian, et al.
Pubblicazione: (2024)

FEPLB: Exploiting Copy Engines for Nearly Free MoE Load Balancing in Distributed Training
di: Qi, Shuyao, et al.
Pubblicazione: (2026)

DualMap: Enabling Both Cache Affinity and Load Balancing for Distributed LLM Serving
di: Yuan, Ying, et al.
Pubblicazione: (2026)

Accelerating Long-Tail Generation in Synchronous RLHF Training via Adaptive Tensor Parallelism
di: Zhao, Long, et al.
Pubblicazione: (2026)

GriNNder: Breaking the Memory Capacity Wall in Full-Graph GNN Training with Storage Offloading
di: Song, Jaeyong, et al.
Pubblicazione: (2026)

TD-Orch: Scalable Load-Balancing for Distributed Systems with Applications to Graph Processing
di: Zhao, Yiwei, et al.
Pubblicazione: (2025)

Oases: Efficient Large-Scale Model Training on Commodity Servers via Overlapped and Automated Tensor Model Parallelism
di: Li, Shengwei, et al.
Pubblicazione: (2023)

Distributed Load Balancing with Workload-Dependent Service Rates
di: Zhang, Wenxin, et al.
Pubblicazione: (2024)

Amoeba: Runtime Tensor Parallel Transformation for LLM Inference Services
di: Chen, Haoyu, et al.
Pubblicazione: (2025)

Tackling the Data-Parallel Load Balancing Bottleneck in LLM Serving: Practical Online Routing at Scale
di: Bu, Tianci, et al.
Pubblicazione: (2026)

Balancing Pipeline Parallelism with Vocabulary Parallelism
di: Yeung, Man Tsung, et al.
Pubblicazione: (2024)

RapidGNN: Communication Efficient Large-Scale Distributed Training of Graph Neural Networks
di: Niam, Arefin, et al.
Pubblicazione: (2025)

CondenseGraph: Communication-Efficient Distributed GNN Training via On-the-Fly Graph Condensation
di: Zhang, Zizhao, et al.
Pubblicazione: (2026)

Synergistic Tensor and Pipeline Parallelism
di: Qi, Mengshi, et al.
Pubblicazione: (2025)

Malleus: Straggler-Resilient Hybrid Parallel Training of Large-scale Models via Malleable Data and Model Parallelization
di: Li, Haoyang, et al.
Pubblicazione: (2024)

Zeppelin: Balancing Variable-length Workloads in Data Parallel Large Model Training
di: Chen, Chang, et al.
Pubblicazione: (2025)

RailS: Load Balancing for All-to-All Communication in Distributed Mixture-of-Experts Training
di: Xu, Heng, et al.
Pubblicazione: (2025)

Lagom: Unleashing the Power of Communication and Computation Overlapping for Distributed LLM Training
di: Xu, Guanbin, et al.
Pubblicazione: (2026)

Plexus: Taming Billion-edge Graphs with 3D Parallel Full-graph GNN Training
di: Ranjan, Aditya K., et al.
Pubblicazione: (2025)

A Unified CPU-GPU Protocol for GNN Training
di: Lin, Yi-Chien, et al.
Pubblicazione: (2024)

Hiding Communication Cost in Distributed LLM Training via Micro-batch Co-execution
di: Wang, Haiquan, et al.
Pubblicazione: (2024)

Neutron particle transport 3D method of characteristic Multi GPU platform Parallel Computing
di: Zhou, Faguo, et al.
Pubblicazione: (2025)

AMDP: Asynchronous Multi-Directional Pipeline Parallelism for Large-Scale Models Training
di: Chen, Ling, et al.
Pubblicazione: (2026)

gLLM: Global Balanced Pipeline Parallelism System for Distributed LLM Serving with Token Throttling
di: Guo, Tianyu, et al.
Pubblicazione: (2025)

ZeroPP: Unleashing Exceptional Parallelism Efficiency through Tensor-Parallelism-Free Methodology
di: Tang, Ding, et al.
Pubblicazione: (2024)

Hydraulis: Balancing Large Transformer Model Training via Co-designing Parallel Strategies and Data Assignment
di: Li, Haoyang, et al.
Pubblicazione: (2024)

Para-B&B: Load-Balanced Deterministic Parallelization of Solving MIP
di: Zhang, Jinyu, et al.
Pubblicazione: (2026)

TACO: Efficient Communication Compression of Intermediate Tensors for Scalable Tensor-Parallel LLM Training
di: Liu, Man, et al.
Pubblicazione: (2026)

Nitsum: Serving Tiered LLM Requests with Adaptive Tensor Parallelism
di: Srivatsa, Vikranth, et al.
Pubblicazione: (2026)

LoongTrain: Efficient Training of Long-Sequence LLMs with Head-Context Parallelism
di: Gu, Diandian, et al.
Pubblicazione: (2024)

Bandwidth-Aware and Cost-Efficient Pipeline Parallel Scheduling in Geo-Distributed LLM Training
di: Zhang, Han, et al.
Pubblicazione: (2026)