Image de couverture de livre

TawPipe: Topology-Aware Weight Pipeline Parallelism for Accelerating Long-Context Large Models Training

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Wu, Houming, Chen, Ling
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning Artificial Intelligence Distributed, Parallel, and Cluster Computing
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

BitPipe: Bidirectional Interleaved Pipeline Parallelism for Accelerating Large Models Training
par: Wu, Houming, et autres
Publié: (2024)

AMDP: Asynchronous Multi-Directional Pipeline Parallelism for Large-Scale Models Training
par: Chen, Ling, et autres
Publié: (2026)

GraphPipe: Improving Performance and Scalability of DNN Training with Graph Pipeline Parallelism
par: Jeon, Byungsoo, et autres
Publié: (2024)

PipeOffload: Improving Scalability of Pipeline Parallelism with Memory Optimization
par: Wan, Xinyi, et autres
Publié: (2025)

InfiniPipe: Elastic Pipeline Parallelism for Efficient Variable-Length Long-Context LLM Training
par: Wang, Shiju, et autres
Publié: (2025)

Training Ultra Long Context Language Model with Fully Pipelined Distributed Transformer
par: Yao, Jinghan, et autres
Publié: (2024)

HelixPipe: Efficient Distributed Training of Long Sequence Transformers with Attention Parallel Pipeline Parallelism
par: Zhang, Geng, et autres
Publié: (2025)

Piper: Efficient Large-Scale MoE Training via Resource Modeling and Pipelined Hybrid Parallelism
par: Dash, Sajal, et autres
Publié: (2026)

NestPipe: Large-Scale Recommendation Training on 1,500+ Accelerators via Nested Pipelining
par: Jiang, Zhida, et autres
Publié: (2026)

DisagMoE: Computation-Communication overlapped MoE Training via Disaggregated AF-Pipe Parallelism
par: Zeng, Zhichen, et autres
Publié: (2026)

Zero Bubble Pipeline Parallelism
par: Qi, Penghui, et autres
Publié: (2023)

Efficient Training on Multiple Consumer GPUs with RoundPipe
par: Luo, Yibin, et autres
Publié: (2026)

EE-LLM: Large-Scale Training and Inference of Early-Exit Large Language Models with 3D Parallelism
par: Chen, Yanxi, et autres
Publié: (2023)

ResBM: Residual Bottleneck Models for Low-Bandwidth Pipeline Parallelism
par: Aboudib, Alan, et autres
Publié: (2026)

Scalable and Adaptive Parallel Training of Graph Transformer on Large Graphs
par: Lin, Jun-Liang, et autres
Publié: (2026)

Mitigating Staleness in Asynchronous Pipeline Parallelism via Basis Rotation
par: Jung, Hyunji, et autres
Publié: (2026)

OptPipe: Memory- and Scheduling-Optimized Pipeline Parallelism for LLM Training
par: Li, Hongpei, et autres
Publié: (2025)

PipeLive: Efficient Live In-place Pipeline Parallelism Reconfiguration for Dynamic LLM Serving
par: Bai, Xu, et autres
Publié: (2026)

Context Parallelism for Scalable Million-Token Inference
par: Yang, Amy, et autres
Publié: (2024)

SkipPipe: Partial and Reordered Pipelining Framework for Training LLMs in Heterogeneous Networks
par: Blagoev, Nikolay, et autres
Publié: (2025)

Hermes: Memory-Efficient Pipeline Inference for Large Models on Edge Devices
par: Han, Xueyuan, et autres
Publié: (2024)

DCP: Addressing Input Dynamism In Long-Context Training via Dynamic Context Parallelism
par: Jiang, Chenyu, et autres
Publié: (2025)

Acceleration for Deep Reinforcement Learning using Parallel and Distributed Computing: A Survey
par: Liu, Zhihong, et autres
Publié: (2024)

SPD: Sync-Point Drop for Efficient Tensor Parallelism of Large Language Models
par: Kim, Han-Byul, et autres
Publié: (2025)

PipeFill: Using GPUs During Bubbles in Pipeline-parallel LLM Training
par: Arfeen, Daiyaan, et autres
Publié: (2024)

A-3PO: Accelerating Asynchronous LLM Training with Staleness-aware Proximal Policy Approximation
par: Li, Xiaocan, et autres
Publié: (2025)

A Parallel Alternative for Energy-Efficient Neural Network Training and Inferencing
par: Seal, Sudip K., et autres
Publié: (2025)

Topology-Aware Knowledge Propagation in Decentralized Learning
par: Sakarvadia, Mansi, et autres
Publié: (2025)

TAPAS: Fast and Automatic Derivation of Tensor Parallel Strategies for Large Neural Networks
par: Shi, Ziji, et autres
Publié: (2023)

FlashMLA-ETAP: Efficient Transpose Attention Pipeline for Accelerating MLA Inference on NVIDIA H20 GPUs
par: Dege, Pengcuo, et autres
Publié: (2025)

A Fast and Flat Federated Learning Method via Weighted Momentum and Sharpness-Aware Minimization
par: Li, Tianle, et autres
Publié: (2025)

FlexSP: Accelerating Large Language Model Training via Flexible Sequence Parallelism
par: Wang, Yujie, et autres
Publié: (2024)

PIPO: Pipelined Offloading for Efficient Inference on Consumer Devices
par: Liu, Yangyijian, et autres
Publié: (2025)

A Tabular Schedule Abstraction for Communication-Aware Evaluation of Pipeline-Parallel LLM Training
par: Barley, Daniel, et autres
Publié: (2026)

LoongServe: Efficiently Serving Long-Context Large Language Models with Elastic Sequence Parallelism
par: Wu, Bingyang, et autres
Publié: (2024)

Plexus: Taming Billion-edge Graphs with 3D Parallel Full-graph GNN Training
par: Ranjan, Aditya K., et autres
Publié: (2025)

Communication-free Sampling and 4D Hybrid Parallelism for Scalable Mini-batch GNN Training
par: Wei, Cunyang, et autres
Publié: (2026)

Accelerating Long-Tail Generation in Synchronous RLHF Training via Adaptive Tensor Parallelism
par: Zhao, Long, et autres
Publié: (2026)

AdaPtis: Reducing Pipeline Bubbles with Adaptive Pipeline Parallelism on Heterogeneous Models
par: Guo, Jihu, et autres
Publié: (2025)

Accelerating Privacy-Preserving Federated Learning in Large-Scale LEO Satellite Systems
par: Guo, Binquan, et autres
Publié: (2025)