Image de couverture de livre

FreeRide: Harvesting Bubbles in Pipeline Parallelism

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Zhang, Jiashu, Pan, Zihan, Molly, Xu, Daudjee, Khuzaima, Liu, Sihang
Format:	Preprint
Publié:	2024
Sujets:	Distributed, Parallel, and Cluster Computing Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Cephalo: Harnessing Heterogeneous GPU Clusters for Training Transformer Models
par: Guo, Runsheng Benson, et autres
Publié: (2024)

Zorse: Optimizing LLM Training Efficiency on Heterogeneous GPU Clusters
par: Guo, Runsheng Benson, et autres
Publié: (2025)

AdaPtis: Reducing Pipeline Bubbles with Adaptive Pipeline Parallelism on Heterogeneous Models
par: Guo, Jihu, et autres
Publié: (2025)

Zero Bubble Pipeline Parallelism
par: Qi, Penghui, et autres
Publié: (2023)

TimelyFreeze: Adaptive Parameter Freezing Mechanism for Pipeline Parallelism
par: Cho, Seonghye, et autres
Publié: (2026)

InfiniPipe: Elastic Pipeline Parallelism for Efficient Variable-Length Long-Context LLM Training
par: Wang, Shiju, et autres
Publié: (2025)

OptPipe: Memory- and Scheduling-Optimized Pipeline Parallelism for LLM Training
par: Li, Hongpei, et autres
Publié: (2025)

FlowSpec: Continuous Pipelined Speculative Decoding for Efficient Distributed LLM Inference
par: Liu, Xing, et autres
Publié: (2025)

Opara: Exploiting Operator Parallelism for Expediting DNN Inference on GPUs
par: Chen, Aodong, et autres
Publié: (2023)

Para-B&B: Load-Balanced Deterministic Parallelization of Solving MIP
par: Zhang, Jinyu, et autres
Publié: (2026)

xDiT: an Inference Engine for Diffusion Transformers (DiTs) with Massive Parallelism
par: Fang, Jiarui, et autres
Publié: (2024)

Efficient MoE Inference with Fine-Grained Scheduling of Disaggregated Expert Parallelism
par: Pan, Xinglin, et autres
Publié: (2025)

PipeOffload: Improving Scalability of Pipeline Parallelism with Memory Optimization
par: Wan, Xinyi, et autres
Publié: (2025)

Mitigating Staleness in Asynchronous Pipeline Parallelism via Basis Rotation
par: Jung, Hyunji, et autres
Publié: (2026)

SimpleFSDP: Simpler Fully Sharded Data Parallel with torch.compile
par: Zhang, Ruisi, et autres
Publié: (2024)

Automated Planning for Optimal Data Pipeline Instantiation
par: Amado, Leonardo Rosa, et autres
Publié: (2025)

ResBM: Residual Bottleneck Models for Low-Bandwidth Pipeline Parallelism
par: Aboudib, Alan, et autres
Publié: (2026)

Research on Model Parallelism and Data Parallelism Optimization Methods in Large Language Model-Based Recommendation Systems
par: Yang, Haowei, et autres
Publié: (2025)

GraphPipe: Improving Performance and Scalability of DNN Training with Graph Pipeline Parallelism
par: Jeon, Byungsoo, et autres
Publié: (2024)

BitPipe: Bidirectional Interleaved Pipeline Parallelism for Accelerating Large Models Training
par: Wu, Houming, et autres
Publié: (2024)

Accelerating End-Cloud Collaborative Inference via Near Bubble-free Pipeline Optimization
par: Gao, Luyao, et autres
Publié: (2024)

Scepsy: Serving Agentic Workflows Using Aggregate LLM Pipelines
par: Wagenländer, Marcel, et autres
Publié: (2026)

TACO: Efficient Communication Compression of Intermediate Tensors for Scalable Tensor-Parallel LLM Training
par: Liu, Man, et autres
Publié: (2026)

Understand and Accelerate Memory Processing Pipeline for Large Language Model Inference
par: He, Zifan, et autres
Publié: (2026)

Artificial Intelligence for Cost-Aware Resource Prediction in Big Data Pipelines
par: Goyal, Harshit
Publié: (2025)

DIP: Efficient Large Multimodal Model Training with Dynamic Interleaved Pipeline
par: Xue, Zhenliang, et autres
Publié: (2025)

SpecBranch: Speculative Decoding via Hybrid Drafting and Rollback-Aware Branch Parallelism
par: Shen, Yuhao, et autres
Publié: (2025)

SPECTRE: Hybrid Ordinary-Parallel Speculative Serving for Resource-Efficient LLM Inference
par: Xie, Jincheng, et autres
Publié: (2026)

WORKSWORLD: A Domain for Integrated Numeric Planning and Scheduling of Distributed Pipelined Workflows
par: Paul, Taylor, et autres
Publié: (2026)

Can Large Language Models Write Parallel Code?
par: Nichols, Daniel, et autres
Publié: (2024)

Rethinking Dynamic Networks and Heterogeneous Computing with Automatic Parallelization
par: Wu, Ruilong, et autres
Publié: (2025)

Collaborative Split Federated Learning with Parallel Training and Aggregation
par: Papageorgiou, Yiannis, et autres
Publié: (2025)

Vec-LUT: Vector Table Lookup for Parallel Ultra-Low-Bit LLM Inference on Edge Devices
par: Li, Xiangyu, et autres
Publié: (2025)

DWDP: Distributed Weight Data Parallelism for High-Performance LLM Inference on NVL72
par: Li, Wanqian, et autres
Publié: (2026)

Bandwidth-Aware and Cost-Efficient Pipeline Parallel Scheduling in Geo-Distributed LLM Training
par: Zhang, Han, et autres
Publié: (2026)

Balancing Pipeline Parallelism with Vocabulary Parallelism
par: Yeung, Man Tsung, et autres
Publié: (2024)

Synergistic Tensor and Pipeline Parallelism
par: Qi, Mengshi, et autres
Publié: (2025)

HPC-Coder: Modeling Parallel Programs using Large Language Models
par: Nichols, Daniel, et autres
Publié: (2023)

Dora: QoE-Aware Hybrid Parallelism for Distributed Edge AI
par: Jin, Jianli, et autres
Publié: (2025)

TawPipe: Topology-Aware Weight Pipeline Parallelism for Accelerating Long-Context Large Models Training
par: Wu, Houming, et autres
Publié: (2025)