InfiniPipe: Elastic Pipeline Parallelism for Efficient Variable-Length Long-Context LLM Training

Documenti analoghi

• FlexSP: Accelerating Large Language Model Training via Flexible Sequence Parallelism
  di: Wang, Yujie, et al.
  Pubblicazione: (2024)
• Malleus: Straggler-Resilient Hybrid Parallel Training of Large-scale Models via Malleable Data and Model Parallelization
  di: Li, Haoyang, et al.
  Pubblicazione: (2024)
• JanusPipe: Efficient Pipeline Parallel Training for Machine Learning Interatomic Potentials
  di: Wang, Hongyu, et al.
  Pubblicazione: (2026)
• HelixPipe: Efficient Distributed Training of Long Sequence Transformers with Attention Parallel Pipeline Parallelism
  di: Zhang, Geng, et al.
  Pubblicazione: (2025)
• TD-Pipe: Temporally-Disaggregated Pipeline Parallelism Architecture for High-Throughput LLM Inference
  di: Zhang, Hongbin, et al.
  Pubblicazione: (2025)