SlimPipe: Memory-Thrifty and Efficient Pipeline Parallelism for Long-Context LLM Training

Documenti analoghi

• SlimPack: Fine-Grained Asymmetric Packing for Balanced and Efficient Variable-Length LLM Training
  di: Liu, Yuliang, et al.
  Pubblicazione: (2025)
• InfiniPipe: Elastic Pipeline Parallelism for Efficient Variable-Length Long-Context LLM Training
  di: Wang, Shiju, et al.
  Pubblicazione: (2025)
• OptPipe: Memory- and Scheduling-Optimized Pipeline Parallelism for LLM Training
  di: Li, Hongpei, et al.
  Pubblicazione: (2025)
• TawPipe: Topology-Aware Weight Pipeline Parallelism for Accelerating Long-Context Large Models Training
  di: Wu, Houming, et al.
  Pubblicazione: (2025)
• HelixPipe: Efficient Distributed Training of Long Sequence Transformers with Attention Parallel Pipeline Parallelism
  di: Zhang, Geng, et al.
  Pubblicazione: (2025)