Folding Tensor and Sequence Parallelism for Memory-Efficient Transformer Training & Inference

Documenti analoghi

• Amoeba: Runtime Tensor Parallel Transformation for LLM Inference Services
  di: Chen, Haoyu, et al.
  Pubblicazione: (2025)
• Training Foundation Models on a Full-Stack AMD Platform: Compute, Networking, and System Design
  di: Anthony, Quentin, et al.
  Pubblicazione: (2025)
• DeInfer: Efficient Parallel Inferencing for Decomposed Large Language Models
  di: Huang, You-Liang, et al.
  Pubblicazione: (2026)
• StarTrail: Concentric Ring Sequence Parallelism for Efficient Near-Infinite-Context Transformer Model Training
  di: Liu, Ziming, et al.
  Pubblicazione: (2024)
• LoongTrain: Efficient Training of Long-Sequence LLMs with Head-Context Parallelism
  di: Gu, Diandian, et al.
  Pubblicazione: (2024)