Piper: Efficient Large-Scale MoE Training via Resource Modeling and Pipelined Hybrid Parallelism

Documenti analoghi

• X-MoE: Enabling Scalable Training for Emerging Mixture-of-Experts Architectures on HPC Platforms
  di: Yuan, Yueming, et al.
  Pubblicazione: (2025)
• A Parallel Alternative for Energy-Efficient Neural Network Training and Inferencing
  di: Seal, Sudip K., et al.
  Pubblicazione: (2025)
• DisagMoE: Computation-Communication overlapped MoE Training via Disaggregated AF-Pipe Parallelism
  di: Zeng, Zhichen, et al.
  Pubblicazione: (2026)
• MoE Parallel Folding: Heterogeneous Parallelism Mappings for Efficient Large-Scale MoE Model Training with Megatron Core
  di: Liu, Dennis, et al.
  Pubblicazione: (2025)
• Semantic Parallelism: Redefining Efficient MoE Inference via Model-Data Co-Scheduling
  di: Li, Yan, et al.
  Pubblicazione: (2025)