Pro-Prophet: A Systematic Load Balancing Method for Efficient Parallel Training of Large-scale MoE Models

Documenti analoghi

• Oases: Efficient Large-Scale Model Training on Commodity Servers via Overlapped and Automated Tensor Model Parallelism
  di: Li, Shengwei, et al.
  Pubblicazione: (2023)
• Fine-grained MoE Load Balancing with Linear Programming
  di: Zhao, Chenqi, et al.
  Pubblicazione: (2025)
• FEPLB: Exploiting Copy Engines for Nearly Free MoE Load Balancing in Distributed Training
  di: Qi, Shuyao, et al.
  Pubblicazione: (2026)
• UniEP: Unified Expert-Parallel MoE MegaKernel for LLM Training
  di: Zheng, Size, et al.
  Pubblicazione: (2026)
• ReaLB: Real-Time Load Balancing for Multimodal MoE Inference
  di: Wang, Yingping, et al.
  Pubblicazione: (2026)