Scalable Pretraining of Large Mixture of Experts Language Models on Aurora Super Computer

Documenti analoghi

• Comet: Fine-grained Computation-communication Overlapping for Mixture-of-Experts
  di: Zhang, Shulai, et al.
  Pubblicazione: (2025)
• Harnessing Deep Learning and HPC Kernels via High-Level Loop and Tensor Abstractions on CPU Architectures
  di: Georganas, Evangelos, et al.
  Pubblicazione: (2023)
• Exploiting Inter-Layer Expert Affinity for Accelerating Mixture-of-Experts Model Inference
  di: Yao, Jinghan, et al.
  Pubblicazione: (2024)
• WDMoE: Wireless Distributed Mixture of Experts for Large Language Models
  di: Xue, Nan, et al.
  Pubblicazione: (2024)
• Scaling Performance of Large Language Model Pretraining
  di: Interrante-Grant, Alexander, et al.
  Pubblicazione: (2025)