MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs

Documenti analoghi

• MegaScale-Data: Scaling Dataloader for Multisource Large Foundation Model Training
  di: Zhao, Juntao, et al.
  Pubblicazione: (2025)
• MegaScale-Infer: Serving Mixture-of-Experts at Scale with Disaggregated Expert Parallelism
  di: Zhu, Ruidong, et al.
  Pubblicazione: (2025)
• MegaScale-MoE: Large-Scale Communication-Efficient Training of Mixture-of-Experts Models in Production
  di: Jin, Chao, et al.
  Pubblicazione: (2025)
• MegaScale-Omni: A Hyper-Scale, Workload-Resilient System for MultiModal LLM Training in Production
  di: Xue, Chunyu, et al.
  Pubblicazione: (2026)
• Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model
  di: Hu, Jingcheng, et al.
  Pubblicazione: (2025)