MegaScale-Data: Scaling Dataloader for Multisource Large Foundation Model Training

Documenti analoghi

• MegaScale-Omni: A Hyper-Scale, Workload-Resilient System for MultiModal LLM Training in Production
  di: Xue, Chunyu, et al.
  Pubblicazione: (2026)
• MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs
  di: Jiang, Ziheng, et al.
  Pubblicazione: (2024)
• MegaScale-MoE: Large-Scale Communication-Efficient Training of Mixture-of-Experts Models in Production
  di: Jin, Chao, et al.
  Pubblicazione: (2025)
• MegaScale-Infer: Serving Mixture-of-Experts at Scale with Disaggregated Expert Parallelism
  di: Zhu, Ruidong, et al.
  Pubblicazione: (2025)
• LLM-PQ: Serving LLM on Heterogeneous Clusters with Phase-Aware Partition and Adaptive Quantization
  di: Zhao, Juntao, et al.
  Pubblicazione: (2024)