veScale-FSDP: Flexible and High-Performance FSDP at Scale

Documenti analoghi

• veScale: Consistent and Efficient Tensor Programming with Eager-Mode SPMD
  di: Li, Youjie, et al.
  Pubblicazione: (2025)
• SimpleFSDP: Simpler Fully Sharded Data Parallel with torch.compile
  di: Zhang, Ruisi, et al.
  Pubblicazione: (2024)
• Performance Characterization of Distributed Deep Learning Strategies: A Quantitative Evaluation of DDP, FSDP, and Parameter Server Architectures on GPU Clusters
  di: Ovi, Md Sultanul Islam
  Pubblicazione: (2025)
• VeOmni: Scaling Any Modality Model Training with Model-Centric Distributed Recipe Zoo
  di: Ma, Qianli, et al.
  Pubblicazione: (2025)
• MegaScale-MoE: Large-Scale Communication-Efficient Training of Mixture-of-Experts Models in Production
  di: Jin, Chao, et al.
  Pubblicazione: (2025)