DistTrain: Addressing Model and Data Heterogeneity with Disaggregated Training for Multimodal Large Language Models

Documenti analoghi

• DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving
  di: Zhong, Yinmin, et al.
  Pubblicazione: (2024)
• StreamRL: Scalable, Heterogeneous, and Elastic RL for LLMs with Disaggregated Stream Generation
  di: Zhong, Yinmin, et al.
  Pubblicazione: (2025)
• DIP: Efficient Large Multimodal Model Training with Dynamic Interleaved Pipeline
  di: Xue, Zhenliang, et al.
  Pubblicazione: (2025)
• Optimizing RLHF Training for Large Language Models with Stage Fusion
  di: Zhong, Yinmin, et al.
  Pubblicazione: (2024)
• Efficient Long-context Language Model Training by Core Attention Disaggregation
  di: Zhuang, Yonghao, et al.
  Pubblicazione: (2025)