DISTFLASHATTN: Distributed Memory-efficient Attention for Long-context LLMs Training

Documenti analoghi

• Efficient Long-context Language Model Training by Core Attention Disaggregation
  di: Zhuang, Yonghao, et al.
  Pubblicazione: (2025)
• Unleashing Scalable Context Parallelism for Foundation Models Pre-Training via FCP
  di: Zhao, Yilong, et al.
  Pubblicazione: (2026)
• Pie: Pooling CPU Memory for LLM Inference
  di: Xu, Yi, et al.
  Pubblicazione: (2024)
• Jenga: Effective Memory Management for Serving LLM with Heterogeneity
  di: Zhang, Chen, et al.
  Pubblicazione: (2025)
• On Optimizing the Communication of Model Parallelism
  di: Zhuang, Yonghao, et al.
  Pubblicazione: (2022)