Mesh-Attention: A New Communication-Efficient Distributed Attention with Improved Data Locality

Documenti analoghi

• ATTENTION2D: Communication Efficient Distributed Self-Attention Mechanism
  di: Elango, Venmugil
  Pubblicazione: (2025)
• EDiT: A Local-SGD-Based Efficient Distributed Training Method for Large Language Models
  di: Cheng, Jialiang, et al.
  Pubblicazione: (2024)
• MegaScale-Data: Scaling Dataloader for Multisource Large Foundation Model Training
  di: Zhao, Juntao, et al.
  Pubblicazione: (2025)
• OpenTinker: Separating Concerns in Agentic Reinforcement Learning
  di: Zhu, Siqi, et al.
  Pubblicazione: (2026)
• FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Serving
  di: Ye, Zihao, et al.
  Pubblicazione: (2025)