AMSP: Reducing Communication Overhead of ZeRO for Efficient LLM Training

Documenti analoghi

• LoongTrain: Efficient Training of Long-Sequence LLMs with Head-Context Parallelism
  di: Gu, Diandian, et al.
  Pubblicazione: (2024)
• AsyncHZP: Hierarchical ZeRO Parallelism with Asynchronous Scheduling for Scalable LLM Training
  di: Bai, Huawei, et al.
  Pubblicazione: (2025)
• InternEvo: Efficient Long-sequence Large Language Model Training via Hybrid Parallelism and Redundant Sharding
  di: Chen, Qiaoling, et al.
  Pubblicazione: (2024)
• SPPO:Efficient Long-sequence LLM Training via Adaptive Sequence Pipeline Parallel Offloading
  di: Chen, Qiaoling, et al.
  Pubblicazione: (2025)
• CONCUR: High-Throughput Agentic Batch Inference of LLM via Congestion-Based Concurrency Control
  di: Chen, Qiaoling, et al.
  Pubblicazione: (2026)