GhostServe: A Lightweight Checkpointing System in the Shadow for Fault-Tolerant LLM Serving

Documenti analoghi

• EdgeProfiler: A Fast Profiling Framework for Lightweight LLMs on Edge Using Analytical Model
  di: Pinnock, Alyssa, et al.
  Pubblicazione: (2025)
• Prompt-Aware Scheduling for Low-Latency LLM Serving
  di: Tao, Yiheng, et al.
  Pubblicazione: (2025)
• CoServe: Efficient Collaboration-of-Experts (CoE) Model Inference with Limited Memory
  di: Suo, Jiashun, et al.
  Pubblicazione: (2025)
• Prism: Unleashing GPU Sharing for Cost-Efficient Multi-LLM Serving
  di: Yu, Shan, et al.
  Pubblicazione: (2025)
• Fault-Tolerant Hybrid-Parallel Training at Scale with Reliable and Efficient In-memory Checkpointing
  di: Wang, Yuxin, et al.
  Pubblicazione: (2023)