Foundry: Template-Based CUDA Graph Context Materialization for Fast LLM Serving Cold Start

Documenti analoghi

• HeterMoE: Efficient Training of Mixture-of-Experts Models on Heterogeneous GPUs
  di: Wu, Yongji, et al.
  Pubblicazione: (2025)
• RLBoost: Harvesting Preemptible Resources for Cost-Efficient Reinforcement Learning on LLMs
  di: Wu, Yongji, et al.
  Pubblicazione: (2025)
• Lazarus: Resilient and Elastic Training of Mixture-of-Experts Models
  di: Wu, Yongji, et al.
  Pubblicazione: (2024)
• HydraServe: Minimizing Cold Start Latency for Serverless LLM Serving in Public Clouds
  di: Lou, Chiheng, et al.
  Pubblicazione: (2025)
• MuxServe: Flexible Spatial-Temporal Multiplexing for Multiple LLM Serving
  di: Duan, Jiangfei, et al.
  Pubblicazione: (2024)