Cost-Efficient LLM Serving in the Cloud: VM Selection with KV Cache Offloading

Documenti analoghi

• DeepVM: Integrating Spot and On-Demand VMs for Cost-Efficient Deep Learning Clusters in the Cloud
  di: Kim, Yoochan, et al.
  Pubblicazione: (2024)
• TraCT: Disaggregated LLM Serving with CXL Shared Memory KV Cache at Rack-Scale
  di: Yoon, Dongha, et al.
  Pubblicazione: (2025)
• DUAL-BLADE: Dual-Path NVMe-Direct KV-Cache Offloading for Edge LLM Inference
  di: Jeong, Bodon, et al.
  Pubblicazione: (2026)
• Understanding Bottlenecks for Efficiently Serving LLM Inference With KV Offloading
  di: Meng, William, et al.
  Pubblicazione: (2025)
• CacheFlow: Efficient LLM Serving with 3D-Parallel KV Cache Restoration
  di: Nian, Sean, et al.
  Pubblicazione: (2026)