Prism: Unleashing GPU Sharing for Cost-Efficient Multi-LLM Serving

Documenti analoghi

• ConServe: Fine-Grained GPU Harvesting for LLM Online and Offline Co-Serving
  di: Qiao, Yifan, et al.
  Pubblicazione: (2024)
• Unleashing the Power of Preemptive Priority-based Scheduling for Real-Time GPU Tasks
  di: Wang, Yidi, et al.
  Pubblicazione: (2024)
• The Energy Cost of Execution-Idle in GPU Clusters
  di: Lei, Yiran, et al.
  Pubblicazione: (2026)
• Towards Efficient and Practical GPU Multitasking in the Era of LLM
  di: Xing, Jiarong, et al.
  Pubblicazione: (2025)
• FastDecode: High-Throughput GPU-Efficient LLM Serving using Heterogeneous Pipelines
  di: He, Jiaao, et al.
  Pubblicazione: (2024)