AccelGen: Heterogeneous SLO-Guaranteed High-Throughput LLM Inference Serving for Diverse Applications

Documenti analoghi

• EconoServe: Maximizing Multi-Resource Utilization with SLO Guarantees in LLM Serving
  di: Shen, Haiying, et al.
  Pubblicazione: (2024)
• Mitigating KV Cache Competition to Enhance User Experience in LLM Inference
  di: Shen, Haiying, et al.
  Pubblicazione: (2025)
• FlexLLM: Token-Level Co-Serving of LLM Inference and Finetuning with SLO Guarantees
  di: Oliaro, Gabriele, et al.
  Pubblicazione: (2024)
• AdaServe: Accelerating Multi-SLO LLM Serving with SLO-Customized Speculative Decoding
  di: Li, Zikun, et al.
  Pubblicazione: (2025)
• MACE: A Hybrid LLM Serving System with Colocated SLO-aware Continuous Retraining Alignment
  di: Li, Yufei, et al.
  Pubblicazione: (2025)