Efficient LLM Serving on Hybrid Real-time and Best-effort Requests

Documenti analoghi

• LLM-PQ: Serving LLM on Heterogeneous Clusters with Phase-Aware Partition and Adaptive Quantization
  di: Zhao, Juntao, et al.
  Pubblicazione: (2024)
• Sandwich: Joint Configuration Search and Hot-Switching for Efficient CPU LLM Serving
  di: Zhao, Juntao, et al.
  Pubblicazione: (2025)
• Ascendra: Dynamic Request Prioritization for Efficient LLM Serving
  di: Ikram, Azam, et al.
  Pubblicazione: (2025)
• Learned Best-Effort LLM Serving
  di: Jha, Siddharth, et al.
  Pubblicazione: (2024)
• Combining the Best of Both Worlds: A Method for Hybrid NMT and LLM Translation
  di: Wu, Zhanglin, et al.
  Pubblicazione: (2025)