Online Scheduling for LLM Inference with KV Cache Constraints

Documenti analoghi

• A Queueing-Theoretic Framework for Stability Analysis of LLM Inference with KV Cache Memory Constraints
  di: Nie, Chengyi, et al.
  Pubblicazione: (2026)
• When Should you Offer an Upgrade: Online Upgrading Mechanisms for Resource Allocation
  di: Jaillet, Patrick, et al.
  Pubblicazione: (2024)
• Grace Period is All You Need: Individual Fairness without Revenue Loss in Revenue Management
  di: Jaillet, Patrick, et al.
  Pubblicazione: (2024)
• Adaptively Robust LLM Inference Optimization under Prediction Uncertainty
  di: Chen, Zixi, et al.
  Pubblicazione: (2025)
• Online Resource Allocation with Convex-set Machine-Learned Advice
  di: Golrezaei, Negin, et al.
  Pubblicazione: (2023)