A Queueing-Theoretic Framework for Stability Analysis of LLM Inference with KV Cache Memory Constraints

Documenti analoghi

• Online Scheduling for LLM Inference with KV Cache Constraints
  di: Jaillet, Patrick, et al.
  Pubblicazione: (2025)
• Adaptively Robust LLM Inference Optimization under Prediction Uncertainty
  di: Chen, Zixi, et al.
  Pubblicazione: (2025)
• A Queueing-Theoretic Framework for Dynamic Attack Surfaces: Data-Integrated Risk Analysis and Adaptive Defense
  di: Yun, Jihyeon, et al.
  Pubblicazione: (2026)
• LLM Serving Optimization with Variable Prefill and Decode Lengths
  di: Wang, Meixuan, et al.
  Pubblicazione: (2025)
• Optimizing LLM Inference: Fluid-Guided Online Scheduling with Memory Constraints
  di: Ao, Ruicheng, et al.
  Pubblicazione: (2025)