TokenScale: Timely and Accurate Autoscaling for Disaggregated LLM Serving with Token Velocity

Documenti analoghi

• FASER: Fine-Grained Phase Management for Speculative Decoding in Dynamic LLM Serving
  di: Chen, Wenyan, et al.
  Pubblicazione: (2026)
• Taming the Chaos: Coordinated Autoscaling for Heterogeneous and Disaggregated LLM Inference
  di: Li, Rongzhi, et al.
  Pubblicazione: (2025)
• The High Cost of Keeping Warm: Characterizing Overhead in Serverless Autoscaling Policies
  di: Kondrashov, Leonid, et al.
  Pubblicazione: (2025)
• DualScale: Energy-Efficient Disaggregated LLM Serving via Phase-Aware Placement and DVFS
  di: Basit, Omar, et al.
  Pubblicazione: (2026)
• HydraInfer: Hybrid Disaggregated Scheduling for Multimodal Large Language Model Serving
  di: Dong, Xianzhe, et al.
  Pubblicazione: (2025)