ELIS: Efficient LLM Iterative Scheduling System with Response Length Predictor

Documenti analoghi

• DSDE: Dynamic Speculative Decoding with KLD Stability for Real-World Serving
  di: Yang, Mingyu, et al.
  Pubblicazione: (2025)
• FineServe: Precision-Aware KV Slab and Two-Level Scheduling for Heterogeneous Precision LLM Serving
  di: Bin, Kyungmin, et al.
  Pubblicazione: (2025)
• Justitia: Fair and Efficient Scheduling of Task-parallel LLM Agents with Selective Pampering
  di: Yang, Mingyan, et al.
  Pubblicazione: (2025)
• Tackling the Dynamicity in a Production LLM Serving System with SOTA Optimizations via Hybrid Prefill/Decode/Verify Scheduling on Efficient Meta-kernels
  di: Song, Mingcong, et al.
  Pubblicazione: (2024)
• Sustainable Carbon-Aware and Water-Efficient LLM Scheduling in Geo-Distributed Cloud Datacenters
  di: Moore, Hayden, et al.
  Pubblicazione: (2025)