Semi-Clairvoyant Scheduling of Speculative Decoding Requests to Minimize LLM Inference Latency

Documenti analoghi

• Accelerating LLM Inference with Lossless Speculative Decoding Algorithms for Heterogeneous Vocabularies
  di: Timor, Nadav, et al.
  Pubblicazione: (2025)
• Speculative Decoding with CTC-based Draft Model for LLM Inference Acceleration
  di: Wen, Zhuofan, et al.
  Pubblicazione: (2024)
• Balancing Coverage and Draft Latency in Vocabulary Trimming for Faster Speculative Decoding
  di: Shoham, Ofir Ben
  Pubblicazione: (2026)
• Clover-2: Accurate Inference for Regressive Lightweight Speculative Decoding
  di: Xiao, Bin, et al.
  Pubblicazione: (2024)
• Traversal Verification for Speculative Tree Decoding
  di: Weng, Yepeng, et al.
  Pubblicazione: (2025)