SpecPipe: Accelerating Pipeline Parallelism-based LLM Inference with Speculative Decoding

Documenti analoghi

• ParallelSpec: Parallel Drafter for Efficient Speculative Decoding
  di: Xiao, Zilin, et al.
  Pubblicazione: (2024)
• PipeInfer: Accelerating LLM Inference using Asynchronous Pipelined Speculation
  di: Butler, Branden, et al.
  Pubblicazione: (2024)
• DistrAttention: An Efficient and Flexible Self-Attention Mechanism on Modern GPUs
  di: Jin, Haolin, et al.
  Pubblicazione: (2025)
• CAS-Spec: Cascade Adaptive Self-Speculative Decoding for On-the-Fly Lossless Inference Acceleration of LLMs
  di: Ning, Zhiyuan, et al.
  Pubblicazione: (2025)
• SwiftSpec: Ultra-Low Latency LLM Decoding by Scaling Asynchronous Speculative Decoding
  di: Zhang, Ziyi, et al.
  Pubblicazione: (2025)