PipeInfer: Accelerating LLM Inference using Asynchronous Pipelined Speculation

Documenti analoghi

• AMUSD: Asynchronous Multi-Device Speculative Decoding for LLM Acceleration
  di: McDanel, Bradley
  Pubblicazione: (2024)
• SuperSFL: Resource-Heterogeneous Federated Split Learning with Weight-Sharing Super-Networks
  di: Asif, Abdullah Al, et al.
  Pubblicazione: (2026)
• SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification
  di: Miao, Xupeng, et al.
  Pubblicazione: (2023)
• SwiftSpec: Ultra-Low Latency LLM Decoding by Scaling Asynchronous Speculative Decoding
  di: Zhang, Ziyi, et al.
  Pubblicazione: (2025)
• Collaborative Speculative Inference for Efficient LLM Inference Serving
  di: Gao, Luyao, et al.
  Pubblicazione: (2025)