Speculative Decoding in Decentralized LLM Inference: Turning Communication Latency into Computation Throughput

Documenti analoghi

• Parallax: Efficient LLM Inference Service over Decentralized Environment
  di: Tong, Chris, et al.
  Pubblicazione: (2025)
• Lattica: A Decentralized Cross-NAT Communication Framework for Scalable AI Inference and Training
  di: Yang, Ween, et al.
  Pubblicazione: (2025)
• Accelerating Mixture-of-Experts Inference by Hiding Offloading Latency with Speculative Decoding
  di: Wang, Zhibin, et al.
  Pubblicazione: (2025)
• Shift Parallelism: Low-Latency, High-Throughput LLM Inference for Dynamic Workloads
  di: Hidayetoglu, Mert, et al.
  Pubblicazione: (2025)
• Accelerating OpenPangu Inference on NPU via Speculative Decoding
  di: Dai, Yuntao, et al.
  Pubblicazione: (2026)