MoE-SpeQ: Speculative Quantized Decoding with Proactive Expert Prefetching and Offloading for Mixture-of-Experts

Documenti analoghi

• HOBBIT: A Mixed Precision Expert Offloading System for Fast MoE Inference
  di: Tang, Peng, et al.
  Pubblicazione: (2024)
• SP-MoE: Speculative Decoding and Prefetching for Accelerating MoE-based Model Inference
  di: Chen, Liangkun, et al.
  Pubblicazione: (2025)
• Accelerating Mixture-of-Experts Inference by Hiding Offloading Latency with Speculative Decoding
  di: Wang, Zhibin, et al.
  Pubblicazione: (2025)
• A Survey on Inference Optimization Techniques for Mixture of Experts Models
  di: Liu, Jiacheng, et al.
  Pubblicazione: (2024)
• PCR: A Prefetch-Enhanced Cache Reuse System for Low-Latency RAG Serving
  di: Wang, Wenfeng, et al.
  Pubblicazione: (2026)