HOBBIT: A Mixed Precision Expert Offloading System for Fast MoE Inference

Documenti analoghi

• MoE-SpeQ: Speculative Quantized Decoding with Proactive Expert Prefetching and Offloading for Mixture-of-Experts
  di: Wang, Wenfeng, et al.
  Pubblicazione: (2025)
• A Survey on Inference Optimization Techniques for Mixture of Experts Models
  di: Liu, Jiacheng, et al.
  Pubblicazione: (2024)
• OD-MoE: On-Demand Expert Loading for Cacheless Edge-Distributed MoE Inference
  di: Wang, Liujianfu, et al.
  Pubblicazione: (2025)
• Janus: Disaggregating Attention and Experts for Scalable MoE Inference
  di: Zhang, Zhexiang, et al.
  Pubblicazione: (2025)
• Surviving Partial Rank Failures in Wide Expert-Parallel MoE Inference
  di: Sun, Xun, et al.
  Pubblicazione: (2026)