MoE-Lightning: High-Throughput MoE Inference on Memory-constrained GPUs

Documenti analoghi

• OD-MoE: On-Demand Expert Loading for Cacheless Edge-Distributed MoE Inference
  di: Wang, Liujianfu, et al.
  Pubblicazione: (2025)
• MoE-Gen: High-Throughput MoE Inference on a Single GPU with Module-Based Batching
  di: Xu, Tairan, et al.
  Pubblicazione: (2025)
• SP-MoE: Speculative Decoding and Prefetching for Accelerating MoE-based Model Inference
  di: Chen, Liangkun, et al.
  Pubblicazione: (2025)
• MoE-Compression: How the Compression Error of Experts Affects the Inference Accuracy of MoE Model?
  di: Ma, Songkai, et al.
  Pubblicazione: (2025)
• GRACE-MoE: Grouping and Replication with Locality-Aware Routing for Efficient Distributed MoE Inference
  di: Han, Yu, et al.
  Pubblicazione: (2025)