MoE-Gen: High-Throughput MoE Inference on a Single GPU with Module-Based Batching

Documenti analoghi

• MoE-Lightning: High-Throughput MoE Inference on Memory-constrained GPUs
  di: Cao, Shiyi, et al.
  Pubblicazione: (2024)
• HarMoEny: Efficient Multi-GPU Inference of MoE Models
  di: Doucet, Zachary, et al.
  Pubblicazione: (2025)
• Staleness-Centric Optimizations for Parallel Diffusion MoE Inference
  di: Luo, Jiajun, et al.
  Pubblicazione: (2024)
• MoE-CAP: Benchmarking Cost, Accuracy and Performance of Sparse Mixture-of-Experts Systems
  di: Jiang, Yinsicheng, et al.
  Pubblicazione: (2025)
• MoE-CAP: Benchmarking Cost, Accuracy and Performance of Sparse Mixture-of-Experts Systems
  di: Jiang, Yinsicheng, et al.
  Pubblicazione: (2024)