Klotski: Efficient Mixture-of-Expert Inference via Expert-Aware Multi-Batch Pipeline

Documenti analoghi

• Fate: Fast Edge Inference of Mixture-of-Experts Models via Cross-Layer Gate
  di: Fang, Zhiyuan, et al.
  Pubblicazione: (2025)
• DyMoE: Dynamic Expert Orchestration with Mixed-Precision Quantization for Efficient MoE Inference on Edge
  di: Huang, Yuegui, et al.
  Pubblicazione: (2026)
• Krul: Efficient State Restoration for Multi-turn Conversations with Dynamic Cross-layer KV Sharing
  di: Wen, Junyi, et al.
  Pubblicazione: (2025)
• DiEP: Adaptive Mixture-of-Experts Compression through Differentiable Expert Pruning
  di: Bai, Sikai, et al.
  Pubblicazione: (2025)
• Training and Serving System of Foundation Models: A Comprehensive Survey
  di: Zhou, Jiahang, et al.
  Pubblicazione: (2024)