MoE-Lens: Towards the Hardware Limit of High-Throughput MoE LLM Serving Under Resource Constraints

Documenti analoghi

• KAIROS: Stateful, Context-Aware Power-Efficient Agentic Inference Serving
  di: Yuan, Yichao, et al.
  Pubblicazione: (2026)
• D$^{2}$MoE: Dual Routing and Dynamic Scheduling for Efficient On-Device MoE-based LLM Serving
  di: Wang, Haodong, et al.
  Pubblicazione: (2025)
• MoE-Lightning: High-Throughput MoE Inference on Memory-constrained GPUs
  di: Cao, Shiyi, et al.
  Pubblicazione: (2024)
• ProMoE: Fast MoE-based LLM Serving using Proactive Caching
  di: Song, Xiaoniu, et al.
  Pubblicazione: (2024)
• MoEless: Efficient MoE LLM Serving via Serverless Computing
  di: Yu, Hanfei, et al.
  Pubblicazione: (2026)