DuoServe-MoE: Dual-Phase Expert Prefetch and Caching for LLM Inference QoS Assurance

Documenti analoghi

• SP-MoE: Speculative Decoding and Prefetching for Accelerating MoE-based Model Inference
  di: Chen, Liangkun, et al.
  Pubblicazione: (2025)
• PROBE: Co-Balancing Computation and Communication in MoE Inference via Real-Time Predictive Prefetching
  di: Zhu, Qianchao, et al.
  Pubblicazione: (2026)
• ProMoE: Fast MoE-based LLM Serving using Proactive Caching
  di: Song, Xiaoniu, et al.
  Pubblicazione: (2024)
• OD-MoE: On-Demand Expert Loading for Cacheless Edge-Distributed MoE Inference
  di: Wang, Liujianfu, et al.
  Pubblicazione: (2025)
• AgentServe: Algorithm-System Co-Design for Efficient Agentic AI Serving on a Consumer-Grade GPU
  di: Zhang, Yuning, et al.
  Pubblicazione: (2026)