PCR: A Prefetch-Enhanced Cache Reuse System for Low-Latency RAG Serving

Documenti analoghi

• MoE-SpeQ: Speculative Quantized Decoding with Proactive Expert Prefetching and Offloading for Mixture-of-Experts
  di: Wang, Wenfeng, et al.
  Pubblicazione: (2025)
• DuoServe-MoE: Dual-Phase Expert Prefetch and Caching for LLM Inference QoS Assurance
  di: Zhang, Yuning, et al.
  Pubblicazione: (2025)
• HOBBIT: A Mixed Precision Expert Offloading System for Fast MoE Inference
  di: Tang, Peng, et al.
  Pubblicazione: (2024)
• Adaptive KV Cache Reuse for Fast Long-Context LLM Serving
  di: li, Fei, et al.
  Pubblicazione: (2026)
• Kairos: Low-latency Multi-Agent Serving with Shared LLMs and Excessive Loads in the Public Cloud
  di: Chen, Jinyuan, et al.
  Pubblicazione: (2025)