EFIM: Efficient Serving of LLMs for Infilling Tasks with Improved KV Cache Reuse

Documenti analoghi

• DynamicKV: Task-Aware Adaptive KV Cache Compression for Long Context LLMs
  di: Zhou, Xiabin, et al.
  Pubblicazione: (2024)
• KVShare: An LLM Service System with Efficient and Effective Multi-Tenant KV Cache Reuse
  di: Yang, Huan, et al.
  Pubblicazione: (2025)
• KVLink: Accelerating Large Language Models via Efficient KV Cache Reuse
  di: Yang, Jingbo, et al.
  Pubblicazione: (2025)
• PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling
  di: Cai, Zefan, et al.
  Pubblicazione: (2024)
• WindowKV: Task-Adaptive Group-Wise KV Cache Window Selection for Efficient LLM Inference
  di: Zuo, Youhui, et al.
  Pubblicazione: (2025)