Rethinking Key-Value Cache Compression Techniques for Large Language Model Serving

Documenti analoghi

• WKVQuant: Quantizing Weight and Key/Value Cache for Large Language Models Gains More
  di: Yue, Yuxuan, et al.
  Pubblicazione: (2024)
• ReasonCache: Accelerating Large Reasoning Model Serving through KV Cache Sharing
  di: Chen, Kaiwen, et al.
  Pubblicazione: (2025)
• MTServe: Efficient Serving for Generative Recommendation Models with Hierarchical Caches
  di: Wang, Xin, et al.
  Pubblicazione: (2026)
• MiniCache: KV Cache Compression in Depth Dimension for Large Language Models
  di: Liu, Akide, et al.
  Pubblicazione: (2024)
• TorchGT: A Holistic System for Large-scale Graph Transformer Training
  di: Zhang, Meng, et al.
  Pubblicazione: (2024)