KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache

Documenti analoghi

• KV Cache Compression, But What Must We Give in Return? A Comprehensive Benchmark of Long Context Capable Approaches
  di: Yuan, Jiayi, et al.
  Pubblicazione: (2024)
• QJL: 1-Bit Quantized JL Transform for KV Cache Quantization with Zero Overhead
  di: Zandieh, Amir, et al.
  Pubblicazione: (2024)
• OSCAR: Offline Spectral Covariance-Aware Rotation for 2-bit KV Cache Quantization
  di: Zhou, Zhongzhu, et al.
  Pubblicazione: (2026)
• Hold Onto That Thought: Assessing KV Cache Compression On Reasoning
  di: Liu, Minghui, et al.
  Pubblicazione: (2025)
• When Quantization Is Free: An int4 KV Cache That Outruns fp16 on Apple Silicon
  di: Bergach, Mohamed Amine
  Pubblicazione: (2026)