KVTuner: Sensitivity-Aware Layer-Wise Mixed-Precision KV Cache Quantization for Efficient and Nearly Lossless LLM Inference

Documenti analoghi

• Beyond Speedup -- Utilizing KV Cache for Sampling and Reasoning
  di: Xing, Zeyu, et al.
  Pubblicazione: (2026)
• SVDq: 1.25-bit and 410x Key Cache Compression for LLM Attention
  di: Yankun, Hong, et al.
  Pubblicazione: (2025)
• MixPE: Quantization and Hardware Co-design for Efficient LLM Inference
  di: Zhang, Yu, et al.
  Pubblicazione: (2024)
• KVmix: Gradient-Based Layer Importance-Aware Mixed-Precision Quantization for KV Cache
  di: Li, Fei, et al.
  Pubblicazione: (2025)
• WindowQuant: Mixed-Precision KV Cache Quantization based on Window-Level Similarity for VLMs Inference Optimization
  di: Tao, Wei, et al.
  Pubblicazione: (2026)