KV Cache is 1 Bit Per Channel: Efficient Large Language Model Inference with Coupled Quantization

Documenti analoghi

• NoMAD-Attention: Efficient LLM Inference on CPUs Through Multiply-add-free Attention
  di: Zhang, Tianyi, et al.
  Pubblicazione: (2024)
• LeanQuant: Accurate and Scalable Large Language Model Quantization with Loss-error-aware Grid
  di: Zhang, Tianyi, et al.
  Pubblicazione: (2024)
• Sketch to Adapt: Fine-Tunable Sketches for Efficient LLM Adaptation
  di: Zhang, Tianyi, et al.
  Pubblicazione: (2024)
• To Compress or Not? Pushing the Frontier of Lossless GenAI Model Weights Compression with Exponent Concentration
  di: Yang, Zeyu, et al.
  Pubblicazione: (2025)
• Scout Before You Attend: Sketch-and-Walk Sparse Attention for Efficient LLM Inference
  di: Le, Hoang Anh Duy, et al.
  Pubblicazione: (2026)