QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving

Documenti analoghi

• LServe: Efficient Long-sequence LLM Serving with Unified Sparse Attention
  di: Yang, Shang, et al.
  Pubblicazione: (2025)
• SparseX: Efficient Segment-Level KV Cache Sharing for Interleaved LLM Serving
  di: Zhang, Quqing, et al.
  Pubblicazione: (2026)
• When Quantization Is Free: An int4 KV Cache That Outruns fp16 on Apple Silicon
  di: Bergach, Mohamed Amine
  Pubblicazione: (2026)
• GPU-Accelerated INT8 Quantization for KV Cache Compression in Large Language Models
  di: Taneja, Maanas, et al.
  Pubblicazione: (2026)
• Opt4GPTQ: Co-Optimizing Memory and Computation for 4-bit GPTQ Quantized LLM Inference on Heterogeneous Platforms
  di: Zhang, Yaozheng, et al.
  Pubblicazione: (2025)