AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration

Documenti analoghi

• QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving
  di: Lin, Yujun, et al.
  Pubblicazione: (2024)
• DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads
  di: Xiao, Guangxuan, et al.
  Pubblicazione: (2024)
• LServe: Efficient Long-sequence LLM Serving with Unified Sparse Attention
  di: Yang, Shang, et al.
  Pubblicazione: (2025)
• Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference
  di: Tang, Jiaming, et al.
  Pubblicazione: (2024)
• MCUNetV2: Memory-Efficient Patch-based Inference for Tiny Deep Learning
  di: Lin, Ji, et al.
  Pubblicazione: (2021)