GQSA: Group Quantization and Sparsity for Accelerating Large Language Model Inference

Documenti analoghi

• ABQ-LLM: Arbitrary-Bit Quantized Inference Acceleration for Large Language Models
  di: Zeng, Chao, et al.
  Pubblicazione: (2024)
• FoldGPT: Simple and Effective Large Language Model Compression Scheme
  di: Liu, Songwei, et al.
  Pubblicazione: (2024)
• Error Propagation Mechanisms and Compensation Strategies for Quantized Diffusion
  di: Liu, Songwei, et al.
  Pubblicazione: (2025)
• Hybrid SD: Edge-Cloud Collaborative Inference for Stable Diffusion Models
  di: Yan, Chenqian, et al.
  Pubblicazione: (2024)
• ALISA: Accelerating Large Language Model Inference via Sparsity-Aware KV Caching
  di: Zhao, Youpeng, et al.
  Pubblicazione: (2024)