Copertina

GQSA: Group Quantization and Sparsity for Accelerating Large Language Model Inference

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Zeng, Chao, Liu, Songwei, Yang, Shu, Chen, Fangmin, Fu, Lean, Mei, Xing
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

ABQ-LLM: Arbitrary-Bit Quantized Inference Acceleration for Large Language Models
di: Zeng, Chao, et al.
Pubblicazione: (2024)

FoldGPT: Simple and Effective Large Language Model Compression Scheme
di: Liu, Songwei, et al.
Pubblicazione: (2024)

Error Propagation Mechanisms and Compensation Strategies for Quantized Diffusion
di: Liu, Songwei, et al.
Pubblicazione: (2025)

Hybrid SD: Edge-Cloud Collaborative Inference for Stable Diffusion Models
di: Yan, Chenqian, et al.
Pubblicazione: (2024)

ALISA: Accelerating Large Language Model Inference via Sparsity-Aware KV Caching
di: Zhao, Youpeng, et al.
Pubblicazione: (2024)

S2O: Early Stopping for Sparse Attention via Online Permutation
di: Zhang, Yu, et al.
Pubblicazione: (2026)

I-LLM: Efficient Integer-Only Inference for Fully-Quantized Low-Bit Large Language Models
di: Hu, Xing, et al.
Pubblicazione: (2024)

Toward Inference-optimal Mixture-of-Expert Large Language Models
di: Yun, Longfei, et al.
Pubblicazione: (2024)

Determining Layer-wise Sparsity for Large Language Models Through a Theoretical Perspective
di: Huang, Weizhong, et al.
Pubblicazione: (2025)

LLM-Inference-Bench: Inference Benchmarking of Large Language Models on AI Accelerators
di: Chitty-Venkata, Krishna Teja, et al.
Pubblicazione: (2024)

RSAVQ: Riemannian Sensitivity-Aware Vector Quantization for Large Language Models
di: Xu, Zukang, et al.
Pubblicazione: (2025)

SQ-DM: Accelerating Diffusion Models with Aggressive Quantization and Temporal Sparsity
di: Fan, Zichen, et al.
Pubblicazione: (2025)

Pivoting Factorization: A Compact Meta Low-Rank Representation of Sparsity for Efficient Inference in Large Language Models
di: Zhao, Jialin, et al.
Pubblicazione: (2025)

Large Language Model Inference Acceleration: A Comprehensive Hardware Perspective
di: Li, Jinhao, et al.
Pubblicazione: (2024)

QQQ: Quality Quattuor-Bit Quantization for Large Language Models
di: Zhang, Ying, et al.
Pubblicazione: (2024)

ActTail: Global Activation Sparsity in Large Language Models
di: Hou, Wenwen, et al.
Pubblicazione: (2026)

FinLoRA: Finetuning Quantized Financial Large Language Models Using Low-Rank Adaptation
di: Wang, Dannong, et al.
Pubblicazione: (2024)

Window-Diffusion: Accelerating Diffusion Language Model Inference with Windowed Token Pruning and Caching
di: Zuo, Fengrui, et al.
Pubblicazione: (2026)

Saliency-Aware Regularized Quantization Calibration for Large Language Models
di: Zhao, Yanlong, et al.
Pubblicazione: (2026)

E-Sparse: Boosting the Large Language Model Inference through Entropy-based N:M Sparsity
di: Li, Yun, et al.
Pubblicazione: (2023)

Edge Intelligence Optimization for Large Language Model Inference with Batching and Quantization
di: Zhang, Xinyuan, et al.
Pubblicazione: (2024)

Universal Properties of Activation Sparsity in Modern Large Language Models
di: Szatkowski, Filip, et al.
Pubblicazione: (2025)

Activity Sparsity Complements Weight Sparsity for Efficient RNN Inference
di: Mukherji, Rishav, et al.
Pubblicazione: (2023)

WINA: Weight Informed Neuron Activation for Accelerating Large Language Model Inference
di: Chen, Sihan, et al.
Pubblicazione: (2025)

Accelerating Transformer Inference and Training with 2:4 Activation Sparsity
di: Haziza, Daniel, et al.
Pubblicazione: (2025)

Learn To be Efficient: Build Structured Sparsity in Large Language Models
di: Zheng, Haizhong, et al.
Pubblicazione: (2024)

GPU-Accelerated INT8 Quantization for KV Cache Compression in Large Language Models
di: Taneja, Maanas, et al.
Pubblicazione: (2026)

BAQ: Efficient Bit Allocation Quantization for Large Language Models
di: Zhang, Chao, et al.
Pubblicazione: (2025)

AffineQuant: Affine Transformation Quantization for Large Language Models
di: Ma, Yuexiao, et al.
Pubblicazione: (2024)

Turning Trash into Treasure: Accelerating Inference of Large Language Models with Token Recycling
di: Luo, Xianzhen, et al.
Pubblicazione: (2024)

QuickLLaMA: Query-aware Inference Acceleration for Large Language Models
di: Li, Jingyao, et al.
Pubblicazione: (2024)

Amber Pruner: Leveraging N:M Activation Sparsity for Efficient Prefill in Large Language Models
di: An, Tai, et al.
Pubblicazione: (2025)

KBVQ-MoE: KLT-guided SVD with Bias-Corrected Vector Quantization for MoE Large Language Models
di: Xu, Zukang, et al.
Pubblicazione: (2026)

ExAnte: A Benchmark for Ex-Ante Inference in Large Language Models
di: Liu, Yachuan, et al.
Pubblicazione: (2025)

Accelerating Transformer Pre-training with 2:4 Sparsity
di: Hu, Yuezhou, et al.
Pubblicazione: (2024)

A Comprehensive Evaluation on Quantization Techniques for Large Language Models
di: Liu, Yutong, et al.
Pubblicazione: (2025)

KV Cache is 1 Bit Per Channel: Efficient Large Language Model Inference with Coupled Quantization
di: Zhang, Tianyi, et al.
Pubblicazione: (2024)

Accelerating Large Language Model Inference with Self-Supervised Early Exits
di: Valade, Florian
Pubblicazione: (2024)

Compression Scaling Laws:Unifying Sparsity and Quantization
di: Frantar, Elias, et al.
Pubblicazione: (2025)

OPAL: Outlier-Preserved Microscaling Quantization Accelerator for Generative Large Language Models
di: Koo, Jahyun, et al.
Pubblicazione: (2024)