Opt-GPTQ: An Optimized GPTQ Combining Sparse Attention and Quantization Techniques
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908443927904256 |
|---|---|
| author | Kong, Jie Zhang, Junxiang Xu, Jiheng Li, Yalong Zhang, Shouhua Zhou, Jiehan Liu, Yuhai Liang, Peng Zhang, Quan Jiang, Luohan |
| author_facet | Kong, Jie Zhang, Junxiang Xu, Jiheng Li, Yalong Zhang, Shouhua Zhou, Jiehan Liu, Yuhai Liang, Peng Zhang, Quan Jiang, Luohan |
| contents | In the field of deep learning, traditional attention mechanisms face significant challenges related to high computational complexity and large memory consumption when processing long sequence data. To address these limitations, we propose Opt-GPTQ, an optimized Gradient-based Post Training Quantization (GPTQ) combining the Grouped Query Attention (GQA) mechanism with paging memory management, optimizing the traditional Multi-Head Attention (MHA) mechanism by grouping query heads and sharing key-value vectors. Optimized GQA (Opt-GQA) effectively reduces computational complexity, minimizes memory fragmentation, and enhances memory utilization for large-scale models. Opt-GPTQ is optimized for Data Center Units (DCUs) and integrated into the vLLM model to maximize hardware efficiency. It customizes GPU kernels to further enhance attention computation by reducing memory access latency and boosting parallel computing capabilities. Opt-GQA integrates Attention with Linear Biases (ALiBi) to reduce overhead and enhance long-sequence processing. Experimental results show that Opt-GPTQ significantly reduces computation time and memory usage while improving model performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_02351 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Opt-GPTQ: An Optimized GPTQ Combining Sparse Attention and Quantization Techniques Kong, Jie Zhang, Junxiang Xu, Jiheng Li, Yalong Zhang, Shouhua Zhou, Jiehan Liu, Yuhai Liang, Peng Zhang, Quan Jiang, Luohan Distributed, Parallel, and Cluster Computing In the field of deep learning, traditional attention mechanisms face significant challenges related to high computational complexity and large memory consumption when processing long sequence data. To address these limitations, we propose Opt-GPTQ, an optimized Gradient-based Post Training Quantization (GPTQ) combining the Grouped Query Attention (GQA) mechanism with paging memory management, optimizing the traditional Multi-Head Attention (MHA) mechanism by grouping query heads and sharing key-value vectors. Optimized GQA (Opt-GQA) effectively reduces computational complexity, minimizes memory fragmentation, and enhances memory utilization for large-scale models. Opt-GPTQ is optimized for Data Center Units (DCUs) and integrated into the vLLM model to maximize hardware efficiency. It customizes GPU kernels to further enhance attention computation by reducing memory access latency and boosting parallel computing capabilities. Opt-GQA integrates Attention with Linear Biases (ALiBi) to reduce overhead and enhance long-sequence processing. Experimental results show that Opt-GPTQ significantly reduces computation time and memory usage while improving model performance. |
| title | Opt-GPTQ: An Optimized GPTQ Combining Sparse Attention and Quantization Techniques |
| topic | Distributed, Parallel, and Cluster Computing |
| url | https://arxiv.org/abs/2505.02351 |