Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2503.06518 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912265861595136 |
|---|---|
| author | Zhang, Feng Liu, Yanbin Li, Weihua Lv, Jie Wang, Xiaodan Bai, Quan |
| author_facet | Zhang, Feng Liu, Yanbin Li, Weihua Lv, Jie Wang, Xiaodan Bai, Quan |
| contents | Large Vision and Language Models have exhibited remarkable human-like intelligence in tasks such as natural language comprehension, problem-solving, logical reasoning, and knowledge retrieval. However, training and serving these models require substantial computational resources, posing a significant barrier to their widespread application and further research. To mitigate this challenge, various model compression techniques have been developed to reduce computational requirements. Nevertheless, existing methods often employ uniform quantization configurations, failing to account for the varying difficulties across different layers in quantizing large neural network models. This paper tackles this issue by leveraging layer-sensitivity features, such as activation sensitivity and weight distribution Kurtosis, to identify layers that are challenging to quantize accurately and allocate additional memory budget. The proposed methods, named SensiBoost and KurtBoost, respectively, demonstrate notable improvement in quantization accuracy, achieving up to 9% lower perplexity with only a 2% increase in memory budget on LLama models compared to the baseline. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_06518 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Towards Superior Quantization Accuracy: A Layer-sensitive Approach Zhang, Feng Liu, Yanbin Li, Weihua Lv, Jie Wang, Xiaodan Bai, Quan Machine Learning Artificial Intelligence Large Vision and Language Models have exhibited remarkable human-like intelligence in tasks such as natural language comprehension, problem-solving, logical reasoning, and knowledge retrieval. However, training and serving these models require substantial computational resources, posing a significant barrier to their widespread application and further research. To mitigate this challenge, various model compression techniques have been developed to reduce computational requirements. Nevertheless, existing methods often employ uniform quantization configurations, failing to account for the varying difficulties across different layers in quantizing large neural network models. This paper tackles this issue by leveraging layer-sensitivity features, such as activation sensitivity and weight distribution Kurtosis, to identify layers that are challenging to quantize accurately and allocate additional memory budget. The proposed methods, named SensiBoost and KurtBoost, respectively, demonstrate notable improvement in quantization accuracy, achieving up to 9% lower perplexity with only a 2% increase in memory budget on LLama models compared to the baseline. |
| title | Towards Superior Quantization Accuracy: A Layer-sensitive Approach |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2503.06518 |