Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2509.03054 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918325666185216 |
|---|---|
| author | Zheng, Xinzhe Yang, Zhen-Qun Liu, Zishan Xie, Haoran Qin, S. Joe Chen, Arlene Lin, Fangzhen |
| author_facet | Zheng, Xinzhe Yang, Zhen-Qun Liu, Zishan Xie, Haoran Qin, S. Joe Chen, Arlene Lin, Fangzhen |
| contents | Large Language Models (LLMs) deliver strong performance but are difficult to deploy under tight memory and compute constraints. Low-bit post-training quantization (PTQ) is a promising direction; however, it typically relies on calibration data, auxiliary transformations, and GPU tools. To address these limitations, we propose MSB (Multi Scale Binary), a calibration-free and transformation-free PTQ method that generalizes binary quantization to multi-bit settings. MSB optimizes a dynamic grouping criterion that minimizes within group variance, yielding group-wise multiscale levels that can be applied consistently across granularities from per tensor to block-wise configurations with 64 elements groups per row, without calibration or intermediate transforms. We implement the optimization in a CPU based solver for the quantization step and evaluate using standard bfloat16 execution without low-bit packing. On Llama 3.2 3B, MSB achieves 8.43 perplexity on WikiText-2 under 4-bit weight only block-wise quantization, compared to 7.81 in full precision and 12.23 with GPTQ its default setup. Overall, MSB provides a new optimization perspective for low-bit PTQ while simplifying the pipeline by removing calibration and transformations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_03054 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Calibration and Transformation-Free Weight-Only LLMs Quantization via Dynamic Grouping Zheng, Xinzhe Yang, Zhen-Qun Liu, Zishan Xie, Haoran Qin, S. Joe Chen, Arlene Lin, Fangzhen Machine Learning Artificial Intelligence Large Language Models (LLMs) deliver strong performance but are difficult to deploy under tight memory and compute constraints. Low-bit post-training quantization (PTQ) is a promising direction; however, it typically relies on calibration data, auxiliary transformations, and GPU tools. To address these limitations, we propose MSB (Multi Scale Binary), a calibration-free and transformation-free PTQ method that generalizes binary quantization to multi-bit settings. MSB optimizes a dynamic grouping criterion that minimizes within group variance, yielding group-wise multiscale levels that can be applied consistently across granularities from per tensor to block-wise configurations with 64 elements groups per row, without calibration or intermediate transforms. We implement the optimization in a CPU based solver for the quantization step and evaluate using standard bfloat16 execution without low-bit packing. On Llama 3.2 3B, MSB achieves 8.43 perplexity on WikiText-2 under 4-bit weight only block-wise quantization, compared to 7.81 in full precision and 12.23 with GPTQ its default setup. Overall, MSB provides a new optimization perspective for low-bit PTQ while simplifying the pipeline by removing calibration and transformations. |
| title | Calibration and Transformation-Free Weight-Only LLMs Quantization via Dynamic Grouping |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2509.03054 |