Saved in:
| Main Authors: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2602.04163 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914568419147776 |
|---|---|
| author | Chen, Junyu Li, Jungang Xiong, Jing Wang, Wenjie Yang, Qingyao Xiao, He Li, Zhen Wu, Taiqiang Chen, Mengzhao Peng, Zhen Tao, Chaofan Shi, Long Yang, Hongxia Wong, Ngai |
| author_facet | Chen, Junyu Li, Jungang Xiong, Jing Wang, Wenjie Yang, Qingyao Xiao, He Li, Zhen Wu, Taiqiang Chen, Mengzhao Peng, Zhen Tao, Chaofan Shi, Long Yang, Hongxia Wong, Ngai |
| contents | Large language model inference is often bounded by memory footprint and bandwidth in resource-constrained deployments, making quantization fundamental to efficient serving. While post-training quantization (PTQ) maintains high fidelity at 4-bit, it deteriorates at 2-3 bits. In essence, existing methods enforce a shape-invariant quantization grid (e.g., the fixed uniform intervals of UINT2) for each group, severely restricting the feasible set for error minimization. To address this, we propose Bit-Plane Decomposition Quantization (BPDQ), which constructs a variable quantization grid via bit-planes and scalar coefficients, and iteratively refines them using second-order information while progressively compensating for quantization errors to minimize output discrepancy. In the 2-bit regime, BPDQ enables serving Qwen2.5-72B on a single RTX 3090 with 83.85\% GSM8K accuracy (vs. 90.83\% at 16-bit). Moreover, we theoretically show that the variable grid expands the feasible set, and that the quantization process consistently aligns with the optimization objective in Hessian-induced geometry. The code is available at https://github.com/KingdalfGoodman/BPDQ. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_04163 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | BPDQ: Bit-Plane Decomposition Quantization on a Variable Grid for Large Language Models Chen, Junyu Li, Jungang Xiong, Jing Wang, Wenjie Yang, Qingyao Xiao, He Li, Zhen Wu, Taiqiang Chen, Mengzhao Peng, Zhen Tao, Chaofan Shi, Long Yang, Hongxia Wong, Ngai Machine Learning Large language model inference is often bounded by memory footprint and bandwidth in resource-constrained deployments, making quantization fundamental to efficient serving. While post-training quantization (PTQ) maintains high fidelity at 4-bit, it deteriorates at 2-3 bits. In essence, existing methods enforce a shape-invariant quantization grid (e.g., the fixed uniform intervals of UINT2) for each group, severely restricting the feasible set for error minimization. To address this, we propose Bit-Plane Decomposition Quantization (BPDQ), which constructs a variable quantization grid via bit-planes and scalar coefficients, and iteratively refines them using second-order information while progressively compensating for quantization errors to minimize output discrepancy. In the 2-bit regime, BPDQ enables serving Qwen2.5-72B on a single RTX 3090 with 83.85\% GSM8K accuracy (vs. 90.83\% at 16-bit). Moreover, we theoretically show that the variable grid expands the feasible set, and that the quantization process consistently aligns with the optimization objective in Hessian-induced geometry. The code is available at https://github.com/KingdalfGoodman/BPDQ. |
| title | BPDQ: Bit-Plane Decomposition Quantization on a Variable Grid for Large Language Models |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2602.04163 |