TruncQuant: Truncation-Ready Quantization for DNNs with Flexible Weight Bit Precision
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911003533377536 |
|---|---|
| author | Kim, Jinhee Yoon, Seoyeon Lee, Taeho Lee, Joo Chan Jeon, Kang Eun Ko, Jong Hwan |
| author_facet | Kim, Jinhee Yoon, Seoyeon Lee, Taeho Lee, Joo Chan Jeon, Kang Eun Ko, Jong Hwan |
| contents | The deployment of deep neural networks on edge devices is a challenging task due to the increasing complexity of state-of-the-art models, requiring efforts to reduce model size and inference latency. Recent studies explore models operating at diverse quantization settings to find the optimal point that balances computational efficiency and accuracy. Truncation, an effective approach for achieving lower bit precision mapping, enables a single model to adapt to various hardware platforms with little to no cost. However, formulating a training scheme for deep neural networks to withstand the associated errors introduced by truncation remains a challenge, as the current quantization-aware training schemes are not designed for the truncation process. We propose TruncQuant, a novel truncation-ready training scheme allowing flexible bit precision through bit-shifting in runtime. We achieve this by aligning TruncQuant with the output of the truncation process, demonstrating strong robustness across bit-width settings, and offering an easily implementable training scheme within existing quantization-aware frameworks. Our code is released at https://github.com/a2jinhee/TruncQuant. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_11431 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | TruncQuant: Truncation-Ready Quantization for DNNs with Flexible Weight Bit Precision Kim, Jinhee Yoon, Seoyeon Lee, Taeho Lee, Joo Chan Jeon, Kang Eun Ko, Jong Hwan Machine Learning The deployment of deep neural networks on edge devices is a challenging task due to the increasing complexity of state-of-the-art models, requiring efforts to reduce model size and inference latency. Recent studies explore models operating at diverse quantization settings to find the optimal point that balances computational efficiency and accuracy. Truncation, an effective approach for achieving lower bit precision mapping, enables a single model to adapt to various hardware platforms with little to no cost. However, formulating a training scheme for deep neural networks to withstand the associated errors introduced by truncation remains a challenge, as the current quantization-aware training schemes are not designed for the truncation process. We propose TruncQuant, a novel truncation-ready training scheme allowing flexible bit precision through bit-shifting in runtime. We achieve this by aligning TruncQuant with the output of the truncation process, demonstrating strong robustness across bit-width settings, and offering an easily implementable training scheme within existing quantization-aware frameworks. Our code is released at https://github.com/a2jinhee/TruncQuant. |
| title | TruncQuant: Truncation-Ready Quantization for DNNs with Flexible Weight Bit Precision |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2506.11431 |