Guardado en:
| Autores principales: | , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2023
|
| Materias: | |
| Acceso en línea: | https://arxiv.org/abs/2302.12510 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866929241384288256 |
|---|---|
| author | Zhou, Jiajun Wu, Jiajun Gao, Yizhao Ding, Yuhao Tao, Chaofan Li, Boyu Tu, Fengbin Cheng, Kwang-Ting So, Hayden Kwok-Hay Wong, Ngai |
| author_facet | Zhou, Jiajun Wu, Jiajun Gao, Yizhao Ding, Yuhao Tao, Chaofan Li, Boyu Tu, Fengbin Cheng, Kwang-Ting So, Hayden Kwok-Hay Wong, Ngai |
| contents | To accelerate the inference of deep neural networks (DNNs), quantization with low-bitwidth numbers is actively researched. A prominent challenge is to quantize the DNN models into low-bitwidth numbers without significant accuracy degradation, especially at very low bitwidths (< 8 bits). This work targets an adaptive data representation with variable-length encoding called DyBit. DyBit can dynamically adjust the precision and range of separate bit-field to be adapted to the DNN weights/activations distribution. We also propose a hardware-aware quantization framework with a mixed-precision accelerator to trade-off the inference accuracy and speedup. Experimental results demonstrate that the inference accuracy via DyBit is 1.997% higher than the state-of-the-art at 4-bit quantization, and the proposed framework can achieve up to 8.1x speedup compared with the original model. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2302_12510 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | DyBit: Dynamic Bit-Precision Numbers for Efficient Quantized Neural Network Inference Zhou, Jiajun Wu, Jiajun Gao, Yizhao Ding, Yuhao Tao, Chaofan Li, Boyu Tu, Fengbin Cheng, Kwang-Ting So, Hayden Kwok-Hay Wong, Ngai Machine Learning To accelerate the inference of deep neural networks (DNNs), quantization with low-bitwidth numbers is actively researched. A prominent challenge is to quantize the DNN models into low-bitwidth numbers without significant accuracy degradation, especially at very low bitwidths (< 8 bits). This work targets an adaptive data representation with variable-length encoding called DyBit. DyBit can dynamically adjust the precision and range of separate bit-field to be adapted to the DNN weights/activations distribution. We also propose a hardware-aware quantization framework with a mixed-precision accelerator to trade-off the inference accuracy and speedup. Experimental results demonstrate that the inference accuracy via DyBit is 1.997% higher than the state-of-the-art at 4-bit quantization, and the proposed framework can achieve up to 8.1x speedup compared with the original model. |
| title | DyBit: Dynamic Bit-Precision Numbers for Efficient Quantized Neural Network Inference |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2302.12510 |