Outlier-Aware Training for Low-Bit Quantization of Structural Re-Parameterized Networks
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909102078164992 |
|---|---|
| author | Niu, Muqun Ren, Yuan Li, Boyu Ding, Chenchen |
| author_facet | Niu, Muqun Ren, Yuan Li, Boyu Ding, Chenchen |
| contents | Lightweight design of Convolutional Neural Networks (CNNs) requires co-design efforts in the model architectures and compression techniques. As a novel design paradigm that separates training and inference, a structural re-parameterized (SR) network such as the representative RepVGG revitalizes the simple VGG-like network with a high accuracy comparable to advanced and often more complicated networks. However, the merging process in SR networks introduces outliers into weights, making their distribution distinct from conventional networks and thus heightening difficulties in quantization. To address this, we propose an operator-level improvement for training called Outlier Aware Batch Normalization (OABN). Additionally, to meet the demands of limited bitwidths while upkeeping the inference accuracy, we develop a clustering-based non-uniform quantization framework for Quantization-Aware Training (QAT) named ClusterQAT. Integrating OABN with ClusterQAT, the quantized performance of RepVGG is largely enhanced, particularly when the bitwidth falls below 8. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_07200 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Outlier-Aware Training for Low-Bit Quantization of Structural Re-Parameterized Networks Niu, Muqun Ren, Yuan Li, Boyu Ding, Chenchen Computer Vision and Pattern Recognition Machine Learning Neural and Evolutionary Computing Lightweight design of Convolutional Neural Networks (CNNs) requires co-design efforts in the model architectures and compression techniques. As a novel design paradigm that separates training and inference, a structural re-parameterized (SR) network such as the representative RepVGG revitalizes the simple VGG-like network with a high accuracy comparable to advanced and often more complicated networks. However, the merging process in SR networks introduces outliers into weights, making their distribution distinct from conventional networks and thus heightening difficulties in quantization. To address this, we propose an operator-level improvement for training called Outlier Aware Batch Normalization (OABN). Additionally, to meet the demands of limited bitwidths while upkeeping the inference accuracy, we develop a clustering-based non-uniform quantization framework for Quantization-Aware Training (QAT) named ClusterQAT. Integrating OABN with ClusterQAT, the quantized performance of RepVGG is largely enhanced, particularly when the bitwidth falls below 8. |
| title | Outlier-Aware Training for Low-Bit Quantization of Structural Re-Parameterized Networks |
| topic | Computer Vision and Pattern Recognition Machine Learning Neural and Evolutionary Computing |
| url | https://arxiv.org/abs/2402.07200 |