Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2512.16476 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911325761830912 |
|---|---|
| author | Sun, Pengfei Jiang, Wenyu Chee, Piew Yoong Devos, Paul Botteldooren, Dick |
| author_facet | Sun, Pengfei Jiang, Wenyu Chee, Piew Yoong Devos, Paul Botteldooren, Dick |
| contents | Quantised neural networks (QNNs) shrink models and reduce inference energy through low-bit arithmetic, yet most still depend on a running statistics batch normalisation (BN) layer, preventing true integer-only deployment. Prior attempts remove BN by parameter folding or tailored initialisation; while helpful, they rarely recover BN's stability and accuracy and often impose bespoke constraints. We present a BN-free, fully integer QNN trained via a progressive, layer-wise distillation scheme that slots into existing low-bit pipelines. Starting from a pretrained BN-enabled teacher, we use layer-wise targets and progressive compensation to train a student that performs inference exclusively with integer arithmetic and contains no BN operations. On ImageNet with AlexNet, the BN-free model attains competitive Top-1 accuracy under aggressive quantisation. The procedure integrates directly with standard quantisation workflows, enabling end-to-end integer-only inference for resource-constrained settings such as edge and embedded devices. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_16476 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Batch Normalization-Free Fully Integer Quantized Neural Networks via Progressive Tandem Learning Sun, Pengfei Jiang, Wenyu Chee, Piew Yoong Devos, Paul Botteldooren, Dick Machine Learning Signal Processing Quantised neural networks (QNNs) shrink models and reduce inference energy through low-bit arithmetic, yet most still depend on a running statistics batch normalisation (BN) layer, preventing true integer-only deployment. Prior attempts remove BN by parameter folding or tailored initialisation; while helpful, they rarely recover BN's stability and accuracy and often impose bespoke constraints. We present a BN-free, fully integer QNN trained via a progressive, layer-wise distillation scheme that slots into existing low-bit pipelines. Starting from a pretrained BN-enabled teacher, we use layer-wise targets and progressive compensation to train a student that performs inference exclusively with integer arithmetic and contains no BN operations. On ImageNet with AlexNet, the BN-free model attains competitive Top-1 accuracy under aggressive quantisation. The procedure integrates directly with standard quantisation workflows, enabling end-to-end integer-only inference for resource-constrained settings such as edge and embedded devices. |
| title | Batch Normalization-Free Fully Integer Quantized Neural Networks via Progressive Tandem Learning |
| topic | Machine Learning Signal Processing |
| url | https://arxiv.org/abs/2512.16476 |