Rescaling-Aware Training for Efficient Deployment of Deep Learning Models on Full-Integer Hardware
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914090312531968 |
|---|---|
| author | Mueller, Lion Garcia-Ortiz, Alberto Najafi, Ardalan Fuks, Adam Bamberg, Lennart |
| author_facet | Mueller, Lion Garcia-Ortiz, Alberto Najafi, Ardalan Fuks, Adam Bamberg, Lennart |
| contents | Integer AI inference significantly reduces computational complexity in embedded systems. Quantization-aware training (QAT) helps mitigate accuracy degradation associated with post-training quantization but still overlooks the impact of integer rescaling during inference, which is a hardware costly operation in integer-only AI inference. This work shows that rescaling cost can be dramatically reduced post-training, by applying a stronger quantization to the rescale multiplicands at no model-quality loss. Furthermore, we introduce Rescale-Aware Training, a fine tuning method for ultra-low bit-width rescaling multiplicands. Experiments show that even with 8x reduced rescaler widths, the full accuracy is preserved through minimal incremental retraining. This enables more energy-efficient and cost-efficient AI inference for resource-constrained embedded systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_11484 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Rescaling-Aware Training for Efficient Deployment of Deep Learning Models on Full-Integer Hardware Mueller, Lion Garcia-Ortiz, Alberto Najafi, Ardalan Fuks, Adam Bamberg, Lennart Machine Learning Hardware Architecture Integer AI inference significantly reduces computational complexity in embedded systems. Quantization-aware training (QAT) helps mitigate accuracy degradation associated with post-training quantization but still overlooks the impact of integer rescaling during inference, which is a hardware costly operation in integer-only AI inference. This work shows that rescaling cost can be dramatically reduced post-training, by applying a stronger quantization to the rescale multiplicands at no model-quality loss. Furthermore, we introduce Rescale-Aware Training, a fine tuning method for ultra-low bit-width rescaling multiplicands. Experiments show that even with 8x reduced rescaler widths, the full accuracy is preserved through minimal incremental retraining. This enables more energy-efficient and cost-efficient AI inference for resource-constrained embedded systems. |
| title | Rescaling-Aware Training for Efficient Deployment of Deep Learning Models on Full-Integer Hardware |
| topic | Machine Learning Hardware Architecture |
| url | https://arxiv.org/abs/2510.11484 |