Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2411.11038 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910702416953344 |
|---|---|
| author | Ashkboos, Saleh Verhoef, Bram Hoefler, Torsten Eleftheriou, Evangelos Dazzi, Martino |
| author_facet | Ashkboos, Saleh Verhoef, Bram Hoefler, Torsten Eleftheriou, Evangelos Dazzi, Martino |
| contents | Quantization-aware training (QAT) schemes have been shown to achieve near-full precision accuracy. They accomplish this by training a quantized model for multiple epochs. This is computationally expensive, mainly because of the full precision backward pass. On the other hand, post-training quantization (PTQ) schemes do not involve training and are therefore computationally cheap, but they usually result in a significant accuracy drop. We address these challenges by proposing EfQAT, which generalizes both schemes by optimizing only a subset of the parameters of a quantized model. EfQAT starts by applying a PTQ scheme to a pre-trained model and only updates the most critical network parameters while freezing the rest, accelerating the backward pass. We demonstrate the effectiveness of EfQAT on various CNNs and Transformer-based models using different GPUs. Specifically, we show that EfQAT is significantly more accurate than PTQ with little extra compute. Furthermore, EfQAT can accelerate the QAT backward pass between 1.44-1.64x while retaining most accuracy. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_11038 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | EfQAT: An Efficient Framework for Quantization-Aware Training Ashkboos, Saleh Verhoef, Bram Hoefler, Torsten Eleftheriou, Evangelos Dazzi, Martino Machine Learning Quantization-aware training (QAT) schemes have been shown to achieve near-full precision accuracy. They accomplish this by training a quantized model for multiple epochs. This is computationally expensive, mainly because of the full precision backward pass. On the other hand, post-training quantization (PTQ) schemes do not involve training and are therefore computationally cheap, but they usually result in a significant accuracy drop. We address these challenges by proposing EfQAT, which generalizes both schemes by optimizing only a subset of the parameters of a quantized model. EfQAT starts by applying a PTQ scheme to a pre-trained model and only updates the most critical network parameters while freezing the rest, accelerating the backward pass. We demonstrate the effectiveness of EfQAT on various CNNs and Transformer-based models using different GPUs. Specifically, we show that EfQAT is significantly more accurate than PTQ with little extra compute. Furthermore, EfQAT can accelerate the QAT backward pass between 1.44-1.64x while retaining most accuracy. |
| title | EfQAT: An Efficient Framework for Quantization-Aware Training |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2411.11038 |