Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2410.24214 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917440984711168 |
|---|---|
| author | Yang, Yuchen Zhao, Yifan Ugare, Shubham Singh, Gagandeep Misailovic, Sasa |
| author_facet | Yang, Yuchen Zhao, Yifan Ugare, Shubham Singh, Gagandeep Misailovic, Sasa |
| contents | Mixed precision quantization has become an important technique for optimizing the execution of deep neural networks (DNNs). Certified robustness, which provides provable guarantees about a model's ability to withstand different adversarial perturbations, has rarely been addressed in quantization due to the unacceptably high cost of certifying robustness. This paper introduces ARQ, an innovative mixed-precision quantization method that not only preserves the clean accuracy of the smoothed classifiers, but also maintains their certified robustness. ARQ uses reinforcement learning to find accurate and robust DNN quantization, while efficiently leveraging randomized smoothing, a popular class of statistical DNN verification algorithms. ARQ consistently performs better than multiple state-of-the-art quantization techniques across all the benchmarks and the input perturbation levels. The performance of ARQ quantized networks reaches that of the original DNN with floating-point weights, while using only 1.5% instructions and the highest certified radius. ARQ's code is available at https://github.com/uiuc-arc/ARQ. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_24214 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | ARQ: A Mixed-Precision Quantization Framework for Accurate and Certifiably Robust DNNs Yang, Yuchen Zhao, Yifan Ugare, Shubham Singh, Gagandeep Misailovic, Sasa Machine Learning Cryptography and Security Computer Vision and Pattern Recognition Mixed precision quantization has become an important technique for optimizing the execution of deep neural networks (DNNs). Certified robustness, which provides provable guarantees about a model's ability to withstand different adversarial perturbations, has rarely been addressed in quantization due to the unacceptably high cost of certifying robustness. This paper introduces ARQ, an innovative mixed-precision quantization method that not only preserves the clean accuracy of the smoothed classifiers, but also maintains their certified robustness. ARQ uses reinforcement learning to find accurate and robust DNN quantization, while efficiently leveraging randomized smoothing, a popular class of statistical DNN verification algorithms. ARQ consistently performs better than multiple state-of-the-art quantization techniques across all the benchmarks and the input perturbation levels. The performance of ARQ quantized networks reaches that of the original DNN with floating-point weights, while using only 1.5% instructions and the highest certified radius. ARQ's code is available at https://github.com/uiuc-arc/ARQ. |
| title | ARQ: A Mixed-Precision Quantization Framework for Accurate and Certifiably Robust DNNs |
| topic | Machine Learning Cryptography and Security Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2410.24214 |