Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2412.18184 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910954381377536 |
|---|---|
| author | Zhang, Haoyu Saab, Rayan |
| author_facet | Zhang, Haoyu Saab, Rayan |
| contents | Quantization and pruning are two essential techniques for compressing neural networks, yet they are often treated independently, with limited theoretical analysis connecting them. This paper introduces a unified framework for post-training quantization and pruning using stochastic path-following algorithms. Our approach builds on the Stochastic Path Following Quantization (SPFQ) method, extending its applicability to pruning and low-bit quantization, including challenging 1-bit regimes. By incorporating a scaling parameter and generalizing the stochastic operator, the proposed method achieves robust error correction and yields rigorous theoretical error bounds for both quantization and pruning as well as their combination. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_18184 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Unified Stochastic Framework for Neural Network Quantization and Pruning Zhang, Haoyu Saab, Rayan Machine Learning Numerical Analysis Probability Quantization and pruning are two essential techniques for compressing neural networks, yet they are often treated independently, with limited theoretical analysis connecting them. This paper introduces a unified framework for post-training quantization and pruning using stochastic path-following algorithms. Our approach builds on the Stochastic Path Following Quantization (SPFQ) method, extending its applicability to pruning and low-bit quantization, including challenging 1-bit regimes. By incorporating a scaling parameter and generalizing the stochastic operator, the proposed method achieves robust error correction and yields rigorous theoretical error bounds for both quantization and pruning as well as their combination. |
| title | Unified Stochastic Framework for Neural Network Quantization and Pruning |
| topic | Machine Learning Numerical Analysis Probability |
| url | https://arxiv.org/abs/2412.18184 |