HGQ: High Granularity Quantization for Real-time Neural Networks on FPGAs
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866908723068272640 |
|---|---|
| author | Sun, Chang Que, Zhiqiang Årrestad, Thea K. Loncar, Vladimir Ngadiuba, Jennifer Luk, Wayne Spiropulu, Maria |
| author_facet | Sun, Chang Que, Zhiqiang Årrestad, Thea K. Loncar, Vladimir Ngadiuba, Jennifer Luk, Wayne Spiropulu, Maria |
| contents | Neural networks with sub-microsecond inference latency are required by many critical applications. Targeting such applications deployed on FPGAs, we present High Granularity Quantization (HGQ), a quantization-aware training framework that optimizes parameter bit-widths through gradient descent. Unlike conventional methods, HGQ determines the optimal bit-width for each parameter independently, making it suitable for hardware platforms supporting heterogeneous arbitrary precision arithmetic. In our experiments, HGQ shows superior performance compared to existing network compression methods, achieving orders of magnitude reduction in resource consumption and latency while maintaining the accuracy on several benchmark tasks. These improvements enable the deployment of complex models previously infeasible due to resource or latency constraints. HGQ is open-source and is used for developing next-generation trigger systems at the CERN ATLAS and CMS experiments for particle physics, enabling the use of advanced machine learning models for real-time data selection with sub-microsecond latency. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_00645 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | HGQ: High Granularity Quantization for Real-time Neural Networks on FPGAs Sun, Chang Que, Zhiqiang Årrestad, Thea K. Loncar, Vladimir Ngadiuba, Jennifer Luk, Wayne Spiropulu, Maria Machine Learning Instrumentation and Detectors Neural networks with sub-microsecond inference latency are required by many critical applications. Targeting such applications deployed on FPGAs, we present High Granularity Quantization (HGQ), a quantization-aware training framework that optimizes parameter bit-widths through gradient descent. Unlike conventional methods, HGQ determines the optimal bit-width for each parameter independently, making it suitable for hardware platforms supporting heterogeneous arbitrary precision arithmetic. In our experiments, HGQ shows superior performance compared to existing network compression methods, achieving orders of magnitude reduction in resource consumption and latency while maintaining the accuracy on several benchmark tasks. These improvements enable the deployment of complex models previously infeasible due to resource or latency constraints. HGQ is open-source and is used for developing next-generation trigger systems at the CERN ATLAS and CMS experiments for particle physics, enabling the use of advanced machine learning models for real-time data selection with sub-microsecond latency. |
| title | HGQ: High Granularity Quantization for Real-time Neural Networks on FPGAs |
| topic | Machine Learning Instrumentation and Detectors |
| url | https://arxiv.org/abs/2405.00645 |