Neural Architecture Search of Hybrid Models for NPU-CIM Heterogeneous AR/VR Devices
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913540881776640 |
|---|---|
| author | Zhao, Yiwei Li, Ziyun Khwa, Win-San Sun, Xiaoyu Zhang, Sai Qian Sarwar, Syed Shakib Stangherlin, Kleber Hugo Lu, Yi-Lun Gomez, Jorge Tomas Seo, Jae-Sun Gibbons, Phillip B. De Salvo, Barbara Liu, Chiao |
| author_facet | Zhao, Yiwei Li, Ziyun Khwa, Win-San Sun, Xiaoyu Zhang, Sai Qian Sarwar, Syed Shakib Stangherlin, Kleber Hugo Lu, Yi-Lun Gomez, Jorge Tomas Seo, Jae-Sun Gibbons, Phillip B. De Salvo, Barbara Liu, Chiao |
| contents | Low-Latency and Low-Power Edge AI is essential for Virtual Reality and Augmented Reality applications. Recent advances show that hybrid models, combining convolution layers (CNN) and transformers (ViT), often achieve superior accuracy/performance tradeoff on various computer vision and machine learning (ML) tasks. However, hybrid ML models can pose system challenges for latency and energy-efficiency due to their diverse nature in dataflow and memory access patterns. In this work, we leverage the architecture heterogeneity from Neural Processing Units (NPU) and Compute-In-Memory (CIM) and perform diverse execution schemas to efficiently execute these hybrid models. We also introduce H4H-NAS, a Neural Architecture Search framework to design efficient hybrid CNN/ViT models for heterogeneous edge systems with both NPU and CIM. Our H4H-NAS approach is powered by a performance estimator built with NPU performance results measured on real silicon, and CIM performance based on industry IPs. H4H-NAS searches hybrid CNN/ViT models with fine granularity and achieves significant (up to 1.34%) top-1 accuracy improvement on ImageNet dataset. Moreover, results from our Algo/HW co-design reveal up to 56.08% overall latency and 41.72% energy improvements by introducing such heterogeneous computing over baseline solutions. The framework guides the design of hybrid network architectures and system architectures of NPU+CIM heterogeneous systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_08326 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Neural Architecture Search of Hybrid Models for NPU-CIM Heterogeneous AR/VR Devices Zhao, Yiwei Li, Ziyun Khwa, Win-San Sun, Xiaoyu Zhang, Sai Qian Sarwar, Syed Shakib Stangherlin, Kleber Hugo Lu, Yi-Lun Gomez, Jorge Tomas Seo, Jae-Sun Gibbons, Phillip B. De Salvo, Barbara Liu, Chiao Computer Vision and Pattern Recognition Hardware Architecture Machine Learning Performance Low-Latency and Low-Power Edge AI is essential for Virtual Reality and Augmented Reality applications. Recent advances show that hybrid models, combining convolution layers (CNN) and transformers (ViT), often achieve superior accuracy/performance tradeoff on various computer vision and machine learning (ML) tasks. However, hybrid ML models can pose system challenges for latency and energy-efficiency due to their diverse nature in dataflow and memory access patterns. In this work, we leverage the architecture heterogeneity from Neural Processing Units (NPU) and Compute-In-Memory (CIM) and perform diverse execution schemas to efficiently execute these hybrid models. We also introduce H4H-NAS, a Neural Architecture Search framework to design efficient hybrid CNN/ViT models for heterogeneous edge systems with both NPU and CIM. Our H4H-NAS approach is powered by a performance estimator built with NPU performance results measured on real silicon, and CIM performance based on industry IPs. H4H-NAS searches hybrid CNN/ViT models with fine granularity and achieves significant (up to 1.34%) top-1 accuracy improvement on ImageNet dataset. Moreover, results from our Algo/HW co-design reveal up to 56.08% overall latency and 41.72% energy improvements by introducing such heterogeneous computing over baseline solutions. The framework guides the design of hybrid network architectures and system architectures of NPU+CIM heterogeneous systems. |
| title | Neural Architecture Search of Hybrid Models for NPU-CIM Heterogeneous AR/VR Devices |
| topic | Computer Vision and Pattern Recognition Hardware Architecture Machine Learning Performance |
| url | https://arxiv.org/abs/2410.08326 |