Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2505.19995 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915636931723264 |
|---|---|
| author | Aach, Marcel Blanc, Cyril Lintermann, Andreas De Grave, Kurt |
| author_facet | Aach, Marcel Blanc, Cyril Lintermann, Andreas De Grave, Kurt |
| contents | Artificial intelligence and machine learning models deployed on edge devices, e.g., for quality control in Additive Manufacturing (AM), are frequently small in size. Such models usually have to deliver highly accurate results within a short time frame. Methods that are commonly employed in literature start out with larger trained models and try to reduce their memory and latency footprint by structural pruning, knowledge distillation, or quantization. It is, however, also possible to leverage hardware-aware Neural Architecture Search (NAS), an approach that seeks to systematically explore the architecture space to find optimized configurations. In this study, a hardware-aware NAS workflow is introduced that couples an edge device located in Belgium with a powerful High-Performance Computing system in Germany, to train possible architecture candidates as fast as possible while performing real-time latency measurements on the target hardware. The approach is verified on a use case in the AM domain, based on the open RAISE-LPBF dataset, achieving ~8.8 times faster inference speed while simultaneously enhancing model quality by a factor of ~1.35, compared to a human-designed baseline. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_19995 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Optimizing edge AI models on HPC systems with the edge in the loop Aach, Marcel Blanc, Cyril Lintermann, Andreas De Grave, Kurt Distributed, Parallel, and Cluster Computing Computer Vision and Pattern Recognition I.2.6; D.1.3; I.2.8; I.5.1 Artificial intelligence and machine learning models deployed on edge devices, e.g., for quality control in Additive Manufacturing (AM), are frequently small in size. Such models usually have to deliver highly accurate results within a short time frame. Methods that are commonly employed in literature start out with larger trained models and try to reduce their memory and latency footprint by structural pruning, knowledge distillation, or quantization. It is, however, also possible to leverage hardware-aware Neural Architecture Search (NAS), an approach that seeks to systematically explore the architecture space to find optimized configurations. In this study, a hardware-aware NAS workflow is introduced that couples an edge device located in Belgium with a powerful High-Performance Computing system in Germany, to train possible architecture candidates as fast as possible while performing real-time latency measurements on the target hardware. The approach is verified on a use case in the AM domain, based on the open RAISE-LPBF dataset, achieving ~8.8 times faster inference speed while simultaneously enhancing model quality by a factor of ~1.35, compared to a human-designed baseline. |
| title | Optimizing edge AI models on HPC systems with the edge in the loop |
| topic | Distributed, Parallel, and Cluster Computing Computer Vision and Pattern Recognition I.2.6; D.1.3; I.2.8; I.5.1 |
| url | https://arxiv.org/abs/2505.19995 |