Combining Neural Architecture Search and Automatic Code Optimization: A Survey
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914905523748864 |
|---|---|
| author | Bachiri, Inas Benmeziane, Hadjer Niar, Smail Baghdadi, Riyadh Ouarnoughi, Hamza Aries, Abdelkrime |
| author_facet | Bachiri, Inas Benmeziane, Hadjer Niar, Smail Baghdadi, Riyadh Ouarnoughi, Hamza Aries, Abdelkrime |
| contents | Deep Learning models have experienced exponential growth in complexity and resource demands in recent years. Accelerating these models for efficient execution on resource-constrained devices has become more crucial than ever. Two notable techniques employed to achieve this goal are Hardware-aware Neural Architecture Search (HW-NAS) and Automatic Code Optimization (ACO). HW-NAS automatically designs accurate yet hardware-friendly neural networks, while ACO involves searching for the best compiler optimizations to apply on neural networks for efficient mapping and inference on the target hardware. This survey explores recent works that combine these two techniques within a single framework. We present the fundamental principles of both domains and demonstrate their sub-optimality when performed independently. We then investigate their integration into a joint optimization process that we call Hardware Aware-Neural Architecture and Compiler Optimizations co-Search (NACOS). |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_04116 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Combining Neural Architecture Search and Automatic Code Optimization: A Survey Bachiri, Inas Benmeziane, Hadjer Niar, Smail Baghdadi, Riyadh Ouarnoughi, Hamza Aries, Abdelkrime Machine Learning Programming Languages Deep Learning models have experienced exponential growth in complexity and resource demands in recent years. Accelerating these models for efficient execution on resource-constrained devices has become more crucial than ever. Two notable techniques employed to achieve this goal are Hardware-aware Neural Architecture Search (HW-NAS) and Automatic Code Optimization (ACO). HW-NAS automatically designs accurate yet hardware-friendly neural networks, while ACO involves searching for the best compiler optimizations to apply on neural networks for efficient mapping and inference on the target hardware. This survey explores recent works that combine these two techniques within a single framework. We present the fundamental principles of both domains and demonstrate their sub-optimality when performed independently. We then investigate their integration into a joint optimization process that we call Hardware Aware-Neural Architecture and Compiler Optimizations co-Search (NACOS). |
| title | Combining Neural Architecture Search and Automatic Code Optimization: A Survey |
| topic | Machine Learning Programming Languages |
| url | https://arxiv.org/abs/2408.04116 |