ZO-DARTS++: An Efficient and Size-Variable Zeroth-Order Neural Architecture Search Algorithm
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916646962069504 |
|---|---|
| author | Xie, Lunchen Lomurno, Eugenio Gambella, Matteo Ardagna, Danilo Roveri, Manual Matteucci, Matteo Shi, Qingjiang |
| author_facet | Xie, Lunchen Lomurno, Eugenio Gambella, Matteo Ardagna, Danilo Roveri, Manual Matteucci, Matteo Shi, Qingjiang |
| contents | Differentiable Neural Architecture Search (NAS) provides a promising avenue for automating the complex design of deep learning (DL) models. However, current differentiable NAS methods often face constraints in efficiency, operation selection, and adaptability under varying resource limitations. We introduce ZO-DARTS++, a novel NAS method that effectively balances performance and resource constraints. By integrating a zeroth-order approximation for efficient gradient handling, employing a sparsemax function with temperature annealing for clearer and more interpretable architecture distributions, and adopting a size-variable search scheme for generating compact yet accurate architectures, ZO-DARTS++ establishes a new balance between model complexity and performance. In extensive tests on medical imaging datasets, ZO-DARTS++ improves the average accuracy by up to 1.8\% over standard DARTS-based methods and shortens search time by approximately 38.6\%. Additionally, its resource-constrained variants can reduce the number of parameters by more than 35\% while maintaining competitive accuracy levels. Thus, ZO-DARTS++ offers a versatile and efficient framework for generating high-quality, resource-aware DL models suitable for real-world medical applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_06092 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ZO-DARTS++: An Efficient and Size-Variable Zeroth-Order Neural Architecture Search Algorithm Xie, Lunchen Lomurno, Eugenio Gambella, Matteo Ardagna, Danilo Roveri, Manual Matteucci, Matteo Shi, Qingjiang Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning I.5.1; I.5.4; I.2.6; I.2.10 Differentiable Neural Architecture Search (NAS) provides a promising avenue for automating the complex design of deep learning (DL) models. However, current differentiable NAS methods often face constraints in efficiency, operation selection, and adaptability under varying resource limitations. We introduce ZO-DARTS++, a novel NAS method that effectively balances performance and resource constraints. By integrating a zeroth-order approximation for efficient gradient handling, employing a sparsemax function with temperature annealing for clearer and more interpretable architecture distributions, and adopting a size-variable search scheme for generating compact yet accurate architectures, ZO-DARTS++ establishes a new balance between model complexity and performance. In extensive tests on medical imaging datasets, ZO-DARTS++ improves the average accuracy by up to 1.8\% over standard DARTS-based methods and shortens search time by approximately 38.6\%. Additionally, its resource-constrained variants can reduce the number of parameters by more than 35\% while maintaining competitive accuracy levels. Thus, ZO-DARTS++ offers a versatile and efficient framework for generating high-quality, resource-aware DL models suitable for real-world medical applications. |
| title | ZO-DARTS++: An Efficient and Size-Variable Zeroth-Order Neural Architecture Search Algorithm |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning I.5.1; I.5.4; I.2.6; I.2.10 |
| url | https://arxiv.org/abs/2503.06092 |