Graph is all you need? Lightweight data-agnostic neural architecture search without training
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866908413906124800 |
|---|---|
| author | Huang, Zhenhan Pedapati, Tejaswini Chen, Pin-Yu Jiang, Chunheng Gao, Jianxi |
| author_facet | Huang, Zhenhan Pedapati, Tejaswini Chen, Pin-Yu Jiang, Chunheng Gao, Jianxi |
| contents | Neural architecture search (NAS) enables the automatic design of neural network models. However, training the candidates generated by the search algorithm for performance evaluation incurs considerable computational overhead. Our method, dubbed nasgraph, remarkably reduces the computational costs by converting neural architectures to graphs and using the average degree, a graph measure, as the proxy in lieu of the evaluation metric. Our training-free NAS method is data-agnostic and light-weight. It can find the best architecture among 200 randomly sampled architectures from NAS-Bench201 in 217 CPU seconds. Besides, our method is able to achieve competitive performance on various datasets including NASBench-101, NASBench-201, and NDS search spaces. We also demonstrate that nasgraph generalizes to more challenging tasks on Micro TransNAS-Bench-101. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_01306 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Graph is all you need? Lightweight data-agnostic neural architecture search without training Huang, Zhenhan Pedapati, Tejaswini Chen, Pin-Yu Jiang, Chunheng Gao, Jianxi Machine Learning Neural architecture search (NAS) enables the automatic design of neural network models. However, training the candidates generated by the search algorithm for performance evaluation incurs considerable computational overhead. Our method, dubbed nasgraph, remarkably reduces the computational costs by converting neural architectures to graphs and using the average degree, a graph measure, as the proxy in lieu of the evaluation metric. Our training-free NAS method is data-agnostic and light-weight. It can find the best architecture among 200 randomly sampled architectures from NAS-Bench201 in 217 CPU seconds. Besides, our method is able to achieve competitive performance on various datasets including NASBench-101, NASBench-201, and NDS search spaces. We also demonstrate that nasgraph generalizes to more challenging tasks on Micro TransNAS-Bench-101. |
| title | Graph is all you need? Lightweight data-agnostic neural architecture search without training |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2405.01306 |