Analyzing the Expected Hitting Time of Evolutionary Computation-based Neural Architecture Search Algorithms
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2022
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912086769008640 |
|---|---|
| author | Lv, Zeqiong Qian, Chao Yen, Gary G. Sun, Yanan |
| author_facet | Lv, Zeqiong Qian, Chao Yen, Gary G. Sun, Yanan |
| contents | Evolutionary computation-based neural architecture search (ENAS) is a popular technique for automating architecture design of deep neural networks. Despite its groundbreaking applications, there is no theoretical study for ENAS. The expected hitting time (EHT) is one of the most important theoretical issues, since it implies the average computational time complexity. This paper proposes a general method by integrating theory and experiment for estimating the EHT of ENAS algorithms, which includes common configuration, search space partition, transition probability estimation, population distribution fitting, and hitting time analysis. By exploiting the proposed method, we consider the ($λ$+$λ$)-ENAS algorithms with different mutation operators and estimate the lower bounds of the EHT. Furthermore, we study the EHT on the NAS-Bench-101 problem, and the results demonstrate the validity of the proposed method. To the best of our knowledge, this work is the first attempt to establish a theoretical foundation for ENAS algorithms. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2210_05397 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | Analyzing the Expected Hitting Time of Evolutionary Computation-based Neural Architecture Search Algorithms Lv, Zeqiong Qian, Chao Yen, Gary G. Sun, Yanan Neural and Evolutionary Computing Evolutionary computation-based neural architecture search (ENAS) is a popular technique for automating architecture design of deep neural networks. Despite its groundbreaking applications, there is no theoretical study for ENAS. The expected hitting time (EHT) is one of the most important theoretical issues, since it implies the average computational time complexity. This paper proposes a general method by integrating theory and experiment for estimating the EHT of ENAS algorithms, which includes common configuration, search space partition, transition probability estimation, population distribution fitting, and hitting time analysis. By exploiting the proposed method, we consider the ($λ$+$λ$)-ENAS algorithms with different mutation operators and estimate the lower bounds of the EHT. Furthermore, we study the EHT on the NAS-Bench-101 problem, and the results demonstrate the validity of the proposed method. To the best of our knowledge, this work is the first attempt to establish a theoretical foundation for ENAS algorithms. |
| title | Analyzing the Expected Hitting Time of Evolutionary Computation-based Neural Architecture Search Algorithms |
| topic | Neural and Evolutionary Computing |
| url | https://arxiv.org/abs/2210.05397 |