An investigation on the use of Large Language Models for hyperparameter tuning in Evolutionary Algorithms
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914900653113344 |
|---|---|
| author | Custode, Leonardo Lucio Caraffini, Fabio Yaman, Anil Iacca, Giovanni |
| author_facet | Custode, Leonardo Lucio Caraffini, Fabio Yaman, Anil Iacca, Giovanni |
| contents | Hyperparameter optimization is a crucial problem in Evolutionary Computation. In fact, the values of the hyperparameters directly impact the trajectory taken by the optimization process, and their choice requires extensive reasoning by human operators. Although a variety of self-adaptive Evolutionary Algorithms have been proposed in the literature, no definitive solution has been found. In this work, we perform a preliminary investigation to automate the reasoning process that leads to the choice of hyperparameter values. We employ two open-source Large Language Models (LLMs), namely Llama2-70b and Mixtral, to analyze the optimization logs online and provide novel real-time hyperparameter recommendations. We study our approach in the context of step-size adaptation for (1+1)-ES. The results suggest that LLMs can be an effective method for optimizing hyperparameters in Evolution Strategies, encouraging further research in this direction. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_02451 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | An investigation on the use of Large Language Models for hyperparameter tuning in Evolutionary Algorithms Custode, Leonardo Lucio Caraffini, Fabio Yaman, Anil Iacca, Giovanni Neural and Evolutionary Computing Hyperparameter optimization is a crucial problem in Evolutionary Computation. In fact, the values of the hyperparameters directly impact the trajectory taken by the optimization process, and their choice requires extensive reasoning by human operators. Although a variety of self-adaptive Evolutionary Algorithms have been proposed in the literature, no definitive solution has been found. In this work, we perform a preliminary investigation to automate the reasoning process that leads to the choice of hyperparameter values. We employ two open-source Large Language Models (LLMs), namely Llama2-70b and Mixtral, to analyze the optimization logs online and provide novel real-time hyperparameter recommendations. We study our approach in the context of step-size adaptation for (1+1)-ES. The results suggest that LLMs can be an effective method for optimizing hyperparameters in Evolution Strategies, encouraging further research in this direction. |
| title | An investigation on the use of Large Language Models for hyperparameter tuning in Evolutionary Algorithms |
| topic | Neural and Evolutionary Computing |
| url | https://arxiv.org/abs/2408.02451 |