Multi-Objective Hyperparameter Optimization in Machine Learning -- An Overview
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2022
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913379077062656 |
|---|---|
| author | Karl, Florian Pielok, Tobias Moosbauer, Julia Pfisterer, Florian Coors, Stefan Binder, Martin Schneider, Lennart Thomas, Janek Richter, Jakob Lang, Michel Garrido-Merchán, Eduardo C. Branke, Juergen Bischl, Bernd |
| author_facet | Karl, Florian Pielok, Tobias Moosbauer, Julia Pfisterer, Florian Coors, Stefan Binder, Martin Schneider, Lennart Thomas, Janek Richter, Jakob Lang, Michel Garrido-Merchán, Eduardo C. Branke, Juergen Bischl, Bernd |
| contents | Hyperparameter optimization constitutes a large part of typical modern machine learning workflows. This arises from the fact that machine learning methods and corresponding preprocessing steps often only yield optimal performance when hyperparameters are properly tuned. But in many applications, we are not only interested in optimizing ML pipelines solely for predictive accuracy; additional metrics or constraints must be considered when determining an optimal configuration, resulting in a multi-objective optimization problem. This is often neglected in practice, due to a lack of knowledge and readily available software implementations for multi-objective hyperparameter optimization. In this work, we introduce the reader to the basics of multi-objective hyperparameter optimization and motivate its usefulness in applied ML. Furthermore, we provide an extensive survey of existing optimization strategies, both from the domain of evolutionary algorithms and Bayesian optimization. We illustrate the utility of MOO in several specific ML applications, considering objectives such as operating conditions, prediction time, sparseness, fairness, interpretability and robustness. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2206_07438 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | Multi-Objective Hyperparameter Optimization in Machine Learning -- An Overview Karl, Florian Pielok, Tobias Moosbauer, Julia Pfisterer, Florian Coors, Stefan Binder, Martin Schneider, Lennart Thomas, Janek Richter, Jakob Lang, Michel Garrido-Merchán, Eduardo C. Branke, Juergen Bischl, Bernd Machine Learning Hyperparameter optimization constitutes a large part of typical modern machine learning workflows. This arises from the fact that machine learning methods and corresponding preprocessing steps often only yield optimal performance when hyperparameters are properly tuned. But in many applications, we are not only interested in optimizing ML pipelines solely for predictive accuracy; additional metrics or constraints must be considered when determining an optimal configuration, resulting in a multi-objective optimization problem. This is often neglected in practice, due to a lack of knowledge and readily available software implementations for multi-objective hyperparameter optimization. In this work, we introduce the reader to the basics of multi-objective hyperparameter optimization and motivate its usefulness in applied ML. Furthermore, we provide an extensive survey of existing optimization strategies, both from the domain of evolutionary algorithms and Bayesian optimization. We illustrate the utility of MOO in several specific ML applications, considering objectives such as operating conditions, prediction time, sparseness, fairness, interpretability and robustness. |
| title | Multi-Objective Hyperparameter Optimization in Machine Learning -- An Overview |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2206.07438 |