Sample-Efficient Bayesian Optimization with Transfer Learning for Heterogeneous Search Spaces
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917771592335360 |
|---|---|
| author | Deshwal, Aryan Cakmak, Sait Xia, Yuhou Eriksson, David |
| author_facet | Deshwal, Aryan Cakmak, Sait Xia, Yuhou Eriksson, David |
| contents | Bayesian optimization (BO) is a powerful approach to sample-efficient optimization of black-box functions. However, in settings with very few function evaluations, a successful application of BO may require transferring information from historical experiments. These related experiments may not have exactly the same tunable parameters (search spaces), motivating the need for BO with transfer learning for heterogeneous search spaces. In this paper, we propose two methods for this setting. The first approach leverages a Gaussian process (GP) model with a conditional kernel to transfer information between different search spaces. Our second approach treats the missing parameters as hyperparameters of the GP model that can be inferred jointly with the other GP hyperparameters or set to fixed values. We show that these two methods perform well on several benchmark problems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_05325 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Sample-Efficient Bayesian Optimization with Transfer Learning for Heterogeneous Search Spaces Deshwal, Aryan Cakmak, Sait Xia, Yuhou Eriksson, David Machine Learning Artificial Intelligence Bayesian optimization (BO) is a powerful approach to sample-efficient optimization of black-box functions. However, in settings with very few function evaluations, a successful application of BO may require transferring information from historical experiments. These related experiments may not have exactly the same tunable parameters (search spaces), motivating the need for BO with transfer learning for heterogeneous search spaces. In this paper, we propose two methods for this setting. The first approach leverages a Gaussian process (GP) model with a conditional kernel to transfer information between different search spaces. Our second approach treats the missing parameters as hyperparameters of the GP model that can be inferred jointly with the other GP hyperparameters or set to fixed values. We show that these two methods perform well on several benchmark problems. |
| title | Sample-Efficient Bayesian Optimization with Transfer Learning for Heterogeneous Search Spaces |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2409.05325 |