Accelerating Look-ahead in Bayesian Optimization: Multilevel Monte Carlo is All you Need
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866929443530866688 |
|---|---|
| author | Yang, Shangda Zankin, Vitaly Balandat, Maximilian Scherer, Stefan Carlberg, Kevin Walton, Neil Law, Kody J. H. |
| author_facet | Yang, Shangda Zankin, Vitaly Balandat, Maximilian Scherer, Stefan Carlberg, Kevin Walton, Neil Law, Kody J. H. |
| contents | We leverage multilevel Monte Carlo (MLMC) to improve the performance of multi-step look-ahead Bayesian optimization (BO) methods that involve nested expectations and maximizations. Often these expectations must be computed by Monte Carlo (MC). The complexity rate of naive MC degrades for nested operations, whereas MLMC is capable of achieving the canonical MC convergence rate for this type of problem, independently of dimension and without any smoothness assumptions. Our theoretical study focuses on the approximation improvements for twoand three-step look-ahead acquisition functions, but, as we discuss, the approach is generalizable in various ways, including beyond the context of BO. Our findings are verified numerically and the benefits of MLMC for BO are illustrated on several benchmark examples. Code is available at https://github.com/Shangda-Yang/MLMCBO . |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_02111 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Accelerating Look-ahead in Bayesian Optimization: Multilevel Monte Carlo is All you Need Yang, Shangda Zankin, Vitaly Balandat, Maximilian Scherer, Stefan Carlberg, Kevin Walton, Neil Law, Kody J. H. Machine Learning Optimization and Control Probability Computation Methodology We leverage multilevel Monte Carlo (MLMC) to improve the performance of multi-step look-ahead Bayesian optimization (BO) methods that involve nested expectations and maximizations. Often these expectations must be computed by Monte Carlo (MC). The complexity rate of naive MC degrades for nested operations, whereas MLMC is capable of achieving the canonical MC convergence rate for this type of problem, independently of dimension and without any smoothness assumptions. Our theoretical study focuses on the approximation improvements for twoand three-step look-ahead acquisition functions, but, as we discuss, the approach is generalizable in various ways, including beyond the context of BO. Our findings are verified numerically and the benefits of MLMC for BO are illustrated on several benchmark examples. Code is available at https://github.com/Shangda-Yang/MLMCBO . |
| title | Accelerating Look-ahead in Bayesian Optimization: Multilevel Monte Carlo is All you Need |
| topic | Machine Learning Optimization and Control Probability Computation Methodology |
| url | https://arxiv.org/abs/2402.02111 |