We Still Don't Understand High-Dimensional Bayesian Optimization
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911577814335488 |
|---|---|
| author | Doumont, Colin Fan, Donney Maus, Natalie Gardner, Jacob R. Moss, Henry Pleiss, Geoff |
| author_facet | Doumont, Colin Fan, Donney Maus, Natalie Gardner, Jacob R. Moss, Henry Pleiss, Geoff |
| contents | Existing high-dimensional Bayesian optimization (BO) methods aim to overcome the curse of dimensionality by carefully encoding structural assumptions, from locality to sparsity to smoothness, into the optimization procedure. Surprisingly, we demonstrate that these approaches are outperformed by arguably the simplest method imaginable: Bayesian linear regression. After applying a geometric transformation to avoid boundary-seeking behavior, Gaussian processes with linear kernels match state-of-the-art performance on tasks with 60- to 6,000-dimensional search spaces. Linear models offer numerous advantages over their non-parametric counterparts: they afford closed-form sampling and their computation scales linearly with data, a fact we exploit on molecular optimization tasks with >20,000 observations. Coupled with empirical analyses, our results suggest the need to depart from past intuitions about BO methods in high-dimensions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_00170 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | We Still Don't Understand High-Dimensional Bayesian Optimization Doumont, Colin Fan, Donney Maus, Natalie Gardner, Jacob R. Moss, Henry Pleiss, Geoff Machine Learning Existing high-dimensional Bayesian optimization (BO) methods aim to overcome the curse of dimensionality by carefully encoding structural assumptions, from locality to sparsity to smoothness, into the optimization procedure. Surprisingly, we demonstrate that these approaches are outperformed by arguably the simplest method imaginable: Bayesian linear regression. After applying a geometric transformation to avoid boundary-seeking behavior, Gaussian processes with linear kernels match state-of-the-art performance on tasks with 60- to 6,000-dimensional search spaces. Linear models offer numerous advantages over their non-parametric counterparts: they afford closed-form sampling and their computation scales linearly with data, a fact we exploit on molecular optimization tasks with >20,000 observations. Coupled with empirical analyses, our results suggest the need to depart from past intuitions about BO methods in high-dimensions. |
| title | We Still Don't Understand High-Dimensional Bayesian Optimization |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2512.00170 |