Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2402.05367 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911892312686592 |
|---|---|
| author | Xu, Wenjie Wang, Wenbin Jiang, Yuning Svetozarevic, Bratislav Jones, Colin N. |
| author_facet | Xu, Wenjie Wang, Wenbin Jiang, Yuning Svetozarevic, Bratislav Jones, Colin N. |
| contents | We study the problem of preferential Bayesian optimization (BO), where we aim to optimize a black-box function with only preference feedback over a pair of candidate solutions. Inspired by the likelihood ratio idea, we construct a confidence set of the black-box function using only the preference feedback. An optimistic algorithm with an efficient computational method is then developed to solve the problem, which enjoys an information-theoretic bound on the total cumulative regret, a first-of-its-kind for preferential BO. This bound further allows us to design a scheme to report an estimated best solution, with a guaranteed convergence rate. Experimental results on sampled instances from Gaussian processes, standard test functions, and a thermal comfort optimization problem all show that our method stably achieves better or competitive performance as compared to the existing state-of-the-art heuristics, which, however, do not have theoretical guarantees on regret bounds or convergence. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_05367 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Principled Preferential Bayesian Optimization Xu, Wenjie Wang, Wenbin Jiang, Yuning Svetozarevic, Bratislav Jones, Colin N. Machine Learning We study the problem of preferential Bayesian optimization (BO), where we aim to optimize a black-box function with only preference feedback over a pair of candidate solutions. Inspired by the likelihood ratio idea, we construct a confidence set of the black-box function using only the preference feedback. An optimistic algorithm with an efficient computational method is then developed to solve the problem, which enjoys an information-theoretic bound on the total cumulative regret, a first-of-its-kind for preferential BO. This bound further allows us to design a scheme to report an estimated best solution, with a guaranteed convergence rate. Experimental results on sampled instances from Gaussian processes, standard test functions, and a thermal comfort optimization problem all show that our method stably achieves better or competitive performance as compared to the existing state-of-the-art heuristics, which, however, do not have theoretical guarantees on regret bounds or convergence. |
| title | Principled Preferential Bayesian Optimization |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2402.05367 |