Online Posterior Sampling with a Diffusion Prior
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914330107183104 |
|---|---|
| author | Kveton, Branislav Oreshkin, Boris Park, Youngsuk Deshmukh, Aniket Song, Rui |
| author_facet | Kveton, Branislav Oreshkin, Boris Park, Youngsuk Deshmukh, Aniket Song, Rui |
| contents | Posterior sampling in contextual bandits with a Gaussian prior can be implemented exactly or approximately using the Laplace approximation. The Gaussian prior is computationally efficient but it cannot describe complex distributions. In this work, we propose approximate posterior sampling algorithms for contextual bandits with a diffusion model prior. The key idea is to sample from a chain of approximate conditional posteriors, one for each stage of the reverse diffusion process, which are obtained by the Laplace approximation. Our approximations are motivated by posterior sampling with a Gaussian prior, and inherit its simplicity and efficiency. They are asymptotically consistent and perform well empirically on a variety of contextual bandit problems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_03919 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Online Posterior Sampling with a Diffusion Prior Kveton, Branislav Oreshkin, Boris Park, Youngsuk Deshmukh, Aniket Song, Rui Machine Learning Posterior sampling in contextual bandits with a Gaussian prior can be implemented exactly or approximately using the Laplace approximation. The Gaussian prior is computationally efficient but it cannot describe complex distributions. In this work, we propose approximate posterior sampling algorithms for contextual bandits with a diffusion model prior. The key idea is to sample from a chain of approximate conditional posteriors, one for each stage of the reverse diffusion process, which are obtained by the Laplace approximation. Our approximations are motivated by posterior sampling with a Gaussian prior, and inherit its simplicity and efficiency. They are asymptotically consistent and perform well empirically on a variety of contextual bandit problems. |
| title | Online Posterior Sampling with a Diffusion Prior |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2410.03919 |