Global convergence of optimized adaptive importance samplers
Fuente:
arXiv
Guardado en:
| Autor principal: | |
|---|---|
| Formato: | Preprint |
| Publicado: |
2022
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909083856011264 |
|---|---|
| author | Akyildiz, Ömer Deniz |
| author_facet | Akyildiz, Ömer Deniz |
| contents | We analyze the optimized adaptive importance sampler (OAIS) for performing Monte Carlo integration with general proposals. We leverage a classical result which shows that the bias and the mean-squared error (MSE) of the importance sampling scales with the $χ^2$-divergence between the target and the proposal and develop a scheme which performs global optimization of $χ^2$-divergence. While it is known that this quantity is convex for exponential family proposals, the case of the general proposals has been an open problem. We close this gap by utilizing the nonasymptotic bounds for stochastic gradient Langevin dynamics (SGLD) for the global optimization of $χ^2$-divergence and derive nonasymptotic bounds for the MSE by leveraging recent results from non-convex optimization literature. The resulting AIS schemes have explicit theoretical guarantees that are uniform-in-time. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2201_00409 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | Global convergence of optimized adaptive importance samplers Akyildiz, Ömer Deniz Computation Methodology Machine Learning We analyze the optimized adaptive importance sampler (OAIS) for performing Monte Carlo integration with general proposals. We leverage a classical result which shows that the bias and the mean-squared error (MSE) of the importance sampling scales with the $χ^2$-divergence between the target and the proposal and develop a scheme which performs global optimization of $χ^2$-divergence. While it is known that this quantity is convex for exponential family proposals, the case of the general proposals has been an open problem. We close this gap by utilizing the nonasymptotic bounds for stochastic gradient Langevin dynamics (SGLD) for the global optimization of $χ^2$-divergence and derive nonasymptotic bounds for the MSE by leveraging recent results from non-convex optimization literature. The resulting AIS schemes have explicit theoretical guarantees that are uniform-in-time. |
| title | Global convergence of optimized adaptive importance samplers |
| topic | Computation Methodology Machine Learning |
| url | https://arxiv.org/abs/2201.00409 |