VISA: Variational Inference with Sequential Sample-Average Approximations
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911798385442816 |
|---|---|
| author | Zimmermann, Heiko Naesseth, Christian A. van de Meent, Jan-Willem |
| author_facet | Zimmermann, Heiko Naesseth, Christian A. van de Meent, Jan-Willem |
| contents | We present variational inference with sequential sample-average approximation (VISA), a method for approximate inference in computationally intensive models, such as those based on numerical simulations. VISA extends importance-weighted forward-KL variational inference by employing a sequence of sample-average approximations, which are considered valid inside a trust region. This makes it possible to reuse model evaluations across multiple gradient steps, thereby reducing computational cost. We perform experiments on high-dimensional Gaussians, Lotka-Volterra dynamics, and a Pickover attractor, which demonstrate that VISA can achieve comparable approximation accuracy to standard importance-weighted forward-KL variational inference with computational savings of a factor two or more for conservatively chosen learning rates. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_09429 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | VISA: Variational Inference with Sequential Sample-Average Approximations Zimmermann, Heiko Naesseth, Christian A. van de Meent, Jan-Willem Machine Learning We present variational inference with sequential sample-average approximation (VISA), a method for approximate inference in computationally intensive models, such as those based on numerical simulations. VISA extends importance-weighted forward-KL variational inference by employing a sequence of sample-average approximations, which are considered valid inside a trust region. This makes it possible to reuse model evaluations across multiple gradient steps, thereby reducing computational cost. We perform experiments on high-dimensional Gaussians, Lotka-Volterra dynamics, and a Pickover attractor, which demonstrate that VISA can achieve comparable approximation accuracy to standard importance-weighted forward-KL variational inference with computational savings of a factor two or more for conservatively chosen learning rates. |
| title | VISA: Variational Inference with Sequential Sample-Average Approximations |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2403.09429 |