On the Asymptotics of Importance Weighted Variational Inference
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909456958226432 |
|---|---|
| author | Cherief-Abdellatif, Badr-Eddine Douc, Randal Doucet, Arnaud Marival, Hugo |
| author_facet | Cherief-Abdellatif, Badr-Eddine Douc, Randal Doucet, Arnaud Marival, Hugo |
| contents | For complex latent variable models, the likelihood function is not available in closed form. In this context, a popular method to perform parameter estimation is Importance Weighted Variational Inference. It essentially maximizes the expectation of the logarithm of an importance sampling estimate of the likelihood with respect to both the latent variable model parameters and the importance distribution parameters, the expectation being itself with respect to the importance samples. Despite its great empirical success in machine learning, a theoretical analysis of the limit properties of the resulting estimates is still lacking. We fill this gap by establishing consistency when both the Monte Carlo and the observed data sample sizes go to infinity simultaneously. We also establish asymptotic normality and efficiency under additional conditions relating the rate of growth between the Monte Carlo and the observed data samples sizes. We distinguish several regimes related to the smoothness of the importance ratio. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_08477 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | On the Asymptotics of Importance Weighted Variational Inference Cherief-Abdellatif, Badr-Eddine Douc, Randal Doucet, Arnaud Marival, Hugo Statistics Theory 62F12, 62F30, 62B10 (Primary) 60F05, 68T07 (Secondary) G.3 For complex latent variable models, the likelihood function is not available in closed form. In this context, a popular method to perform parameter estimation is Importance Weighted Variational Inference. It essentially maximizes the expectation of the logarithm of an importance sampling estimate of the likelihood with respect to both the latent variable model parameters and the importance distribution parameters, the expectation being itself with respect to the importance samples. Despite its great empirical success in machine learning, a theoretical analysis of the limit properties of the resulting estimates is still lacking. We fill this gap by establishing consistency when both the Monte Carlo and the observed data sample sizes go to infinity simultaneously. We also establish asymptotic normality and efficiency under additional conditions relating the rate of growth between the Monte Carlo and the observed data samples sizes. We distinguish several regimes related to the smoothness of the importance ratio. |
| title | On the Asymptotics of Importance Weighted Variational Inference |
| topic | Statistics Theory 62F12, 62F30, 62B10 (Primary) 60F05, 68T07 (Secondary) G.3 |
| url | https://arxiv.org/abs/2501.08477 |