Bounding Evidence and Estimating Log-Likelihood in VAE
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2022
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866929741583351808 |
|---|---|
| author | Struski, Łukasz Mazur, Marcin Batorski, Paweł Spurek, Przemysław Tabor, Jacek |
| author_facet | Struski, Łukasz Mazur, Marcin Batorski, Paweł Spurek, Przemysław Tabor, Jacek |
| contents | Many crucial problems in deep learning and statistical inference are caused by a variational gap, i.e., a difference between model evidence (log-likelihood) and evidence lower bound (ELBO). In particular, in a classical VAE setting that involves training via an ELBO cost function, it is difficult to provide a robust comparison of the effects of training between models, since we do not know a log-likelihood of data (but only its lower bound). In this paper, to deal with this problem, we introduce a general and effective upper bound, which allows us to efficiently approximate the evidence of data. We provide extensive theoretical and experimental studies of our approach, including its comparison to the other state-of-the-art upper bounds, as well as its application as a tool for the evaluation of models that were trained on various lower bounds. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2206_09453 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | Bounding Evidence and Estimating Log-Likelihood in VAE Struski, Łukasz Mazur, Marcin Batorski, Paweł Spurek, Przemysław Tabor, Jacek Machine Learning Artificial Intelligence Many crucial problems in deep learning and statistical inference are caused by a variational gap, i.e., a difference between model evidence (log-likelihood) and evidence lower bound (ELBO). In particular, in a classical VAE setting that involves training via an ELBO cost function, it is difficult to provide a robust comparison of the effects of training between models, since we do not know a log-likelihood of data (but only its lower bound). In this paper, to deal with this problem, we introduce a general and effective upper bound, which allows us to efficiently approximate the evidence of data. We provide extensive theoretical and experimental studies of our approach, including its comparison to the other state-of-the-art upper bounds, as well as its application as a tool for the evaluation of models that were trained on various lower bounds. |
| title | Bounding Evidence and Estimating Log-Likelihood in VAE |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2206.09453 |