Fast and Unified Path Gradient Estimators for Normalizing Flows
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916173273104384 |
|---|---|
| author | Vaitl, Lorenz Winkler, Ludwig Richter, Lorenz Kessel, Pan |
| author_facet | Vaitl, Lorenz Winkler, Ludwig Richter, Lorenz Kessel, Pan |
| contents | Recent work shows that path gradient estimators for normalizing flows have lower variance compared to standard estimators for variational inference, resulting in improved training. However, they are often prohibitively more expensive from a computational point of view and cannot be applied to maximum likelihood training in a scalable manner, which severely hinders their widespread adoption. In this work, we overcome these crucial limitations. Specifically, we propose a fast path gradient estimator which improves computational efficiency significantly and works for all normalizing flow architectures of practical relevance. We then show that this estimator can also be applied to maximum likelihood training for which it has a regularizing effect as it can take the form of a given target energy function into account. We empirically establish its superior performance and reduced variance for several natural sciences applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_15881 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Fast and Unified Path Gradient Estimators for Normalizing Flows Vaitl, Lorenz Winkler, Ludwig Richter, Lorenz Kessel, Pan Machine Learning Recent work shows that path gradient estimators for normalizing flows have lower variance compared to standard estimators for variational inference, resulting in improved training. However, they are often prohibitively more expensive from a computational point of view and cannot be applied to maximum likelihood training in a scalable manner, which severely hinders their widespread adoption. In this work, we overcome these crucial limitations. Specifically, we propose a fast path gradient estimator which improves computational efficiency significantly and works for all normalizing flow architectures of practical relevance. We then show that this estimator can also be applied to maximum likelihood training for which it has a regularizing effect as it can take the form of a given target energy function into account. We empirically establish its superior performance and reduced variance for several natural sciences applications. |
| title | Fast and Unified Path Gradient Estimators for Normalizing Flows |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2403.15881 |