On the Universality of Volume-Preserving and Coupling-Based Normalizing Flows
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910803793281024 |
|---|---|
| author | Draxler, Felix Wahl, Stefan Schnörr, Christoph Köthe, Ullrich |
| author_facet | Draxler, Felix Wahl, Stefan Schnörr, Christoph Köthe, Ullrich |
| contents | We present a novel theoretical framework for understanding the expressive power of normalizing flows. Despite their prevalence in scientific applications, a comprehensive understanding of flows remains elusive due to their restricted architectures. Existing theorems fall short as they require the use of arbitrarily ill-conditioned neural networks, limiting practical applicability. We propose a distributional universality theorem for well-conditioned coupling-based normalizing flows such as RealNVP. In addition, we show that volume-preserving normalizing flows are not universal, what distribution they learn instead, and how to fix their expressivity. Our results support the general wisdom that affine and related couplings are expressive and in general outperform volume-preserving flows, bridging a gap between empirical results and theoretical understanding. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_06578 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | On the Universality of Volume-Preserving and Coupling-Based Normalizing Flows Draxler, Felix Wahl, Stefan Schnörr, Christoph Köthe, Ullrich Machine Learning We present a novel theoretical framework for understanding the expressive power of normalizing flows. Despite their prevalence in scientific applications, a comprehensive understanding of flows remains elusive due to their restricted architectures. Existing theorems fall short as they require the use of arbitrarily ill-conditioned neural networks, limiting practical applicability. We propose a distributional universality theorem for well-conditioned coupling-based normalizing flows such as RealNVP. In addition, we show that volume-preserving normalizing flows are not universal, what distribution they learn instead, and how to fix their expressivity. Our results support the general wisdom that affine and related couplings are expressive and in general outperform volume-preserving flows, bridging a gap between empirical results and theoretical understanding. |
| title | On the Universality of Volume-Preserving and Coupling-Based Normalizing Flows |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2402.06578 |