Lifting Architectural Constraints of Injective Flows
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914849487847424 |
|---|---|
| author | Sorrenson, Peter Draxler, Felix Rousselot, Armand Hummerich, Sander Zimmermann, Lea Köthe, Ullrich |
| author_facet | Sorrenson, Peter Draxler, Felix Rousselot, Armand Hummerich, Sander Zimmermann, Lea Köthe, Ullrich |
| contents | Normalizing Flows explicitly maximize a full-dimensional likelihood on the training data. However, real data is typically only supported on a lower-dimensional manifold leading the model to expend significant compute on modeling noise. Injective Flows fix this by jointly learning a manifold and the distribution on it. So far, they have been limited by restrictive architectures and/or high computational cost. We lift both constraints by a new efficient estimator for the maximum likelihood loss, compatible with free-form bottleneck architectures. We further show that naively learning both the data manifold and the distribution on it can lead to divergent solutions, and use this insight to motivate a stable maximum likelihood training objective. We perform extensive experiments on toy, tabular and image data, demonstrating the competitive performance of the resulting model. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2306_01843 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Lifting Architectural Constraints of Injective Flows Sorrenson, Peter Draxler, Felix Rousselot, Armand Hummerich, Sander Zimmermann, Lea Köthe, Ullrich Machine Learning Normalizing Flows explicitly maximize a full-dimensional likelihood on the training data. However, real data is typically only supported on a lower-dimensional manifold leading the model to expend significant compute on modeling noise. Injective Flows fix this by jointly learning a manifold and the distribution on it. So far, they have been limited by restrictive architectures and/or high computational cost. We lift both constraints by a new efficient estimator for the maximum likelihood loss, compatible with free-form bottleneck architectures. We further show that naively learning both the data manifold and the distribution on it can lead to divergent solutions, and use this insight to motivate a stable maximum likelihood training objective. We perform extensive experiments on toy, tabular and image data, demonstrating the competitive performance of the resulting model. |
| title | Lifting Architectural Constraints of Injective Flows |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2306.01843 |