Subtractive Mixture Models via Squaring: Representation and Learning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910424935432192 |
|---|---|
| author | Loconte, Lorenzo Sladek, Aleksanteri M. Mengel, Stefan Trapp, Martin Solin, Arno Gillis, Nicolas Vergari, Antonio |
| author_facet | Loconte, Lorenzo Sladek, Aleksanteri M. Mengel, Stefan Trapp, Martin Solin, Arno Gillis, Nicolas Vergari, Antonio |
| contents | Mixture models are traditionally represented and learned by adding several distributions as components. Allowing mixtures to subtract probability mass or density can drastically reduce the number of components needed to model complex distributions. However, learning such subtractive mixtures while ensuring they still encode a non-negative function is challenging. We investigate how to learn and perform inference on deep subtractive mixtures by squaring them. We do this in the framework of probabilistic circuits, which enable us to represent tensorized mixtures and generalize several other subtractive models. We theoretically prove that the class of squared circuits allowing subtractions can be exponentially more expressive than traditional additive mixtures; and, we empirically show this increased expressiveness on a series of real-world distribution estimation tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_00724 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Subtractive Mixture Models via Squaring: Representation and Learning Loconte, Lorenzo Sladek, Aleksanteri M. Mengel, Stefan Trapp, Martin Solin, Arno Gillis, Nicolas Vergari, Antonio Machine Learning Artificial Intelligence Mixture models are traditionally represented and learned by adding several distributions as components. Allowing mixtures to subtract probability mass or density can drastically reduce the number of components needed to model complex distributions. However, learning such subtractive mixtures while ensuring they still encode a non-negative function is challenging. We investigate how to learn and perform inference on deep subtractive mixtures by squaring them. We do this in the framework of probabilistic circuits, which enable us to represent tensorized mixtures and generalize several other subtractive models. We theoretically prove that the class of squared circuits allowing subtractions can be exponentially more expressive than traditional additive mixtures; and, we empirically show this increased expressiveness on a series of real-world distribution estimation tasks. |
| title | Subtractive Mixture Models via Squaring: Representation and Learning |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2310.00724 |