Guardado en:
| Autores principales: | , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2020
|
| Materias: | |
| Acceso en línea: | https://arxiv.org/abs/2004.06231 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866915580117778432 |
|---|---|
| author | Peharz, Robert Lang, Steven Vergari, Antonio Stelzner, Karl Molina, Alejandro Trapp, Martin Broeck, Guy Van den Kersting, Kristian Ghahramani, Zoubin |
| author_facet | Peharz, Robert Lang, Steven Vergari, Antonio Stelzner, Karl Molina, Alejandro Trapp, Martin Broeck, Guy Van den Kersting, Kristian Ghahramani, Zoubin |
| contents | Probabilistic circuits (PCs) are a promising avenue for probabilistic modeling, as they permit a wide range of exact and efficient inference routines. Recent ``deep-learning-style'' implementations of PCs strive for a better scalability, but are still difficult to train on real-world data, due to their sparsely connected computational graphs. In this paper, we propose Einsum Networks (EiNets), a novel implementation design for PCs, improving prior art in several regards. At their core, EiNets combine a large number of arithmetic operations in a single monolithic einsum-operation, leading to speedups and memory savings of up to two orders of magnitude, in comparison to previous implementations. As an algorithmic contribution, we show that the implementation of Expectation-Maximization (EM) can be simplified for PCs, by leveraging automatic differentiation. Furthermore, we demonstrate that EiNets scale well to datasets which were previously out of reach, such as SVHN and CelebA, and that they can be used as faithful generative image models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2004_06231 |
| institution | arXiv |
| publishDate | 2020 |
| record_format | arxiv |
| spellingShingle | Einsum Networks: Fast and Scalable Learning of Tractable Probabilistic Circuits Peharz, Robert Lang, Steven Vergari, Antonio Stelzner, Karl Molina, Alejandro Trapp, Martin Broeck, Guy Van den Kersting, Kristian Ghahramani, Zoubin Machine Learning Probabilistic circuits (PCs) are a promising avenue for probabilistic modeling, as they permit a wide range of exact and efficient inference routines. Recent ``deep-learning-style'' implementations of PCs strive for a better scalability, but are still difficult to train on real-world data, due to their sparsely connected computational graphs. In this paper, we propose Einsum Networks (EiNets), a novel implementation design for PCs, improving prior art in several regards. At their core, EiNets combine a large number of arithmetic operations in a single monolithic einsum-operation, leading to speedups and memory savings of up to two orders of magnitude, in comparison to previous implementations. As an algorithmic contribution, we show that the implementation of Expectation-Maximization (EM) can be simplified for PCs, by leveraging automatic differentiation. Furthermore, we demonstrate that EiNets scale well to datasets which were previously out of reach, such as SVHN and CelebA, and that they can be used as faithful generative image models. |
| title | Einsum Networks: Fast and Scalable Learning of Tractable Probabilistic Circuits |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2004.06231 |