Scalable Expectation Estimation with Subtractive Mixture Models
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915216231497728 |
|---|---|
| author | Zellinger, Lena Branchini, Nicola Elvira, Víctor Vergari, Antonio |
| author_facet | Zellinger, Lena Branchini, Nicola Elvira, Víctor Vergari, Antonio |
| contents | Many Monte Carlo (MC) and importance sampling (IS) methods use mixture models (MMs) for their simplicity and ability to capture multimodal distributions. Recently, subtractive mixture models (SMMs), i.e. MMs with negative coefficients, have shown greater expressiveness and success in generative modeling. However, their negative parameters complicate sampling, requiring costly auto-regressive techniques or accept-reject algorithms that do not scale in high dimensions. In this work, we use the difference representation of SMMs to construct an unbiased IS estimator ($Δ\text{Ex}$) that removes the need to sample from the SMM, enabling high-dimensional expectation estimation with SMMs. In our experiments, we show that $Δ\text{Ex}$ can achieve comparable estimation quality to auto-regressive sampling while being considerably faster in MC estimation. Moreover, we conduct initial experiments with $Δ\text{Ex}$ using hand-crafted proposals, gaining first insights into how to construct safe proposals for $Δ\text{Ex}$. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_21346 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Scalable Expectation Estimation with Subtractive Mixture Models Zellinger, Lena Branchini, Nicola Elvira, Víctor Vergari, Antonio Machine Learning Computation Many Monte Carlo (MC) and importance sampling (IS) methods use mixture models (MMs) for their simplicity and ability to capture multimodal distributions. Recently, subtractive mixture models (SMMs), i.e. MMs with negative coefficients, have shown greater expressiveness and success in generative modeling. However, their negative parameters complicate sampling, requiring costly auto-regressive techniques or accept-reject algorithms that do not scale in high dimensions. In this work, we use the difference representation of SMMs to construct an unbiased IS estimator ($Δ\text{Ex}$) that removes the need to sample from the SMM, enabling high-dimensional expectation estimation with SMMs. In our experiments, we show that $Δ\text{Ex}$ can achieve comparable estimation quality to auto-regressive sampling while being considerably faster in MC estimation. Moreover, we conduct initial experiments with $Δ\text{Ex}$ using hand-crafted proposals, gaining first insights into how to construct safe proposals for $Δ\text{Ex}$. |
| title | Scalable Expectation Estimation with Subtractive Mixture Models |
| topic | Machine Learning Computation |
| url | https://arxiv.org/abs/2503.21346 |