Amortized Probabilistic Conditioning for Optimization, Simulation and Inference
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909525146075136 |
|---|---|
| author | Chang, Paul E. Loka, Nasrulloh Huang, Daolang Remes, Ulpu Kaski, Samuel Acerbi, Luigi |
| author_facet | Chang, Paul E. Loka, Nasrulloh Huang, Daolang Remes, Ulpu Kaski, Samuel Acerbi, Luigi |
| contents | Amortized meta-learning methods based on pre-training have propelled fields like natural language processing and vision. Transformer-based neural processes and their variants are leading models for probabilistic meta-learning with a tractable objective. Often trained on synthetic data, these models implicitly capture essential latent information in the data-generation process. However, existing methods do not allow users to flexibly inject (condition on) and extract (predict) this probabilistic latent information at runtime, which is key to many tasks. We introduce the Amortized Conditioning Engine (ACE), a new transformer-based meta-learning model that explicitly represents latent variables of interest. ACE affords conditioning on both observed data and interpretable latent variables, the inclusion of priors at runtime, and outputs predictive distributions for discrete and continuous data and latents. We show ACE's modeling flexibility and performance in diverse tasks such as image completion and classification, Bayesian optimization, and simulation-based inference. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_15320 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Amortized Probabilistic Conditioning for Optimization, Simulation and Inference Chang, Paul E. Loka, Nasrulloh Huang, Daolang Remes, Ulpu Kaski, Samuel Acerbi, Luigi Machine Learning Amortized meta-learning methods based on pre-training have propelled fields like natural language processing and vision. Transformer-based neural processes and their variants are leading models for probabilistic meta-learning with a tractable objective. Often trained on synthetic data, these models implicitly capture essential latent information in the data-generation process. However, existing methods do not allow users to flexibly inject (condition on) and extract (predict) this probabilistic latent information at runtime, which is key to many tasks. We introduce the Amortized Conditioning Engine (ACE), a new transformer-based meta-learning model that explicitly represents latent variables of interest. ACE affords conditioning on both observed data and interpretable latent variables, the inclusion of priors at runtime, and outputs predictive distributions for discrete and continuous data and latents. We show ACE's modeling flexibility and performance in diverse tasks such as image completion and classification, Bayesian optimization, and simulation-based inference. |
| title | Amortized Probabilistic Conditioning for Optimization, Simulation and Inference |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2410.15320 |