Fuse It or Lose It: Deep Fusion for Multimodal Simulation-Based Inference
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912103785299968 |
|---|---|
| author | Schmitt, Marvin Odole, Leona Radev, Stefan T. Bürkner, Paul-Christian |
| author_facet | Schmitt, Marvin Odole, Leona Radev, Stefan T. Bürkner, Paul-Christian |
| contents | We present multimodal neural posterior estimation (MultiNPE), a method to integrate heterogeneous data from different sources in simulation-based inference with neural networks. Inspired by advances in deep fusion, it allows researchers to analyze data from different domains and infer the parameters of complex mathematical models with increased accuracy. We consider three fusion approaches for MultiNPE (early, late, hybrid) and evaluate their performance in three challenging experiments. MultiNPE not only outperforms single-source baselines on a reference task, but also achieves superior inference on scientific models from cognitive neuroscience and cardiology. We systematically investigate the impact of partially missing data on the different fusion strategies. Across our experiments, late and hybrid fusion techniques emerge as the methods of choice for practical applications of multimodal simulation-based inference. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2311_10671 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Fuse It or Lose It: Deep Fusion for Multimodal Simulation-Based Inference Schmitt, Marvin Odole, Leona Radev, Stefan T. Bürkner, Paul-Christian Machine Learning Artificial Intelligence We present multimodal neural posterior estimation (MultiNPE), a method to integrate heterogeneous data from different sources in simulation-based inference with neural networks. Inspired by advances in deep fusion, it allows researchers to analyze data from different domains and infer the parameters of complex mathematical models with increased accuracy. We consider three fusion approaches for MultiNPE (early, late, hybrid) and evaluate their performance in three challenging experiments. MultiNPE not only outperforms single-source baselines on a reference task, but also achieves superior inference on scientific models from cognitive neuroscience and cardiology. We systematically investigate the impact of partially missing data on the different fusion strategies. Across our experiments, late and hybrid fusion techniques emerge as the methods of choice for practical applications of multimodal simulation-based inference. |
| title | Fuse It or Lose It: Deep Fusion for Multimodal Simulation-Based Inference |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2311.10671 |