Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2022
|
| Materias: | |
| Acceso en línea: | https://arxiv.org/abs/2212.14084 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912962424340480 |
|---|---|
| author | Guarrasi, Valerio Tronchin, Lorenzo Albano, Domenico Faiella, Eliodoro Fazzini, Deborah Santucci, Domiziana Soda, Paolo |
| author_facet | Guarrasi, Valerio Tronchin, Lorenzo Albano, Domenico Faiella, Eliodoro Fazzini, Deborah Santucci, Domiziana Soda, Paolo |
| contents | We are witnessing a widespread adoption of artificial intelligence in healthcare. However, most of the advancements in deep learning in this area consider only unimodal data, neglecting other modalities. Their multimodal interpretation necessary for supporting diagnosis, prognosis and treatment decisions. In this work we present a deep architecture, which jointly learns modality reconstructions and sample classifications using tabular and imaging data. The explanation of the decision taken is computed by applying a latent shift that, simulates a counterfactual prediction revealing the features of each modality that contribute the most to the decision and a quantitative score indicating the modality importance. We validate our approach in the context of COVID-19 pandemic using the AIforCOVID dataset, which contains multimodal data for the early identification of patients at risk of severe outcome. The results show that the proposed method provides meaningful explanations without degrading the classification performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2212_14084 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | Multimodal Explainability via Latent Shift applied to COVID-19 stratification Guarrasi, Valerio Tronchin, Lorenzo Albano, Domenico Faiella, Eliodoro Fazzini, Deborah Santucci, Domiziana Soda, Paolo Artificial Intelligence Machine Learning We are witnessing a widespread adoption of artificial intelligence in healthcare. However, most of the advancements in deep learning in this area consider only unimodal data, neglecting other modalities. Their multimodal interpretation necessary for supporting diagnosis, prognosis and treatment decisions. In this work we present a deep architecture, which jointly learns modality reconstructions and sample classifications using tabular and imaging data. The explanation of the decision taken is computed by applying a latent shift that, simulates a counterfactual prediction revealing the features of each modality that contribute the most to the decision and a quantitative score indicating the modality importance. We validate our approach in the context of COVID-19 pandemic using the AIforCOVID dataset, which contains multimodal data for the early identification of patients at risk of severe outcome. The results show that the proposed method provides meaningful explanations without degrading the classification performance. |
| title | Multimodal Explainability via Latent Shift applied to COVID-19 stratification |
| topic | Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2212.14084 |