PE-MVCNet: Multi-view and Cross-modal Fusion Network for Pulmonary Embolism Prediction
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910412927139840 |
|---|---|
| author | Guo, Zhaoxin Wang, Zhipeng Ge, Ruiquan Yu, Jianxun Qin, Feiwei Tian, Yuan Peng, Yuqing Li, Yonghong Wang, Changmiao |
| author_facet | Guo, Zhaoxin Wang, Zhipeng Ge, Ruiquan Yu, Jianxun Qin, Feiwei Tian, Yuan Peng, Yuqing Li, Yonghong Wang, Changmiao |
| contents | The early detection of a pulmonary embolism (PE) is critical for enhancing patient survival rates. Both image-based and non-image-based features are of utmost importance in medical classification tasks. In a clinical setting, physicians tend to rely on the contextual information provided by Electronic Medical Records (EMR) to interpret medical imaging. However, very few models effectively integrate clinical information with imaging data. To address this shortcoming, we suggest a multimodal fusion methodology, termed PE-MVCNet, which capitalizes on Computed Tomography Pulmonary Angiography imaging and EMR data. This method comprises the Image-only module with an integrated multi-view block, the EMR-only module, and the Cross-modal Attention Fusion (CMAF) module. These modules cooperate to extract comprehensive features that subsequently generate predictions for PE. We conducted experiments using the publicly accessible Stanford University Medical Center dataset, achieving an AUROC of 94.1%, an accuracy rate of 90.2%, and an F1 score of 90.6%. Our proposed model outperforms existing methodologies, corroborating that our multimodal fusion model excels compared to models that use a single data modality. Our source code is available at https://github.com/LeavingStarW/PE-MVCNET. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_17187 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | PE-MVCNet: Multi-view and Cross-modal Fusion Network for Pulmonary Embolism Prediction Guo, Zhaoxin Wang, Zhipeng Ge, Ruiquan Yu, Jianxun Qin, Feiwei Tian, Yuan Peng, Yuqing Li, Yonghong Wang, Changmiao Image and Video Processing Computer Vision and Pattern Recognition The early detection of a pulmonary embolism (PE) is critical for enhancing patient survival rates. Both image-based and non-image-based features are of utmost importance in medical classification tasks. In a clinical setting, physicians tend to rely on the contextual information provided by Electronic Medical Records (EMR) to interpret medical imaging. However, very few models effectively integrate clinical information with imaging data. To address this shortcoming, we suggest a multimodal fusion methodology, termed PE-MVCNet, which capitalizes on Computed Tomography Pulmonary Angiography imaging and EMR data. This method comprises the Image-only module with an integrated multi-view block, the EMR-only module, and the Cross-modal Attention Fusion (CMAF) module. These modules cooperate to extract comprehensive features that subsequently generate predictions for PE. We conducted experiments using the publicly accessible Stanford University Medical Center dataset, achieving an AUROC of 94.1%, an accuracy rate of 90.2%, and an F1 score of 90.6%. Our proposed model outperforms existing methodologies, corroborating that our multimodal fusion model excels compared to models that use a single data modality. Our source code is available at https://github.com/LeavingStarW/PE-MVCNET. |
| title | PE-MVCNet: Multi-view and Cross-modal Fusion Network for Pulmonary Embolism Prediction |
| topic | Image and Video Processing Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2402.17187 |