Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2404.12634 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909392046129152 |
|---|---|
| author | Ma, Danqing Wang, Meng Xiang, Ao Qi, Zongqing Yang, Qin |
| author_facet | Ma, Danqing Wang, Meng Xiang, Ao Qi, Zongqing Yang, Qin |
| contents | This study proposes a multi-modal fusion framework Multitrans based on the Transformer architecture and self-attention mechanism. This architecture combines the study of non-contrast computed tomography (NCCT) images and discharge diagnosis reports of patients undergoing stroke treatment, using a variety of methods based on Transformer architecture approach to predicting functional outcomes of stroke treatment. The results show that the performance of single-modal text classification is significantly better than single-modal image classification, but the effect of multi-modal combination is better than any single modality. Although the Transformer model only performs worse on imaging data, when combined with clinical meta-diagnostic information, both can learn better complementary information and make good contributions to accurately predicting stroke treatment effects.. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_12634 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Transformer-Based Classification Outcome Prediction for Multimodal Stroke Treatment Ma, Danqing Wang, Meng Xiang, Ao Qi, Zongqing Yang, Qin Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning This study proposes a multi-modal fusion framework Multitrans based on the Transformer architecture and self-attention mechanism. This architecture combines the study of non-contrast computed tomography (NCCT) images and discharge diagnosis reports of patients undergoing stroke treatment, using a variety of methods based on Transformer architecture approach to predicting functional outcomes of stroke treatment. The results show that the performance of single-modal text classification is significantly better than single-modal image classification, but the effect of multi-modal combination is better than any single modality. Although the Transformer model only performs worse on imaging data, when combined with clinical meta-diagnostic information, both can learn better complementary information and make good contributions to accurately predicting stroke treatment effects.. |
| title | Transformer-Based Classification Outcome Prediction for Multimodal Stroke Treatment |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2404.12634 |