Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2508.19319 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915465600696320 |
|---|---|
| author | Moradbeiki, Pardis Ghadiri, Nasser Zahabi, Sayed Jalal Wiil, Uffe Kock Brockhattingen, Kristoffer Kittelmann Ebrahimi, Ali |
| author_facet | Moradbeiki, Pardis Ghadiri, Nasser Zahabi, Sayed Jalal Wiil, Uffe Kock Brockhattingen, Kristoffer Kittelmann Ebrahimi, Ali |
| contents | Accurate sarcopenia diagnosis via ultrasound remains challenging due to subtle imaging cues, limited labeled data, and the absence of clinical context in most models. We propose MedVQA-TREE, a multimodal framework that integrates a hierarchical image interpretation module, a gated feature-level fusion mechanism, and a novel multi-hop, multi-query retrieval strategy. The vision module includes anatomical classification, region segmentation, and graph-based spatial reasoning to capture coarse, mid-level, and fine-grained structures. A gated fusion mechanism selectively integrates visual features with textual queries, while clinical knowledge is retrieved through a UMLS-guided pipeline accessing PubMed and a sarcopenia-specific external knowledge base. MedVQA-TREE was trained and evaluated on two public MedVQA datasets (VQA-RAD and PathVQA) and a custom sarcopenia ultrasound dataset. The model achieved up to 99% diagnostic accuracy and outperformed previous state-of-the-art methods by over 10%. These results underscore the benefit of combining structured visual understanding with guided knowledge retrieval for effective AI-assisted diagnosis in sarcopenia. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_19319 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MedVQA-TREE: A Multimodal Reasoning and Retrieval Framework for Sarcopenia Prediction Moradbeiki, Pardis Ghadiri, Nasser Zahabi, Sayed Jalal Wiil, Uffe Kock Brockhattingen, Kristoffer Kittelmann Ebrahimi, Ali Image and Video Processing Artificial Intelligence Computer Vision and Pattern Recognition Accurate sarcopenia diagnosis via ultrasound remains challenging due to subtle imaging cues, limited labeled data, and the absence of clinical context in most models. We propose MedVQA-TREE, a multimodal framework that integrates a hierarchical image interpretation module, a gated feature-level fusion mechanism, and a novel multi-hop, multi-query retrieval strategy. The vision module includes anatomical classification, region segmentation, and graph-based spatial reasoning to capture coarse, mid-level, and fine-grained structures. A gated fusion mechanism selectively integrates visual features with textual queries, while clinical knowledge is retrieved through a UMLS-guided pipeline accessing PubMed and a sarcopenia-specific external knowledge base. MedVQA-TREE was trained and evaluated on two public MedVQA datasets (VQA-RAD and PathVQA) and a custom sarcopenia ultrasound dataset. The model achieved up to 99% diagnostic accuracy and outperformed previous state-of-the-art methods by over 10%. These results underscore the benefit of combining structured visual understanding with guided knowledge retrieval for effective AI-assisted diagnosis in sarcopenia. |
| title | MedVQA-TREE: A Multimodal Reasoning and Retrieval Framework for Sarcopenia Prediction |
| topic | Image and Video Processing Artificial Intelligence Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2508.19319 |