Tri-VQA: Triangular Reasoning Medical Visual Question Answering for Multi-Attribute Analysis
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911928728682496 |
|---|---|
| author | Fan, Lin Gong, Xun Zheng, Cenyang Ou, Yafei |
| author_facet | Fan, Lin Gong, Xun Zheng, Cenyang Ou, Yafei |
| contents | The intersection of medical Visual Question Answering (Med-VQA) is a challenging research topic with advantages including patient engagement and clinical expert involvement for second opinions. However, existing Med-VQA methods based on joint embedding fail to explain whether their provided results are based on correct reasoning or coincidental answers, which undermines the credibility of VQA answers. In this paper, we investigate the construction of a more cohesive and stable Med-VQA structure. Motivated by causal effect, we propose a novel Triangular Reasoning VQA (Tri-VQA) framework, which constructs reverse causal questions from the perspective of "Why this answer?" to elucidate the source of the answer and stimulate more reasonable forward reasoning processes. We evaluate our method on the Endoscopic Ultrasound (EUS) multi-attribute annotated dataset from five centers, and test it on medical VQA datasets. Experimental results demonstrate the superiority of our approach over existing methods. Our codes and pre-trained models are available at https://anonymous.4open.science/r/Tri_VQA. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_15050 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Tri-VQA: Triangular Reasoning Medical Visual Question Answering for Multi-Attribute Analysis Fan, Lin Gong, Xun Zheng, Cenyang Ou, Yafei Machine Learning Artificial Intelligence Computation and Language Computer Vision and Pattern Recognition I.2.7; I.2.10; J.3 The intersection of medical Visual Question Answering (Med-VQA) is a challenging research topic with advantages including patient engagement and clinical expert involvement for second opinions. However, existing Med-VQA methods based on joint embedding fail to explain whether their provided results are based on correct reasoning or coincidental answers, which undermines the credibility of VQA answers. In this paper, we investigate the construction of a more cohesive and stable Med-VQA structure. Motivated by causal effect, we propose a novel Triangular Reasoning VQA (Tri-VQA) framework, which constructs reverse causal questions from the perspective of "Why this answer?" to elucidate the source of the answer and stimulate more reasonable forward reasoning processes. We evaluate our method on the Endoscopic Ultrasound (EUS) multi-attribute annotated dataset from five centers, and test it on medical VQA datasets. Experimental results demonstrate the superiority of our approach over existing methods. Our codes and pre-trained models are available at https://anonymous.4open.science/r/Tri_VQA. |
| title | Tri-VQA: Triangular Reasoning Medical Visual Question Answering for Multi-Attribute Analysis |
| topic | Machine Learning Artificial Intelligence Computation and Language Computer Vision and Pattern Recognition I.2.7; I.2.10; J.3 |
| url | https://arxiv.org/abs/2406.15050 |