Free Form Medical Visual Question Answering in Radiology
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916103811235840 |
|---|---|
| author | Narayanan, Abhishek Musthyala, Rushabh Sankar, Rahul Nistala, Anirudh Prasad Singh, Pranav Cirrone, Jacopo |
| author_facet | Narayanan, Abhishek Musthyala, Rushabh Sankar, Rahul Nistala, Anirudh Prasad Singh, Pranav Cirrone, Jacopo |
| contents | Visual Question Answering (VQA) in the medical domain presents a unique, interdisciplinary challenge, combining fields such as Computer Vision, Natural Language Processing, and Knowledge Representation. Despite its importance, research in medical VQA has been scant, only gaining momentum since 2018. Addressing this gap, our research delves into the effective representation of radiology images and the joint learning of multimodal representations, surpassing existing methods. We innovatively augment the SLAKE dataset, enabling our model to respond to a more diverse array of questions, not limited to the immediate content of radiology or pathology images. Our model achieves a top-1 accuracy of 79.55\% with a less complex architecture, demonstrating comparable performance to current state-of-the-art models. This research not only advances medical VQA but also opens avenues for practical applications in diagnostic settings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_13081 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Free Form Medical Visual Question Answering in Radiology Narayanan, Abhishek Musthyala, Rushabh Sankar, Rahul Nistala, Anirudh Prasad Singh, Pranav Cirrone, Jacopo Computer Vision and Pattern Recognition Artificial Intelligence Visual Question Answering (VQA) in the medical domain presents a unique, interdisciplinary challenge, combining fields such as Computer Vision, Natural Language Processing, and Knowledge Representation. Despite its importance, research in medical VQA has been scant, only gaining momentum since 2018. Addressing this gap, our research delves into the effective representation of radiology images and the joint learning of multimodal representations, surpassing existing methods. We innovatively augment the SLAKE dataset, enabling our model to respond to a more diverse array of questions, not limited to the immediate content of radiology or pathology images. Our model achieves a top-1 accuracy of 79.55\% with a less complex architecture, demonstrating comparable performance to current state-of-the-art models. This research not only advances medical VQA but also opens avenues for practical applications in diagnostic settings. |
| title | Free Form Medical Visual Question Answering in Radiology |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2401.13081 |