MedFrameQA: A Multi-Image Medical VQA Benchmark for Clinical Reasoning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918320214638592 |
|---|---|
| author | Yu, Suhao Wang, Haojin Wu, Juncheng Luo, Luyang Wang, Jingshen Xie, Cihang Rajpurkar, Pranav Yang, Carl Yang, Yang Wang, Kang Yu, Yannan Zhou, Yuyin |
| author_facet | Yu, Suhao Wang, Haojin Wu, Juncheng Luo, Luyang Wang, Jingshen Xie, Cihang Rajpurkar, Pranav Yang, Carl Yang, Yang Wang, Kang Yu, Yannan Zhou, Yuyin |
| contents | Real-world clinical practice demands multi-image comparative reasoning, yet current medical benchmarks remain limited to single-frame interpretation. We present MedFrameQA, the first benchmark explicitly designed to test multi-image medical VQA through educationally-validated diagnostic sequences. To construct this dataset, we develop a scalable pipeline that leverages narrative transcripts from medical education videos to align visual frames with textual concepts, automatically producing 2,851 high-quality multi-image VQA pairs with explicit, transcript-grounded reasoning chains. Our evaluation of 11 advanced MLLMs (including reasoning models) exposes severe deficiencies in multi-image synthesis, where accuracies mostly fall below 50% and exhibit instability across varying image counts. Error analysis demonstrates that models often treat images as isolated instances, failing to track pathological progression or cross-reference anatomical shifts. MedFrameQA provides a rigorous standard for evaluating the next generation of MLLMs in handling complex, temporally grounded medical narratives. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_16964 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MedFrameQA: A Multi-Image Medical VQA Benchmark for Clinical Reasoning Yu, Suhao Wang, Haojin Wu, Juncheng Luo, Luyang Wang, Jingshen Xie, Cihang Rajpurkar, Pranav Yang, Carl Yang, Yang Wang, Kang Yu, Yannan Zhou, Yuyin Computer Vision and Pattern Recognition Computation and Language Real-world clinical practice demands multi-image comparative reasoning, yet current medical benchmarks remain limited to single-frame interpretation. We present MedFrameQA, the first benchmark explicitly designed to test multi-image medical VQA through educationally-validated diagnostic sequences. To construct this dataset, we develop a scalable pipeline that leverages narrative transcripts from medical education videos to align visual frames with textual concepts, automatically producing 2,851 high-quality multi-image VQA pairs with explicit, transcript-grounded reasoning chains. Our evaluation of 11 advanced MLLMs (including reasoning models) exposes severe deficiencies in multi-image synthesis, where accuracies mostly fall below 50% and exhibit instability across varying image counts. Error analysis demonstrates that models often treat images as isolated instances, failing to track pathological progression or cross-reference anatomical shifts. MedFrameQA provides a rigorous standard for evaluating the next generation of MLLMs in handling complex, temporally grounded medical narratives. |
| title | MedFrameQA: A Multi-Image Medical VQA Benchmark for Clinical Reasoning |
| topic | Computer Vision and Pattern Recognition Computation and Language |
| url | https://arxiv.org/abs/2505.16964 |