MedGEN-Bench: Contextually entangled benchmark for open-ended multimodal medical generation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908661650030592 |
|---|---|
| author | Yang, Junjie Yan, Yuhao Wu, Gang Wang, Yuxuan Liang, Ruoyu Jiang, Xinjie Wan, Xiang Fan, Fenglei Zhang, Yongquan Qin, Feiwei Wang, Changmiao |
| author_facet | Yang, Junjie Yan, Yuhao Wu, Gang Wang, Yuxuan Liang, Ruoyu Jiang, Xinjie Wan, Xiang Fan, Fenglei Zhang, Yongquan Qin, Feiwei Wang, Changmiao |
| contents | As Vision-Language Models (VLMs) increasingly gain traction in medical applications, clinicians are progressively expecting AI systems not only to generate textual diagnoses but also to produce corresponding medical images that integrate seamlessly into authentic clinical workflows. Despite the growing interest, existing medical visual benchmarks present notable limitations. They often rely on ambiguous queries that lack sufficient relevance to image content, oversimplify complex diagnostic reasoning into closed-ended shortcuts, and adopt a text-centric evaluation paradigm that overlooks the importance of image generation capabilities. To address these challenges, we introduce MedGEN-Bench, a comprehensive multimodal benchmark designed to advance medical AI research. MedGEN-Bench comprises 6,422 expert-validated image-text pairs spanning six imaging modalities, 16 clinical tasks, and 28 subtasks. It is structured into three distinct formats: Visual Question Answering, Image Editing, and Contextual Multimodal Generation. What sets MedGEN-Bench apart is its focus on contextually intertwined instructions that necessitate sophisticated cross-modal reasoning and open-ended generative outputs, moving beyond the constraints of multiple-choice formats. To evaluate the performance of existing systems, we employ a novel three-tier assessment framework that integrates pixel-level metrics, semantic text analysis, and expert-guided clinical relevance scoring. Using this framework, we systematically assess 10 compositional frameworks, 3 unified models, and 5 VLMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_13135 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MedGEN-Bench: Contextually entangled benchmark for open-ended multimodal medical generation Yang, Junjie Yan, Yuhao Wu, Gang Wang, Yuxuan Liang, Ruoyu Jiang, Xinjie Wan, Xiang Fan, Fenglei Zhang, Yongquan Qin, Feiwei Wang, Changmiao Computer Vision and Pattern Recognition As Vision-Language Models (VLMs) increasingly gain traction in medical applications, clinicians are progressively expecting AI systems not only to generate textual diagnoses but also to produce corresponding medical images that integrate seamlessly into authentic clinical workflows. Despite the growing interest, existing medical visual benchmarks present notable limitations. They often rely on ambiguous queries that lack sufficient relevance to image content, oversimplify complex diagnostic reasoning into closed-ended shortcuts, and adopt a text-centric evaluation paradigm that overlooks the importance of image generation capabilities. To address these challenges, we introduce MedGEN-Bench, a comprehensive multimodal benchmark designed to advance medical AI research. MedGEN-Bench comprises 6,422 expert-validated image-text pairs spanning six imaging modalities, 16 clinical tasks, and 28 subtasks. It is structured into three distinct formats: Visual Question Answering, Image Editing, and Contextual Multimodal Generation. What sets MedGEN-Bench apart is its focus on contextually intertwined instructions that necessitate sophisticated cross-modal reasoning and open-ended generative outputs, moving beyond the constraints of multiple-choice formats. To evaluate the performance of existing systems, we employ a novel three-tier assessment framework that integrates pixel-level metrics, semantic text analysis, and expert-guided clinical relevance scoring. Using this framework, we systematically assess 10 compositional frameworks, 3 unified models, and 5 VLMs. |
| title | MedGEN-Bench: Contextually entangled benchmark for open-ended multimodal medical generation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2511.13135 |