Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2510.12267 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917011823525888 |
|---|---|
| author | Zhang, Chenghanyu Li, Zekun Li, Peipei Cui, Xing Xia, Shuhan Yan, Weixiang Zhang, Yiqiao Zhuang, Qianyu |
| author_facet | Zhang, Chenghanyu Li, Zekun Li, Peipei Cui, Xing Xia, Shuhan Yan, Weixiang Zhang, Yiqiao Zhuang, Qianyu |
| contents | With the increasing integration of Multimodal Large Language Models (MLLMs) into the medical field, comprehensive evaluation of their performance in various medical domains becomes critical. However, existing benchmarks primarily assess general medical tasks, inadequately capturing performance in nuanced areas like the spine, which relies heavily on visual input. To address this, we introduce SpineBench, a comprehensive Visual Question Answering (VQA) benchmark designed for fine-grained analysis and evaluation of MLLMs in the spinal domain. SpineBench comprises 64,878 QA pairs from 40,263 spine images, covering 11 spinal diseases through two critical clinical tasks: spinal disease diagnosis and spinal lesion localization, both in multiple-choice format. SpineBench is built by integrating and standardizing image-label pairs from open-source spinal disease datasets, and samples challenging hard negative options for each VQA pair based on visual similarity (similar but not the same disease), simulating real-world challenging scenarios. We evaluate 12 leading MLLMs on SpineBench. The results reveal that these models exhibit poor performance in spinal tasks, highlighting limitations of current MLLM in the spine domain and guiding future improvements in spinal medicine applications. SpineBench is publicly available at https://zhangchenghanyu.github.io/SpineBench.github.io/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_12267 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | SpineBench: Benchmarking Multimodal LLMs for Spinal Pathology Analysis Zhang, Chenghanyu Li, Zekun Li, Peipei Cui, Xing Xia, Shuhan Yan, Weixiang Zhang, Yiqiao Zhuang, Qianyu Computer Vision and Pattern Recognition With the increasing integration of Multimodal Large Language Models (MLLMs) into the medical field, comprehensive evaluation of their performance in various medical domains becomes critical. However, existing benchmarks primarily assess general medical tasks, inadequately capturing performance in nuanced areas like the spine, which relies heavily on visual input. To address this, we introduce SpineBench, a comprehensive Visual Question Answering (VQA) benchmark designed for fine-grained analysis and evaluation of MLLMs in the spinal domain. SpineBench comprises 64,878 QA pairs from 40,263 spine images, covering 11 spinal diseases through two critical clinical tasks: spinal disease diagnosis and spinal lesion localization, both in multiple-choice format. SpineBench is built by integrating and standardizing image-label pairs from open-source spinal disease datasets, and samples challenging hard negative options for each VQA pair based on visual similarity (similar but not the same disease), simulating real-world challenging scenarios. We evaluate 12 leading MLLMs on SpineBench. The results reveal that these models exhibit poor performance in spinal tasks, highlighting limitations of current MLLM in the spine domain and guiding future improvements in spinal medicine applications. SpineBench is publicly available at https://zhangchenghanyu.github.io/SpineBench.github.io/. |
| title | SpineBench: Benchmarking Multimodal LLMs for Spinal Pathology Analysis |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2510.12267 |