Decoding Scientific Experimental Images: The SPUR Benchmark for Perception, Understanding, and Reasoning
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913162227351552 |
|---|---|
| author | Ding, Junpeng Tang, Zichen E, Haihong Ji, Mengyuan Liu, Yang Tian, Haolin Sun, Haiyang Sun, Pengqi Xu, Yang Liu, Yichen Gao, Haocheng Xi, Zijie Jiang, Ruomeng Zhao, Peizhi Li, Rongjin Li, Yuanze Liu, Jiacheng Yang, Zhongjun Chen, Jintong Lin, Siying |
| author_facet | Ding, Junpeng Tang, Zichen E, Haihong Ji, Mengyuan Liu, Yang Tian, Haolin Sun, Haiyang Sun, Pengqi Xu, Yang Liu, Yichen Gao, Haocheng Xi, Zijie Jiang, Ruomeng Zhao, Peizhi Li, Rongjin Li, Yuanze Liu, Jiacheng Yang, Zhongjun Chen, Jintong Lin, Siying |
| contents | We introduce SPUR, a comprehensive benchmark for scientific experimental image perception, understanding, and reasoning, comprising 4,264 question-answering (QA) pairs derived from 1,084 expert-curated images. SPUR features three key innovations: (1) Panel-Level Fine-Grained Perception: evaluating the visual perception of multimodal large language models (MLLMs) across three dimensions (numerical, morphological, and information localization) on six fine-grained panel types; (2) Cross-Panel Relation Understanding: utilizing complex images with an average of 14.3 panels per sample to evaluate MLLMs' ability to decipher intricate cross-panel relations; (3) Expert-Level Reasoning: assessment of qualitative and quantitative reasoning across five experimental paradigms to determine if models can infer conclusions from evidence as human experts do. Comprehensive evaluation of 20 MLLMs and four multimodal Chain-of-Thought (MCoT) methods reveals that current models fall significantly short of the expert-level requirements for scientific image interpretation, underscoring a critical bottleneck in AI for Science (AI4S) research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_27604 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Decoding Scientific Experimental Images: The SPUR Benchmark for Perception, Understanding, and Reasoning Ding, Junpeng Tang, Zichen E, Haihong Ji, Mengyuan Liu, Yang Tian, Haolin Sun, Haiyang Sun, Pengqi Xu, Yang Liu, Yichen Gao, Haocheng Xi, Zijie Jiang, Ruomeng Zhao, Peizhi Li, Rongjin Li, Yuanze Liu, Jiacheng Yang, Zhongjun Chen, Jintong Lin, Siying Computer Vision and Pattern Recognition Computational Engineering, Finance, and Science We introduce SPUR, a comprehensive benchmark for scientific experimental image perception, understanding, and reasoning, comprising 4,264 question-answering (QA) pairs derived from 1,084 expert-curated images. SPUR features three key innovations: (1) Panel-Level Fine-Grained Perception: evaluating the visual perception of multimodal large language models (MLLMs) across three dimensions (numerical, morphological, and information localization) on six fine-grained panel types; (2) Cross-Panel Relation Understanding: utilizing complex images with an average of 14.3 panels per sample to evaluate MLLMs' ability to decipher intricate cross-panel relations; (3) Expert-Level Reasoning: assessment of qualitative and quantitative reasoning across five experimental paradigms to determine if models can infer conclusions from evidence as human experts do. Comprehensive evaluation of 20 MLLMs and four multimodal Chain-of-Thought (MCoT) methods reveals that current models fall significantly short of the expert-level requirements for scientific image interpretation, underscoring a critical bottleneck in AI for Science (AI4S) research. |
| title | Decoding Scientific Experimental Images: The SPUR Benchmark for Perception, Understanding, and Reasoning |
| topic | Computer Vision and Pattern Recognition Computational Engineering, Finance, and Science |
| url | https://arxiv.org/abs/2604.27604 |