MicroVQA: A Multimodal Reasoning Benchmark for Microscopy-Based Scientific Research
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916654621917184 |
|---|---|
| author | Burgess, James Nirschl, Jeffrey J Bravo-Sánchez, Laura Lozano, Alejandro Gupte, Sanket Rajan Galaz-Montoya, Jesus G. Zhang, Yuhui Su, Yuchang Bhowmik, Disha Coman, Zachary Hasan, Sarina M. Johannesson, Alexandra Leineweber, William D. Nair, Malvika G Yarlagadda, Ridhi Zuraski, Connor Chiu, Wah Cohen, Sarah Hansen, Jan N. Leonetti, Manuel D Liu, Chad Lundberg, Emma Yeung-Levy, Serena |
| author_facet | Burgess, James Nirschl, Jeffrey J Bravo-Sánchez, Laura Lozano, Alejandro Gupte, Sanket Rajan Galaz-Montoya, Jesus G. Zhang, Yuhui Su, Yuchang Bhowmik, Disha Coman, Zachary Hasan, Sarina M. Johannesson, Alexandra Leineweber, William D. Nair, Malvika G Yarlagadda, Ridhi Zuraski, Connor Chiu, Wah Cohen, Sarah Hansen, Jan N. Leonetti, Manuel D Liu, Chad Lundberg, Emma Yeung-Levy, Serena |
| contents | Scientific research demands sophisticated reasoning over multimodal data, a challenge especially prevalent in biology. Despite recent advances in multimodal large language models (MLLMs) for AI-assisted research, existing multimodal reasoning benchmarks only target up to college-level difficulty, while research-level benchmarks emphasize lower-level perception, falling short of the complex multimodal reasoning needed for scientific discovery. To bridge this gap, we introduce MicroVQA, a visual-question answering (VQA) benchmark designed to assess three reasoning capabilities vital in research workflows: expert image understanding, hypothesis generation, and experiment proposal. MicroVQA consists of 1,042 multiple-choice questions (MCQs) curated by biology experts across diverse microscopy modalities, ensuring VQA samples represent real scientific practice. In constructing the benchmark, we find that standard MCQ generation methods induce language shortcuts, motivating a new two-stage pipeline: an optimized LLM prompt structures question-answer pairs into MCQs; then, an agent-based `RefineBot' updates them to remove shortcuts. Benchmarking on state-of-the-art MLLMs reveal a peak performance of 53\%; models with smaller LLMs only slightly underperform top models, suggesting that language-based reasoning is less challenging than multimodal reasoning; and tuning with scientific articles enhances performance. Expert analysis of chain-of-thought responses shows that perception errors are the most frequent, followed by knowledge errors and then overgeneralization errors. These insights highlight the challenges in multimodal scientific reasoning, showing MicroVQA is a valuable resource advancing AI-driven biomedical research. MicroVQA is available at https://huggingface.co/datasets/jmhb/microvqa, and project page at https://jmhb0.github.io/microvqa. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_13399 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MicroVQA: A Multimodal Reasoning Benchmark for Microscopy-Based Scientific Research Burgess, James Nirschl, Jeffrey J Bravo-Sánchez, Laura Lozano, Alejandro Gupte, Sanket Rajan Galaz-Montoya, Jesus G. Zhang, Yuhui Su, Yuchang Bhowmik, Disha Coman, Zachary Hasan, Sarina M. Johannesson, Alexandra Leineweber, William D. Nair, Malvika G Yarlagadda, Ridhi Zuraski, Connor Chiu, Wah Cohen, Sarah Hansen, Jan N. Leonetti, Manuel D Liu, Chad Lundberg, Emma Yeung-Levy, Serena Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language Machine Learning Cell Behavior Scientific research demands sophisticated reasoning over multimodal data, a challenge especially prevalent in biology. Despite recent advances in multimodal large language models (MLLMs) for AI-assisted research, existing multimodal reasoning benchmarks only target up to college-level difficulty, while research-level benchmarks emphasize lower-level perception, falling short of the complex multimodal reasoning needed for scientific discovery. To bridge this gap, we introduce MicroVQA, a visual-question answering (VQA) benchmark designed to assess three reasoning capabilities vital in research workflows: expert image understanding, hypothesis generation, and experiment proposal. MicroVQA consists of 1,042 multiple-choice questions (MCQs) curated by biology experts across diverse microscopy modalities, ensuring VQA samples represent real scientific practice. In constructing the benchmark, we find that standard MCQ generation methods induce language shortcuts, motivating a new two-stage pipeline: an optimized LLM prompt structures question-answer pairs into MCQs; then, an agent-based `RefineBot' updates them to remove shortcuts. Benchmarking on state-of-the-art MLLMs reveal a peak performance of 53\%; models with smaller LLMs only slightly underperform top models, suggesting that language-based reasoning is less challenging than multimodal reasoning; and tuning with scientific articles enhances performance. Expert analysis of chain-of-thought responses shows that perception errors are the most frequent, followed by knowledge errors and then overgeneralization errors. These insights highlight the challenges in multimodal scientific reasoning, showing MicroVQA is a valuable resource advancing AI-driven biomedical research. MicroVQA is available at https://huggingface.co/datasets/jmhb/microvqa, and project page at https://jmhb0.github.io/microvqa. |
| title | MicroVQA: A Multimodal Reasoning Benchmark for Microscopy-Based Scientific Research |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language Machine Learning Cell Behavior |
| url | https://arxiv.org/abs/2503.13399 |