MM-CoT:A Benchmark for Probing Visual Chain-of-Thought Reasoning in Multimodal Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918239175442432 |
|---|---|
| author | Zhang, Jusheng Cai, Kaitong Guo, Xiaoyang Liu, Sidi Lv, Qinhan Chen, Ruiqi Yang, Jing Fan, Yijia Sun, Xiaofei Wang, Jian Chen, Ziliang Lin, Liang Wang, Keze |
| author_facet | Zhang, Jusheng Cai, Kaitong Guo, Xiaoyang Liu, Sidi Lv, Qinhan Chen, Ruiqi Yang, Jing Fan, Yijia Sun, Xiaofei Wang, Jian Chen, Ziliang Lin, Liang Wang, Keze |
| contents | The ability to perform Chain-of-Thought (CoT) reasoning marks a major milestone for multimodal models (MMs), enabling them to solve complex visual reasoning problems. Yet a critical question remains: is such reasoning genuinely grounded in visual evidence and logically coherent? Existing benchmarks emphasize generation but neglect verification, i.e., the capacity to assess whether a reasoning chain is both visually consistent and logically valid. To fill this gap, we introduce MM-CoT, a diagnostic benchmark specifically designed to probe the visual grounding and logical coherence of CoT reasoning in MMs. Instead of generating free-form explanations, models must select the sole event chain that satisfies two orthogonal constraints: (i) visual consistency, ensuring all steps are anchored in observable evidence, and (ii) logical coherence, ensuring causal and commonsense validity. Adversarial distractors are engineered to violate one of these constraints, exposing distinct reasoning failures. We evaluate leading vision-language models on MM-CoT and find that even the most advanced systems struggle, revealing a sharp discrepancy between generative fluency and true reasoning fidelity. MM-CoT shows low correlation with existing benchmarks, confirming that it measures a unique combination of visual grounding and logical reasoning. This benchmark provides a foundation for developing future models that reason not just plausibly, but faithfully and coherently within the visual world. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_08228 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MM-CoT:A Benchmark for Probing Visual Chain-of-Thought Reasoning in Multimodal Models Zhang, Jusheng Cai, Kaitong Guo, Xiaoyang Liu, Sidi Lv, Qinhan Chen, Ruiqi Yang, Jing Fan, Yijia Sun, Xiaofei Wang, Jian Chen, Ziliang Lin, Liang Wang, Keze Computer Vision and Pattern Recognition Artificial Intelligence The ability to perform Chain-of-Thought (CoT) reasoning marks a major milestone for multimodal models (MMs), enabling them to solve complex visual reasoning problems. Yet a critical question remains: is such reasoning genuinely grounded in visual evidence and logically coherent? Existing benchmarks emphasize generation but neglect verification, i.e., the capacity to assess whether a reasoning chain is both visually consistent and logically valid. To fill this gap, we introduce MM-CoT, a diagnostic benchmark specifically designed to probe the visual grounding and logical coherence of CoT reasoning in MMs. Instead of generating free-form explanations, models must select the sole event chain that satisfies two orthogonal constraints: (i) visual consistency, ensuring all steps are anchored in observable evidence, and (ii) logical coherence, ensuring causal and commonsense validity. Adversarial distractors are engineered to violate one of these constraints, exposing distinct reasoning failures. We evaluate leading vision-language models on MM-CoT and find that even the most advanced systems struggle, revealing a sharp discrepancy between generative fluency and true reasoning fidelity. MM-CoT shows low correlation with existing benchmarks, confirming that it measures a unique combination of visual grounding and logical reasoning. This benchmark provides a foundation for developing future models that reason not just plausibly, but faithfully and coherently within the visual world. |
| title | MM-CoT:A Benchmark for Probing Visual Chain-of-Thought Reasoning in Multimodal Models |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2512.08228 |