MM-THEBench: Do Reasoning MLLMs Think Reasonably?
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866912862617731072 |
|---|---|
| author | Huang, Zhidian Yao, Zijun Qi, Ji Tu, Shangqing Ma, Junxian Liu, Jinxin Liu, Weichuan Che, Xiaoyin Hou, Lei Li, Juanzi |
| author_facet | Huang, Zhidian Yao, Zijun Qi, Ji Tu, Shangqing Ma, Junxian Liu, Jinxin Liu, Weichuan Che, Xiaoyin Hou, Lei Li, Juanzi |
| contents | Recent advances in multimodal large language models (MLLMs) mark a shift from non-thinking models to post-trained reasoning models capable of solving complex problems through thinking. However, whether such thinking mitigates hallucinations in multimodal perception and reasoning remains unclear. Self-reflective reasoning enhances robustness but introduces additional hallucinations, and subtle perceptual errors still result in incorrect or coincidentally correct answers. Existing benchmarks primarily focus on models before the emergence of reasoning MLLMs, neglecting the internal thinking process and failing to measure the hallucinations that occur during thinking. To address these challenges, we introduce MM-THEBench, a comprehensive benchmark for assessing hallucinations of intermediate CoTs in reasoning MLLMs. MM-THEBench features a fine-grained taxonomy grounded in cognitive dimensions, diverse data with verified reasoning annotations, and a multi-level automated evaluation framework. Extensive experiments on mainstream reasoning MLLMs reveal insights into how thinking affects hallucination and reasoning capability in various multimodal tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_22735 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | MM-THEBench: Do Reasoning MLLMs Think Reasonably? Huang, Zhidian Yao, Zijun Qi, Ji Tu, Shangqing Ma, Junxian Liu, Jinxin Liu, Weichuan Che, Xiaoyin Hou, Lei Li, Juanzi Computation and Language Recent advances in multimodal large language models (MLLMs) mark a shift from non-thinking models to post-trained reasoning models capable of solving complex problems through thinking. However, whether such thinking mitigates hallucinations in multimodal perception and reasoning remains unclear. Self-reflective reasoning enhances robustness but introduces additional hallucinations, and subtle perceptual errors still result in incorrect or coincidentally correct answers. Existing benchmarks primarily focus on models before the emergence of reasoning MLLMs, neglecting the internal thinking process and failing to measure the hallucinations that occur during thinking. To address these challenges, we introduce MM-THEBench, a comprehensive benchmark for assessing hallucinations of intermediate CoTs in reasoning MLLMs. MM-THEBench features a fine-grained taxonomy grounded in cognitive dimensions, diverse data with verified reasoning annotations, and a multi-level automated evaluation framework. Extensive experiments on mainstream reasoning MLLMs reveal insights into how thinking affects hallucination and reasoning capability in various multimodal tasks. |
| title | MM-THEBench: Do Reasoning MLLMs Think Reasonably? |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2601.22735 |