When Slower Isn't Truer: Inverse Scaling Law of Truthfulness in Multimodal Reasoning
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911596891078656 |
|---|---|
| author | Fang, Sitong Cao, Wenjing Li, Jiahao Wang, Xuyao Dai, Juntao Chan, Chi-Min Han, Sirui Guo, Yike Yang, Yaodong Ji, Jiaming |
| author_facet | Fang, Sitong Cao, Wenjing Li, Jiahao Wang, Xuyao Dai, Juntao Chan, Chi-Min Han, Sirui Guo, Yike Yang, Yaodong Ji, Jiaming |
| contents | Reasoning models have attracted increasing attention for their ability to tackle complex tasks, embodying the System II (slow thinking) paradigm in contrast to System I (fast, intuitive responses). Yet a key question remains: Does slower reasoning necessarily lead to more truthful answers? Our findings suggest otherwise. We conduct the first systematic study of the inverse scaling law in slow-thinking paradigms for multimodal reasoning. We find that when confronted with incomplete or misleading visual inputs, slow-thinking models are more prone to fabricating plausible yet false details to justify untruthful reasoning. To analyze this behavior, we construct a 5,000-sample hierarchical prompt dataset annotated by 50 human participants. The prompts progressively increase in complexity, revealing a consistent pattern: slower reasoning models tend to follow depth-first search (DFS) thinking, persistently exploring flawed premises, while faster chat models favor breadth-first search (BFS) inference, showing greater caution under uncertainty. These findings reveal a critical vulnerability of reasoning models: while effective in structured domains such as math, their DFS-style reasoning becomes fragile when confronted with ambiguous, multimodal inputs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_20214 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | When Slower Isn't Truer: Inverse Scaling Law of Truthfulness in Multimodal Reasoning Fang, Sitong Cao, Wenjing Li, Jiahao Wang, Xuyao Dai, Juntao Chan, Chi-Min Han, Sirui Guo, Yike Yang, Yaodong Ji, Jiaming Artificial Intelligence Reasoning models have attracted increasing attention for their ability to tackle complex tasks, embodying the System II (slow thinking) paradigm in contrast to System I (fast, intuitive responses). Yet a key question remains: Does slower reasoning necessarily lead to more truthful answers? Our findings suggest otherwise. We conduct the first systematic study of the inverse scaling law in slow-thinking paradigms for multimodal reasoning. We find that when confronted with incomplete or misleading visual inputs, slow-thinking models are more prone to fabricating plausible yet false details to justify untruthful reasoning. To analyze this behavior, we construct a 5,000-sample hierarchical prompt dataset annotated by 50 human participants. The prompts progressively increase in complexity, revealing a consistent pattern: slower reasoning models tend to follow depth-first search (DFS) thinking, persistently exploring flawed premises, while faster chat models favor breadth-first search (BFS) inference, showing greater caution under uncertainty. These findings reveal a critical vulnerability of reasoning models: while effective in structured domains such as math, their DFS-style reasoning becomes fragile when confronted with ambiguous, multimodal inputs. |
| title | When Slower Isn't Truer: Inverse Scaling Law of Truthfulness in Multimodal Reasoning |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2505.20214 |