Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2502.11514 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909621735653376 |
|---|---|
| author | Lin, Yujie Wang, Ante Chen, Moye Liu, Jingyao Liu, Hao Su, Jinsong Xiao, Xinyan |
| author_facet | Lin, Yujie Wang, Ante Chen, Moye Liu, Jingyao Liu, Hao Su, Jinsong Xiao, Xinyan |
| contents | Recently, inference-time scaling of chain-of-thought (CoT) has been demonstrated as a promising approach for addressing multi-modal reasoning tasks. While existing studies have predominantly centered on text-based thinking, the integration of both visual and textual modalities within the reasoning process remains unexplored. In this study, we pioneer the exploration of inference-time scaling with multi-modal thought, aiming to bridge this gap. To provide a comprehensive analysis, we systematically investigate popular sampling-based and tree search-based inference-time scaling methods on 10 challenging tasks spanning various domains. Besides, we uniformly adopt a consistency-enhanced verifier to ensure effective guidance for both methods across different thought paradigms. Results show that multi-modal thought promotes better performance against conventional text-only thought, and blending the two types of thought fosters more diverse thinking. Despite these advantages, multi-modal thoughts necessitate higher token consumption for processing richer visual inputs, which raises concerns in practical applications. We hope that our findings on the merits and drawbacks of this research line will inspire future works in the field. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_11514 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Investigating Inference-time Scaling for Chain of Multi-modal Thought: A Preliminary Study Lin, Yujie Wang, Ante Chen, Moye Liu, Jingyao Liu, Hao Su, Jinsong Xiao, Xinyan Computation and Language Recently, inference-time scaling of chain-of-thought (CoT) has been demonstrated as a promising approach for addressing multi-modal reasoning tasks. While existing studies have predominantly centered on text-based thinking, the integration of both visual and textual modalities within the reasoning process remains unexplored. In this study, we pioneer the exploration of inference-time scaling with multi-modal thought, aiming to bridge this gap. To provide a comprehensive analysis, we systematically investigate popular sampling-based and tree search-based inference-time scaling methods on 10 challenging tasks spanning various domains. Besides, we uniformly adopt a consistency-enhanced verifier to ensure effective guidance for both methods across different thought paradigms. Results show that multi-modal thought promotes better performance against conventional text-only thought, and blending the two types of thought fosters more diverse thinking. Despite these advantages, multi-modal thoughts necessitate higher token consumption for processing richer visual inputs, which raises concerns in practical applications. We hope that our findings on the merits and drawbacks of this research line will inspire future works in the field. |
| title | Investigating Inference-time Scaling for Chain of Multi-modal Thought: A Preliminary Study |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2502.11514 |