Integrating Chain-of-Thought for Multimodal Alignment: A Study on 3D Vision-Language Learning
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910876574941184 |
|---|---|
| author | Chen, Yanjun Sun, Yirong Chen, Xinghao Wang, Jian Shen, Xiaoyu Li, Wenjie Zhang, Wei |
| author_facet | Chen, Yanjun Sun, Yirong Chen, Xinghao Wang, Jian Shen, Xiaoyu Li, Wenjie Zhang, Wei |
| contents | Chain-of-Thought (CoT) reasoning has proven effective in natural language tasks but remains underexplored in multimodal alignment. This study investigates its integration into 3D vision-language learning by embedding structured reasoning into alignment training. We introduce the 3D-CoT Benchmark, a dataset with hierarchical CoT annotations covering shape recognition, functional inference, and causal reasoning. Through controlled experiments, we compare CoT-structured and standard textual annotations across large reasoning models (LRMs) and large language models (LLMs). Our evaluation employs a dual-layer framework assessing both intermediate reasoning and final inference quality. Extensive experiments demonstrate that CoT significantly improves 3D semantic grounding, with LRMs leveraging CoT more effectively than LLMs. Furthermore, we highlight that annotation structure influences performance-explicit reasoning markers aid LLMs, while unmarked CoT better aligns with LRM inference patterns. Our analyses suggest that CoT is crucial for enhancing multimodal reasoning, with implications beyond 3D tasks. The dataset will be publicly available at https://huggingface.co/datasets/Battam/3D-CoT |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_06232 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Integrating Chain-of-Thought for Multimodal Alignment: A Study on 3D Vision-Language Learning Chen, Yanjun Sun, Yirong Chen, Xinghao Wang, Jian Shen, Xiaoyu Li, Wenjie Zhang, Wei Computation and Language Computer Vision and Pattern Recognition Chain-of-Thought (CoT) reasoning has proven effective in natural language tasks but remains underexplored in multimodal alignment. This study investigates its integration into 3D vision-language learning by embedding structured reasoning into alignment training. We introduce the 3D-CoT Benchmark, a dataset with hierarchical CoT annotations covering shape recognition, functional inference, and causal reasoning. Through controlled experiments, we compare CoT-structured and standard textual annotations across large reasoning models (LRMs) and large language models (LLMs). Our evaluation employs a dual-layer framework assessing both intermediate reasoning and final inference quality. Extensive experiments demonstrate that CoT significantly improves 3D semantic grounding, with LRMs leveraging CoT more effectively than LLMs. Furthermore, we highlight that annotation structure influences performance-explicit reasoning markers aid LLMs, while unmarked CoT better aligns with LRM inference patterns. Our analyses suggest that CoT is crucial for enhancing multimodal reasoning, with implications beyond 3D tasks. The dataset will be publicly available at https://huggingface.co/datasets/Battam/3D-CoT |
| title | Integrating Chain-of-Thought for Multimodal Alignment: A Study on 3D Vision-Language Learning |
| topic | Computation and Language Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2503.06232 |