CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912298739695616 |
|---|---|
| author | Zhao, Qingqing Lu, Yao Kim, Moo Jin Fu, Zipeng Zhang, Zhuoyang Wu, Yecheng Li, Zhaoshuo Ma, Qianli Han, Song Finn, Chelsea Handa, Ankur Liu, Ming-Yu Xiang, Donglai Wetzstein, Gordon Lin, Tsung-Yi |
| author_facet | Zhao, Qingqing Lu, Yao Kim, Moo Jin Fu, Zipeng Zhang, Zhuoyang Wu, Yecheng Li, Zhaoshuo Ma, Qianli Han, Song Finn, Chelsea Handa, Ankur Liu, Ming-Yu Xiang, Donglai Wetzstein, Gordon Lin, Tsung-Yi |
| contents | Vision-language-action models (VLAs) have shown potential in leveraging pretrained vision-language models and diverse robot demonstrations for learning generalizable sensorimotor control. While this paradigm effectively utilizes large-scale data from both robotic and non-robotic sources, current VLAs primarily focus on direct input--output mappings, lacking the intermediate reasoning steps crucial for complex manipulation tasks. As a result, existing VLAs lack temporal planning or reasoning capabilities. In this paper, we introduce a method that incorporates explicit visual chain-of-thought (CoT) reasoning into vision-language-action models (VLAs) by predicting future image frames autoregressively as visual goals before generating a short action sequence to achieve these goals. We introduce CoT-VLA, a state-of-the-art 7B VLA that can understand and generate visual and action tokens. Our experimental results demonstrate that CoT-VLA achieves strong performance, outperforming the state-of-the-art VLA model by 17% in real-world manipulation tasks and 6% in simulation benchmarks. Project website: https://cot-vla.github.io/ |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_22020 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models Zhao, Qingqing Lu, Yao Kim, Moo Jin Fu, Zipeng Zhang, Zhuoyang Wu, Yecheng Li, Zhaoshuo Ma, Qianli Han, Song Finn, Chelsea Handa, Ankur Liu, Ming-Yu Xiang, Donglai Wetzstein, Gordon Lin, Tsung-Yi Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning Robotics Vision-language-action models (VLAs) have shown potential in leveraging pretrained vision-language models and diverse robot demonstrations for learning generalizable sensorimotor control. While this paradigm effectively utilizes large-scale data from both robotic and non-robotic sources, current VLAs primarily focus on direct input--output mappings, lacking the intermediate reasoning steps crucial for complex manipulation tasks. As a result, existing VLAs lack temporal planning or reasoning capabilities. In this paper, we introduce a method that incorporates explicit visual chain-of-thought (CoT) reasoning into vision-language-action models (VLAs) by predicting future image frames autoregressively as visual goals before generating a short action sequence to achieve these goals. We introduce CoT-VLA, a state-of-the-art 7B VLA that can understand and generate visual and action tokens. Our experimental results demonstrate that CoT-VLA achieves strong performance, outperforming the state-of-the-art VLA model by 17% in real-world manipulation tasks and 6% in simulation benchmarks. Project website: https://cot-vla.github.io/ |
| title | CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning Robotics |
| url | https://arxiv.org/abs/2503.22020 |