RationalVLA: A Rational Vision-Language-Action Model with Dual System
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908407022223360 |
|---|---|
| author | Song, Wenxuan Chen, Jiayi Li, Wenxue He, Xu Zhao, Han Cui, Can Su, Pengxiang Ding Shiyan Tang, Feilong Cheng, Xuelian Wang, Donglin Ge, Zongyuan Zheng, Xinhu Liu, Zhe Wang, Hesheng Li, Haoang |
| author_facet | Song, Wenxuan Chen, Jiayi Li, Wenxue He, Xu Zhao, Han Cui, Can Su, Pengxiang Ding Shiyan Tang, Feilong Cheng, Xuelian Wang, Donglin Ge, Zongyuan Zheng, Xinhu Liu, Zhe Wang, Hesheng Li, Haoang |
| contents | A fundamental requirement for real-world robotic deployment is the ability to understand and respond to natural language instructions. Existing language-conditioned manipulation tasks typically assume that instructions are perfectly aligned with the environment. This assumption limits robustness and generalization in realistic scenarios where instructions may be ambiguous, irrelevant, or infeasible. To address this problem, we introduce RAtional MAnipulation (RAMA), a new benchmark that challenges models with both unseen executable instructions and defective ones that should be rejected. In RAMA, we construct a dataset with over 14,000 samples, including diverse defective instructions spanning six dimensions: visual, physical, semantic, motion, safety, and out-of-context. We further propose the Rational Vision-Language-Action model (RationalVLA). It is a dual system for robotic arms that integrates the high-level vision-language model with the low-level manipulation policy by introducing learnable latent space embeddings. This design enables RationalVLA to reason over instructions, reject infeasible commands, and execute manipulation effectively. Experiments demonstrate that RationalVLA outperforms state-of-the-art baselines on RAMA by a 14.5% higher success rate and 0.94 average task length, while maintaining competitive performance on standard manipulation tasks. Real-world trials further validate its effectiveness and robustness in practical applications. Our project page is https://irpn-eai.github.io/RationalVLA. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_10826 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | RationalVLA: A Rational Vision-Language-Action Model with Dual System Song, Wenxuan Chen, Jiayi Li, Wenxue He, Xu Zhao, Han Cui, Can Su, Pengxiang Ding Shiyan Tang, Feilong Cheng, Xuelian Wang, Donglin Ge, Zongyuan Zheng, Xinhu Liu, Zhe Wang, Hesheng Li, Haoang Robotics A fundamental requirement for real-world robotic deployment is the ability to understand and respond to natural language instructions. Existing language-conditioned manipulation tasks typically assume that instructions are perfectly aligned with the environment. This assumption limits robustness and generalization in realistic scenarios where instructions may be ambiguous, irrelevant, or infeasible. To address this problem, we introduce RAtional MAnipulation (RAMA), a new benchmark that challenges models with both unseen executable instructions and defective ones that should be rejected. In RAMA, we construct a dataset with over 14,000 samples, including diverse defective instructions spanning six dimensions: visual, physical, semantic, motion, safety, and out-of-context. We further propose the Rational Vision-Language-Action model (RationalVLA). It is a dual system for robotic arms that integrates the high-level vision-language model with the low-level manipulation policy by introducing learnable latent space embeddings. This design enables RationalVLA to reason over instructions, reject infeasible commands, and execute manipulation effectively. Experiments demonstrate that RationalVLA outperforms state-of-the-art baselines on RAMA by a 14.5% higher success rate and 0.94 average task length, while maintaining competitive performance on standard manipulation tasks. Real-world trials further validate its effectiveness and robustness in practical applications. Our project page is https://irpn-eai.github.io/RationalVLA. |
| title | RationalVLA: A Rational Vision-Language-Action Model with Dual System |
| topic | Robotics |
| url | https://arxiv.org/abs/2506.10826 |