ForceVLA: Enhancing VLA Models with a Force-aware MoE for Contact-rich Manipulation
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866915501220823040 |
|---|---|
| author | Yu, Jiawen Liu, Hairuo Yu, Qiaojun Ren, Jieji Hao, Ce Ding, Haitong Huang, Guangyu Huang, Guofan Song, Yan Cai, Panpan Lu, Cewu Zhang, Wenqiang |
| author_facet | Yu, Jiawen Liu, Hairuo Yu, Qiaojun Ren, Jieji Hao, Ce Ding, Haitong Huang, Guangyu Huang, Guofan Song, Yan Cai, Panpan Lu, Cewu Zhang, Wenqiang |
| contents | Vision-Language-Action (VLA) models have advanced general-purpose robotic manipulation by leveraging pretrained visual and linguistic representations. However, they struggle with contact-rich tasks that require fine-grained control involving force, especially under visual occlusion or dynamic uncertainty. To address these limitations, we propose ForceVLA, a novel end-to-end manipulation framework that treats external force sensing as a first-class modality within VLA systems. ForceVLA introduces FVLMoE, a force-aware Mixture-of-Experts fusion module that dynamically integrates pretrained visual-language embeddings with real-time 6-axis force feedback during action decoding. This enables context-aware routing across modality-specific experts, enhancing the robot's ability to adapt to subtle contact dynamics. We also introduce \textbf{ForceVLA-Data}, a new dataset comprising synchronized vision, proprioception, and force-torque signals across five contact-rich manipulation tasks. ForceVLA improves average task success by 23.2% over strong pi_0-based baselines, achieving up to 80% success in tasks such as plug insertion. Our approach highlights the importance of multimodal integration for dexterous manipulation and sets a new benchmark for physically intelligent robotic control. Code and data will be released at https://sites.google.com/view/forcevla2025. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_22159 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ForceVLA: Enhancing VLA Models with a Force-aware MoE for Contact-rich Manipulation Yu, Jiawen Liu, Hairuo Yu, Qiaojun Ren, Jieji Hao, Ce Ding, Haitong Huang, Guangyu Huang, Guofan Song, Yan Cai, Panpan Lu, Cewu Zhang, Wenqiang Robotics Computer Vision and Pattern Recognition Vision-Language-Action (VLA) models have advanced general-purpose robotic manipulation by leveraging pretrained visual and linguistic representations. However, they struggle with contact-rich tasks that require fine-grained control involving force, especially under visual occlusion or dynamic uncertainty. To address these limitations, we propose ForceVLA, a novel end-to-end manipulation framework that treats external force sensing as a first-class modality within VLA systems. ForceVLA introduces FVLMoE, a force-aware Mixture-of-Experts fusion module that dynamically integrates pretrained visual-language embeddings with real-time 6-axis force feedback during action decoding. This enables context-aware routing across modality-specific experts, enhancing the robot's ability to adapt to subtle contact dynamics. We also introduce \textbf{ForceVLA-Data}, a new dataset comprising synchronized vision, proprioception, and force-torque signals across five contact-rich manipulation tasks. ForceVLA improves average task success by 23.2% over strong pi_0-based baselines, achieving up to 80% success in tasks such as plug insertion. Our approach highlights the importance of multimodal integration for dexterous manipulation and sets a new benchmark for physically intelligent robotic control. Code and data will be released at https://sites.google.com/view/forcevla2025. |
| title | ForceVLA: Enhancing VLA Models with a Force-aware MoE for Contact-rich Manipulation |
| topic | Robotics Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2505.22159 |