TacVLA: Contact-Aware Tactile Fusion for Robust Vision-Language-Action Manipulation
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911542506684416 |
|---|---|
| author | Zhang, Kaidi Zhang, Heng Xu, Zhengtong Zhang, Zhiyuan Prince, Md Rakibul Islam Li, Xiang Han, Xiaojing Zhou, Yuhao Ajoudani, Arash She, Yu |
| author_facet | Zhang, Kaidi Zhang, Heng Xu, Zhengtong Zhang, Zhiyuan Prince, Md Rakibul Islam Li, Xiang Han, Xiaojing Zhou, Yuhao Ajoudani, Arash She, Yu |
| contents | Vision-Language-Action (VLA) models have demonstrated significant advantages in robotic manipulation. However, their reliance on vision and language often leads to suboptimal performance in tasks involving visual occlusion, fine-grained manipulation, and physical contact. To address these challenges, we propose TacVLA, a fine-tuned VLA model by incorporating tactile modalities into the transformer-based policy to enhance fine-grained manipulation capabilities. Specifically, we introduce a contact-aware gating mechanism that selectively activates tactile tokens only when contact is detected, enabling adaptive multimodal fusion while avoiding irrelevant tactile interference. The fused visual, language, and tactile tokens are jointly processed within the transformer architecture to strengthen cross-modal grounding during contact-rich interaction. Extensive experiments on constraint-locked disassembly, in-box picking and robustness evaluations demonstrate that our model outperforms baselines, improving the performance by averaging 20% success rate in disassembly, 60% in in-box picking and 2.1x improvement in scenarios with visual occlusion. Videos are available at https://sites.google.com/view/tacvla and code will be released. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_12665 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | TacVLA: Contact-Aware Tactile Fusion for Robust Vision-Language-Action Manipulation Zhang, Kaidi Zhang, Heng Xu, Zhengtong Zhang, Zhiyuan Prince, Md Rakibul Islam Li, Xiang Han, Xiaojing Zhou, Yuhao Ajoudani, Arash She, Yu Robotics Vision-Language-Action (VLA) models have demonstrated significant advantages in robotic manipulation. However, their reliance on vision and language often leads to suboptimal performance in tasks involving visual occlusion, fine-grained manipulation, and physical contact. To address these challenges, we propose TacVLA, a fine-tuned VLA model by incorporating tactile modalities into the transformer-based policy to enhance fine-grained manipulation capabilities. Specifically, we introduce a contact-aware gating mechanism that selectively activates tactile tokens only when contact is detected, enabling adaptive multimodal fusion while avoiding irrelevant tactile interference. The fused visual, language, and tactile tokens are jointly processed within the transformer architecture to strengthen cross-modal grounding during contact-rich interaction. Extensive experiments on constraint-locked disassembly, in-box picking and robustness evaluations demonstrate that our model outperforms baselines, improving the performance by averaging 20% success rate in disassembly, 60% in in-box picking and 2.1x improvement in scenarios with visual occlusion. Videos are available at https://sites.google.com/view/tacvla and code will be released. |
| title | TacVLA: Contact-Aware Tactile Fusion for Robust Vision-Language-Action Manipulation |
| topic | Robotics |
| url | https://arxiv.org/abs/2603.12665 |